ML20195G547
| ML20195G547 | |
| Person / Time | |
|---|---|
| Issue date: | 05/08/1998 |
| From: | Miraglia F NRC (Affiliation Not Assigned) |
| To: | Martin T Committee To Review Generic Requirements |
| Shared Package | |
| ML20195E260 | List: |
| References | |
| FRN-64FR19868, RULE-PR-55 AF62-2-004, AF62-2-4, NUDOCS 9906160100 | |
| Download: ML20195G547 (400) | |
Text
{{#Wiki_filter:p
/*% % UNITED STATES ARez-2 ,
g f j NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 205500001 PR
. . . . . ,o May 8, 1998 MEMORANDUM TO: Thomas Martin, Chairman Committee to Review Generic Requirements ,
FROM: Frank J. Miraglia, Jr., Deputy DirectorM s Office of Nuclear Reactor Regulation REQUEST FOR ENDORSEMENT OF A FINAL RULE ON [
SUBJECT:
REQUIREMENTS FOR INITIAL OPERATOR LICENSING EXAMINATIONS (10 CFR 55) i The Office of Nuclear Reactor Regulation (NRR) requests that the Committee to Review Generic Requirements (CRGR) endorse a final amendment to 10 CFR 55, " Operators' Licenses," which requires power reactor facility licensees to prepare the initial operator licensing written examinations and operating tests and to proctor and grade the written examinations. Following endorsement, the final rule will be sent to the Director, Office of Nuclear Reactor ~, Regulation, the Executive Director for Operations, and the Commission for approval.
~ Attachment 1 is the " Final Rule - Requirements for Initial Operator Licensing Examinations" in which Commission approval will be sought to publish final amendments to 10 CFR Part 55 dealing with the preparation of operator licensing examinations by facility licensees. The staff considers this Final Rule to be Category 2.
Attachment 2 is the response to the questions contained in Section IV.B of the CRGR Charter. NRR believes that the changes in the proposed final rule are administrative in nature and do not substantially change the examinations administered to license applicants. NRR also believes that 10 CFR 50.109, "Backfitting," does not apply to the planned change in the operator licensing process for the reasons discussed in paragraph (vii) of the attachment. Attachment 3 is a copy of Revision 8 of NUREG-1021," Operator Licensing Examination Standards for Power Reactors," which will formally implement the revised examination process.
. It incorporates a number of lessons leamed during the pilot examinations and comments from the Regional Offices and the industry /public. Several additional refinements are still being evaluated for inclusion in the final NUREG to be issued in conjunction with issuance of the final rule; however, it is not anticipated that any of the pending changes will significantly affect the overall process.
Attachment 4 provides copies of the public comment letters received. The Office of the General Counsel reviewed this final rule and has no legal objections. The Advisory Committee on Reactor Safeguards review of the proposed final rule is scheduled for May 1,1998. Attachments: 1. " Final Rule - Requirements for initial Operator Licensing Examinations"
- 2. Response to CRGR Charter Questions
- 3. NUREG-1021, Revision 8
- 4. Public Comment Letters 9906140100 990608 PDR PR 55 64FR19868 PDR u
% % \%6\oO
- DISTRIBUTION:
DEDR Director, NRR/DRPM Chief, IRM/lRMB
' Information Management Coordinator, NRR/PMSB HOLB/RF '
%vC File Name: g:t2 pts 6fni.crg am W
Vf{b r e e ,y .,im e . me . m th. nom c - copy withoue ., . chm.nt/.ncio.or. r . copy wein .,e,.cnm.ntiencio.or. m Nocopy 0FFICE HOBL/DRCH lc HO@lDRpl lC TEqi ED DRCH/NR M l OGC (W l NAME SGuentherT7 R@_T \ LSpes$cd STreby D DATE 04/30/98 04/ M /98 04/ /98 0$/ /98 09 /96 mummmmmmmm mmmm m mmmmmmmmmmmmmpum mum ummu OFFICE ATD/NRR DRPM l PDIR R, NAME BSheron JW f]fMirYSfa n DATE 04/30/98 $)Lho/98 06'd/98 T g 0FFICIAL REC'RD ) COPY ' SI \ % 5 k4 f k... .
r 1 Attachment 1
" Final Rule - Requirements for Initial Operator Licensing Examinations" e
m, M^ i i I I EQB: The Commissioners FROM- L. Joseph Callan . Executive Director for Operations
SUBJECT:
FINAL RULE - REQUIREMENTS FOR INITIAL OPERATOR LICENSING j EXAMINATIONS l PURPOSE: ' l To obtain Commission approval to publish a final amendment to 10 CFR Part 55 dealing with the preparation of operator licensing examinations by facility licensees. BACKGROUND On March 24,1995, the staff informed the Commission of its intent to revise the operator
,, ' licensing program to allow greater participation by facility licensees and to eliminate contractor assistance in this area (SECY-95-075, " Proposed Changes to the NRC Operator Licensing Program"). In a. staff requirements memorandum (SRM) of April 18,1995, the Commission
- approved the staff's proposal to initiate a transition process to revise the operator licensing program and directed the staff to carefully consider experience from pilot examinations before 3 fully implementing the changes. 'On August 15,1995, the NRC staff issued Generic Letter (GL) ,
- 95-06, " Changes in the Operator Licensing Program," outlining the revised process for !
- developing examinations, and soliciting volunteers to participate in pilot examinations to ;
evaluate and refine the methodology. ; Between October 1,1995, and April 5,1996, the NRC staff administered 22 operator licensing examinations using the revised process. The' staff documented the results of the pilot ; examinations in SECY-96-123, " Proposed Changes to the NRC Operator Licensing Program," and briefed the Commission on June 18,1996. In an SRM of July 23,1996, the Commission directed the staff to prepare a rulemaking plan to justify the changes to 10 CFR Part 55 and to present additional information on a number of issues related to the revised examination process. The SRM also authorized the staff to continue the pilot program on a voluntary basis pending a final Commission decision on the revised examination process. CONTACT: Robert Gallo, NRR 301-415-1031 e
The Commissioners ' On September 25,1996, the NRC staff issued SECY-96-206, "Rulemaking Plan for Amendmente to 10 CFR Part 55 To Change Licensed Operator Examination Requirements." Upon Commission approval of the rulemaking plan in an SRM of December 17,1996, the staff prepared the proposed rulemaking (SECY-97-79, dated April 8,1997), which was approved and modified in accordance with an SRM of June 26,1997, and published in the Federal Register on August 7,1997 (62 FR 42426). When the comment period expired on October 21,1997,11 comment letters had been received and 2 more arrived after the expiration date. The comments from all 13 letters are summarized, and NRC's responses are provided in the Statement of Consideration for the Final Rule (Enclosure 2). DISCUSSION. From the time the pilot program began in October 1995 through the end of Sep'tember 1997, the NRC staff reviewed, approved, and administered a total of 68 examinations that were voluntarily developed by facility licensees in accordance with the NRC's examination criteria and guidance. Facility licensees prepared the written examinations and the operating tests, proctored the written examinations, and graded the written examinations using the guidance provided by the NRC in GL 95-06 during the early stages of the pilot program, and subsequently in interim Revision 8 of NUREG-1021, " Operator Licensing Examination Standards for Power Reactors." NRC examiners thoroughly reviewed the examinations and tests to determine if they were , consistent with NRC standards, directed facility licensees to make whatever changes were necessary to achieve NRC r,tandards, and approved the examinations and tests before they were administered. NRC examiners independently administered all of the operating tests, reviewed the written examination grading, and made the final licensing recommendations for approval by NRC management pursuant to 10 CFR 55.51. The revised examination process makes the operator licensing program more consistent with the NRC's other oversight programs because it requires NRC examiners to review materials prepared by the facility licensees and holds the facility licensees responsible for the quality of those materials. The revised process has enabled NRC examiners to focus more on ;he psychometric quality of the examinations (e.g., the cognitive level at which the questions are written and the plausibility of the distractors or wrong answer choices) prepared by the facility licensees than on the technical accuracy of the examinations, which was their primary focus when the examinations were prepared by the NRC. This shift in the NRC examiners' focus, which is made possible by the revised examination development process, has the potential to improve the overall quality of the licensing examinations. The NRC reviews of the 68 examinations showed that the quality of the licensee-proposed examinations varied widely; essentially all of the proposed examinations required some changes and many needed significant changes to achieve NRC standards for quality and level of difficulty. In some cases, the examinations were delayed for a period of time in order for the L NRC staff to evaluate the examinations and review the facility licensees' changes. Feedba from the NRC examiners who were involved with the pilot examinations indicates that the facility-prepared examinations that were finally approved by the NRC were comparable, in terms of quality and lovel of difficulty, to examinations prepared by the NRC before and since
m._
' The Commissioners beginning the pilot program. Although the staff expected that the quality of tne proposed examinations would improve as the industry became familiar with NRC examination criteria and expectations and gained experience in preparing the examinations, some of the proposed examinations continue to require more changes than anticipated. The staff's expectations regarding the quality and level of difficulty of the proposed examinations have not yet been achieved. (Enclosure 1 provides a discussion of this issue and actions that are being taken to address it.)
The average passing rates for reactor operators (ROs) and senior reactor operators (SROs) on both the written and operating portions of the 68 facility-prepared examinations administered through the end of fiscal year (FY) 1997 were somewhat lower than the corresponding passing rates on the examinations administered during FY 1995, the last year in which all of the examinations were prepared by the NRC or its contractors. However, the average passing rates were not significantly below the range of passing rates for FYs 1990 through 1994. This supports the NRC staffs conclusion that the facility-prepared examinations, revised as directed by the NRC, are effective. The examinations are discriminating at a conservative and acceptable level (i.e., the average level of difficulty has not declined).-(Enclosure 1 includes a detailed summary and analysis of the examination results.) With regard to the efficiency of the revised examination process, the pilot examination experience to date suggests that the average industry cost will not be significantly different from what it was when the NRC prepared all of the licensing examinations. Feedback from the industry reflects that the cost for some facility licensees was higher than it was for an NRC-prepared examination; other licensees, however, were able to prepare acceptable quality examinations at a lower cost than if the examinations had been prepared by the NRC. The industry generally attributed the higher cost to the revised examination and administrative
~
criteria under the pilot examination process. Although the NRC staff acknowledges that the f~ revised criteria contribute to the elevated cost, many of the variables that affect the final cost of the examination (e.g., its initial quality) are now under the control of facility licensees. Those facility licensees that submit acceptable quality examinations are likely to save resources despite the additional administrative criteria that the NRC considers necessary under the revised examination process. However, those facility licensees that submit examinations that require many changes because they do not meet NRC standards, may have increased costs. The higher cost of revising a low quality examination should provide an incentive for facility licensees to submit acceptable quality examinations. (Enclosure 3 provides an analysis of the industry resource burden.) Comments from the NRC chief examiners who worked on the pilot examinations indicate that the average amount of time spent reviewing and revising the facility-prepared examinations was generally consistent with the estimates developed before starting the pilot program. Although it took longer than expected to revise a number of the examinations to meet NRC standards for quality, the resource burden was generally offset by other examinations that required less effort to review and revise. Therefore, the NRC staff has concluded that the original objective of the pilot examination and transition program has been achieved. The program has demonstrated that the NRC can eliminate the use of contractors in the operator licensing program (except for the generic fundamentals' examination) and can conduct the initial operator licensing and b
The Commissioners requalification inspection programs with the existing NRC staff. (Enclosures 1 and 3 provide an analysis of the NRC resource burden.)- ' As discussed in SECY-96-206 and the proposed rulemaking, the NRC staff is focusing additional attention on examination security. Therefore, the staff has amended 10 CFR 55.49 in the Final Rule to ensure that applicants, licensees, and facility licensees understand the scope and intent of the regulation and to require facility licensees to establish procedures to control examination security and integrity. The staff has also strengthened the discussion of examination security in final Revision 8 of NUREG-1021 and modified NUREG-1600, " General Statement of Policy and Procedures for NRC Enforcement Actions," to' address enforcement action relative to the requirements in 10 CFR 55.49. As a separate action, the NRC will delay any examination that may have been compromised until the scope of the poten,tial compromise is determined and measures can be taken to address the integrity and validity of the examination. In summary, the NRC staff has concluded that the revised examination process is both effective and efficient. The staff has reviewed the vulnerabilities discussed in SECY-96-206 (i.e., quality and consistency; independence and public perception; examination security; NRC resources; program stability; and examiner proficiency) and evaluated the measures that have been taken to mitigate the vulnerabilities. (Enclosure 1 includes a summary analysis of each vulnerability and the potential effects on reactor safety.) The revised program provides the staff the ability to ; maintein acceptable standards of performance in each of these areas, and the pilot program has demonstrated that it is capable of doing so. NRC license examiners will continue to review each examination to ensure that it conforms to the guidelines in NUREG-1021 regarding content, format, quality, and levels of knowledge and difficulty. The NRC will direct facility licensees to revise the examinations, as necessary, and will approve every examination before it is given. The NRC will delay those examinations that do not meet NRC standards by the scheduled administration date and, to the extent that facility employees do not follow the examination criteria in NUREG-1021,' address deficiencies in the submitted examinations as licensee performance issues. Moreover, NRC examiners will directly observe and evaluate the ' performance of every license applicant on both the dynamic simulator and walk-through portions of the operating test. The staff believes that this willimprove the NRC's ability to focus on operator performance issues and enable NRC examiners to accrue more experience in a shorter period of time. COORDINATION The Office of the General Counsel (OGC) has no legal objection to this final rule. As discussed in the FederalRegister notice (Enclosure 2), OGC has determined that the backfd rule (10 CFR 50.109) does not apply. The Office of the Chief Financial Officer has reviewed this Commission paper for resource impacts and has no objections. This final rule has also been reviewed by the Advisory Committee on Reactor Safeguards (ACRS) and the Committee to Review Generic Requirements (CRGR). 1
The Commissioners RESOURCES-This action has allowed the staff to discontinue the use of contractor support for the development and administration of site-specific initial operator licensing examinations. Before initiating the pilot examination and transition process at the beginning of Fiscal Year (FY) 1996, the NRC spent approximately $3 million per year on contractor assistance for initial ~ examinations and requalification inspections.: During FY 1996, when approximately 40 percent
. of the examinations were voluntarily prepared by facility licensees, the NRC spent -
approximately $1.6 million for contractor assistance (to prepare and administer initial examinations and assist with requalification inspections). In FY 1997, facility participation
~ increased to include approximately 75 percent of the examinations, and the NRC's spending on contractor assistance for the licensing examinations and requalification inspections decreased to approximately $0.5 million.
The FY 1998 and FY 1999 budgets are consistent with this proposal and reflect the elimination of contractor support for the operator licensing program (with the exception of the generic fundamentals examination). If the Commission decides not to amend 10 CFR Part 55 as recommended by the NRC staff, other measures will be needed to ensure that the facility licensees' requirements for licensed operators are satisfied. Such measures might include (1) examination length or format changes that will enable the existing staff to conduct the examination and inspection programs, (2) scheduling the examinations based on the NRC resources available (which may require the staff to prioritize its activities based on the facility licensees' needs), (3) an increase in direct examiner resources in each regional office to satisfy the demand for initial licensing examinations, or (4) the restoration of contractor support for the
, operatorlicensing program.
RECOMMENDATIONS: ; That the Commission
- 1. Anprove the notice of final rulemaking (Enclosure 2) for publication in the Federal Register.
l
. 2. ' Gadsfg that this rule, if promulgated, would not have a significant economic impact on a substantial number of small entities to satisfy the requirements of the Regulatory Flexibility Act,5 U.S.C. 605(b).
- 3. Determine that the backfit rule,10 CFR 50.109, does not apply to this rule.
~4. Determine tnat neither an environmental impact statement nor an environmental assessment has been prepared because this proposed rule is eligible for a categorical ,
. exclusion as defined in 10 CFR 51.22(c)(1). :
l l
1 l
' The Commissioners L . 5. hktig that .
l ) a. The Chief Counsel for Advocacy of the Small Business Administration will be informed of the certification regarding economic impact on small entities and the
- reasons for it as required by the Regulatory Flexibility Act.
- b. The final rule contains information collection requirements that have been i
reviewed a' nd approved by the Office of Management and Budget (OMB).
. c. A regulatory analysis has been prepared (Enclosure 3).
- d. .'A revision to the enforcement policy is being prepared and will be issued
- separately.
- e. This is not a " major" rule as defined in the Small Business Regulatory Enforcement Faimess Act of 1996,5 U.S.C. 804(2).
- f. The appropriate Congressional committees will be informed (Enclosure 4).
- g. A public announcement will be issued when the final rulemaking is filed with the Office of the Federal Register (Enclosure 5).
. h. Copies of the notice of final rulemaking and final Revision 8 of NUREG-1021,
' " Operator Licensing Examination Standards for Power Reactors " will be
, , distributed to all facility licensees. I l
- l. l L. Joseph Callan Executive Director for Operations ;
i
Enclosures:
As stated (5)
.s 1
-4 4
e
) '
The Commissioners 4 . 5. Nola that
- a. The Chief Councel for Advocacy of the Small Business Administration will be informed of the certification regarding economic impact on small entities and the reasons for it as required by the Regulatory Flexibility Act.
- b. ' The final rule contains'information collection requirements that have been reviewed and approved by the Office of Management and Budget (OMB).
- c. A regulatory smalysis has been prepared (Enclosure 3).
d.' A revision to the enforcement policy is being prepared and will be issued separately.
- e. This is not a " major" rule as defined in the Small Business Regulatory l Enforcement Fairness Act of 1996,5 U.S.C. 804(2).
- f. The appropriate Congressional committees will be informed (Enclosure 4).
- g. A public announcement will be issued when the final rulemaking is filed with the Office of the Federal Register (Enclosure 5).
- h. Copies of the notice of final rulemaking and final Revision 8 of NUREG-1021,
" Operator Licensing Examination Standards for Power Reactors," will be ,, distributed to all facility licensees.
L Joseph Callan
- Executive Director for Operations
Enclosures:
As stated (5)- DISTRIBUTION: Commissioners
- OGC 'OPA OlG ClO EDO ACRS OCCA OCA- CFO SECY REGIONS To secolve a com ' of this clocument, Indicate in the hos: *C" a Copy w/o suechmenWenciosure "E" a copy w/ ettechmenuenclosure "It'
- No copy OFFICE NMSS- l oGC I OE l ADM l OCFO l NAME CJ Papenello JR Grey J Uebermen DL Meyer JL Funches DATE- / 198 / 198 / /98 / 198 / 198 OFFICE 0C10 l NRR l EDO I l l NAME sJ Shelton s Colhne LJ Callen DATE / 19 8 # 198 / 198 / /98 / 198 OFFICIAL RECORD COPY
l l l 1 I l l Enclosure 1 Pilot Program Results, l Changes, and Vulnerabilities ! l l j l
o t PILOT PROGRAM RESULTS, CHANGES, AND VULNERABILITIES Backaround In a staff requirements memorandum (SRM) of April 18,1995, the Commission approved the NRC staff's proposal to initiate a transition process to revise the operator licensing program to require power reactor licensees to prepare the initial licensing examinations for their own facilities and directed the NRC staff (hereafter staff) to carefully consider experience gained from a pilot examination program before fully implementing the changes. With the Commission's approval, the staff issued Generic Letter (GL) 95-06, " Changes in the Operator Licensing Program," on August 15,1995, outlining the revised examination development process and soliciting volunteers to participate in a pilot examination program to evaluate and . refine the methodology. Between October 1,1995, and April 5,1996, the staff reviewed and approved 22 operator-licensing examinations prepared by facility licensees within the framework of a pilot program. These examinations were prepared using the guidance in Revision 7 (Supplement 1) of NUREG-1021, " Operator Licensing Examiner Standards," and the additional guidance in GL 95-06. The results of the pilot examinations were discussed in SECY-96-123, " Proposed Changes to the NRC Operator Licensing Program," dated June 10,1996, and lessons leamed from those examinations were incorporated in interim Revision 8 of NUREG-1021, which was retitled " Operator Licensing Examination Standards for Power Reactors." Based on the results of the pilot examinations, the staff recommended that the Commission approve the implementation of the new examination process on a voluntary basis until rulemaking could be completed to require all power reactor facility licensees to prepare the initial licensing examinations and to proctor and grade the written portion of the examinations. In an SRM of July 23,1996, the Commission authorized the staff to continue the pilot examination process on a voluntary basis and directed the staff to develop a detailed rulemaking plan to justify the changes that may be necessary to 10 CFR Part 55. The
. Commission also directed the staff to address the pros, cons, and vulnerabilities associated with the revised examination process to facilitate a Commission decision on whether to implement the revised process on an industrywide basis. With the Commission's approval, the staff resumed the pilot program in August 1996 (after having completed the original 22 examinations in April 1996), and by the end of Fiscal Year (FY) 1997 had reviewed, approved, ;
and administered 46 addition # waminations developed by facility licensees in accordance with NRC guidance, thereby raisiw t a total number of pilot-style examinations to 6d. The results of those examinations are disec ,d in Section I below. 1 On September 25,1996, the staff forwarded the requested rulemaking plan and a discussion of l
. the pros, cons, and vulnerabilities of the pilot process (including a detailed discussion of the measures taken to mitigate the vulnerabilities) to the Commission in SECY-96-206, ;
"Rulemaking Plan for Amendments to 10 CFR Part 55 to Change Licensed Operator
- Examination Requirements." In an SRM of December 17,1996, the Commission directed the staff to implement interim Revision 8 of NUREG-1021 on a voluntary basis and to proceed with the proposed rulemaking. The staff forwarded the proposed rule (SECY-97-079, " Proposed Rule - Initial Licensed Operator Examination Requirements") to the Commission on 1
w April 8,1997, and on June 26,1997, the Commission approved it's publication in the Federal ,
' Registerfor a 75-day comment period. The notice of proposed rulemaking (62 FR 42426) )
dated August 8,1997, also invited public comments on interim Revision 8 of NUREG-1021. : Public comments related to the regulation are addressed in the Federa/ Register notice (FRN)
- (Enclosure 2), and significant comments regarding interim Revision 8 of NUREG-1021 are
' discussed in Section il below (the staff also addressed several editorial comments during the revision process), j
- The staff has reviewed the vulnerabilities discursed in SECY-96-206 (i.e., quality and I consistency,' independence and public perceptien, examination security, NRC resources, :
program stability, and examiner proficiency) in light of the additional experience with the revised l examination process since that paper was issued on September 25,1996. The status of each ! vulnerability, including an overview of the compensatory measures, and the potential effects
! that the revised process might have on reactor safety are summarized in Section ill below.
Pilot' Prooram Overview From the time the pilot program began in October 1995 through the end of September 1997, l the NRC staff reviewed, approved, and administered a total of 68 examinations that were i voluntarily developed by facility licensees under the pilot examination and transition program. Facility licensees prepared the written examinations and the operating tests, proctored the ! written examinations, and graded the written examinations using the guidance provided by the NRC in GL 95-06 during the early stages of the pilot program, and subsequently in interim Revision 8 of NUREG-1021, " Operator Licensing Examination Standards for Power Reactors." NRC examiners thoroughly reviewed the examinations and tests to determine if they were consistent with NRC standards, directed facility licensees to make whatever changes were ; necessary to achieve NRC standards if the submitted examinations and tests were deficient, and approved the examinations and tests before they were administered. NRC examiners independently administered all of the operating tests, reviewed the written examination grading, and made the final licensing recommendations for approval by NRC management. I i 2 i
SECTION I . PILOT PROGRAM RESULTS ) l The SRM for SECY-96-206' directed the staff to (1) continue monitoring the results of the pilot f examinations for indications that the revised examination development process may not be appropriate and (2) to present the current data t.o the Commission along with the proposed and ' final rulemaking packages. Accordingly, Table 1.1 summarizes the results of the 68 pilot examinations conducted between October 1995 (the beginning of the pilot program) and the - end of September 1997. For purposes of comparison, Table 1.1 also includes the examination results for FYs 1990 through 1995 (the last year in which the NRC prepared all of the examinations) and the examinations prepared by the NRC since the pilot program was initiated (i.e., for FYs 1996 and 1997). In conducting its analysis, the staff has focused primarily on the annual examination passing rates because the small number of applicants on most examinations makes it difficult to draw programmatic conclusions from the individual examination results. l 4 Table 1.1 - Comparison of Examination Passing Rates' Exams RO Written RO Operating SRO Written SRO Operating i FY 1990 - 1994 Ran9e 92 - 95 % 95- 98 % 94- 98 % 93- 96 % FY 1995 (Last year all NRC) 94 % 98% 96 % 95 % Facility (27) 93% 94 % 94 % 95 % FY 1996 NRC (41) 98% 94 % 96 % 92 % ; Total (68) 96% 94 % 96 % 94 % f Facdy(41) 89% 94 % 93 % 92 % I i' 2 FY 1997 NRC (13)8 96 % 93 % 91 % 2 84 % Tetal(54) ' 91 % 93% 93% 91 % ! Facility Total (68) 91 % 94 % 93% 94 % 1 (FY 1996 and FY 1997)
- 1. The results indicate the percentage of applicants who passed the examination or test.
- 2. One additional NRC-prepared SRO examination was administered, but the facility licensee subsequently withdrew the applicebons and the results were nullified. If those results are included, the written and operating test passing rates would be 82 and 76 percent, respectively.
The average passing rates for reactor operators (ROs) and senior reactor operators (SROs) on both the written and operating portions of the 68 facility-prepared examinations administered through the end of FY 1997 were only slightly lower than the corresponding passing rates on the NRC-prepared examinations administered during FY 1995, the last year in which all of the examinations were prepared by the NRC or its contractors. However, as shown in Table 1.1, the average passing rates tend to fluctuate, and the passing rates on the facility-prepared examinations were generally consistent with the historical range. The fluctuations in the 3
passing rates from year to year and between the examinations prepared by the NRC and facility y licensees could be caused by a number of factors including variations in the average level of
' experience of the license applicants, changes in the quality of the training or the facility licensee's threshold for screening its applicants before they take the licensing examination, or - variations in the average level of difficulty of the examinations. Even if it is assumed that all of the variation in the passing rates on the facility-prepared examinations has been caused by the q changes in the examinations, the magnitude and direction of the variation supports the staff's i conclusion that, on the average, the facility-prepared examinations, revised as directed by the l NRC, are effective; The examinations are discriminating at a conservative and acceptable level !
(i.e., the average level of difficulty has not declined and any possible increase is not considered l to be of practical significance), and the safe operation of the facilities is not being compromised. " If the lower passing rate on the facility-prepared written examinations is indeed an indication that the level of difficulty of the examinations has increased, it may be a reflecti.on of the NRC examiners' concentration on the psychometric quality (e.g., the cognitive level at which the questions are written and the plausibility of the distractors - those answer choices that should appear reasonable but, nevertheless, are wrong) in addition to the technical quality of the examinations. ' Although the staff did not intend for the level of difficulty or the failure rate on the ; examinations to increase, the examiners' efforts to achieve NRC standards regarding the , cognitive level of questions and to improve the plausibility of the distractors may have improved
. the discrimination validity of the examinations. Consequently, those applicants who may have passed an examination containing lower cognitive level questions on which some of the distractors could be eliminated as implausible are now having more difficulty selecting the correct answers. The NRC examiners' efforts to improve the plausibility of the distractors has reduced the applicants' chances of passing the examination by simply guessing the correct answers. However, considering the historical fluctuation in the average examination passing ,
, rates and the other factors (e.g., training deficiencies or less stringent screening of applicants -
- by facility licensees) that could be responsible for some or all of the observed decline, the staff has concluded that any increase in the level of difficulty is not significant. (The level of difficulty -
of the written examinations is further discussed below in connection with the review of pilot . examination avality.) { The RO passing rates on the facility-prepared operating tests during FYs 1996 and 1997 were i consistent with the passing rates on the tests prepared by the NRC during those same years. 1
' The 98 percent RO operating test passing rate during FY 1995 was unusually high compared to y prior years. Although the passing rate on the facility-prepared SRO operating tests during FY 1997 was slightly lower than the FY 1995 baseline, it was the same as the passing rate on the j NRC-prepared operating tests during FY 1996.
l The SRO passing rates on the NRC-prepared written examinations and operating tests during FY 1997 were lower than the FY 1995 baseline, the historical ranges, and the facility-prepared results during FY.1997. The low passing rates on the NRC-prepared examinations are in large
. part attributable to two examinations on which the applicants' performance was unusually poor for reasons that may include variations in the quality of the applicants' training and the difficulty of the examinations. This result illustrates that the applicants' performance on the examinations can vary from year to year regardless of whether the examinations are prepared by the NRC or by facility licensees.
4
c:7 , , i j I Another indicator of the relative consistency in the level of difficulty of the NRC-prepared and facility-prepared examinations is the amount of variation in the applicants' average grades on j
.the written portion of the licensing examinations. .. Table 1.2 summarizes the average.RO and SRO grades for FYs 1995 through 1997. The data are consistent with the passing rates
- summarized in Table 1.1 and support the staff's conclusion that the facility-prepared
- l. . examinations are discriminating at a conservative and appropriate level. The staff expects that i l minimally quahfied applicants will be able to achieve a score of at least 80 percent on an NRC l written licensing examination.
j Table 1.2 - Comparison of Average Written Examination Grades l Exams . RO SRO FY 1995 (All NRC) 87.0 % 88.5 % Facility 87.1 % 88.5 % ; FY 1996 : ! NRC 87,7 % 88.4 % Total 87.5 % 88.5 % Facility 84.4 % 86.4 % FY 1997 l NRC 86.6 % , 86.8% Total ' 84.9 % 86.5 % l L ! l [ Examination Quality Although the overall examination results indicate that the pilot-style examinations are effective at discriminating between those applicants who have' mastered the job requirements and those who have not,'the NRC reviews showed that essentially all of the facility-prepared examinations required some changes. According to questionnaires completed by the NRC chief examiners responsible.for the pilot examinations, the average facility-prepared examination required i approximately 10 to 20 changes, which is consistent with the number of changes often required on examinations prepared by NRC contract examiners. However, a number (about 40 percent l in FY 1997) of the facility-prepared examinations required significant changes to achieve NRC standards for quality and level of difficulty, and, in some cases, the examinations were delayed in order for the staff to fully evaluate the examination and allow time for the facility personnel to change the examinations, in FY 1997, about 10 percent of the examinations had to be delayed
- some for only a week and others for up to a month - until the changes could be made. The written examinations generally required the most changes, The majority of the changes were necessary to improve the plausibility of the distractors. Both the walk-through and dynamic simulator portions of the. operating tests required changes to replace or enhance nondiscriminatory tasks, events, and questions. l The staff evaluated comments made by the NRC chief examiners and facility licensees involved with the examinations to gain an understanding of why so many of the examinations have required significant changes. Four possible explanations have been considered, but their order 5
and level of significance is' uncertain:, (1) the examination criteria and guidance in interim Revision 8 of NUREG-1021 may have been unclear; (2) the facility personnel assigned to prepare the licensing examinations may not have had the expertise to construct examinations Lmeeting NRC standards of quality; (3) the facility personnel assigned to prepare the examinations may have been given insufficient time or other resources to prepare an adequate examination; and (4) it is possible that the facility personnel assigned _to prepare the examinations may not have followed the guidance in NUREG-1021 because of a bias toward writing examinations that were less difficult. Each of these possibilities is discussed below. With regard to the examination criteria and guidance in interim Revision 8 of NUREG-1021, the NRC chief examiners who responded to surveys on their experiences during the pilot-style examinations reported that the instructions were clear. They recommended some clarifications in the guidance, but no major changes to the examination criteria that have a direct bearing on examination quality. As discussed in SECY-96-206, the staff solicited industry. comments on a draft of Revision 8 of NUREG-1021 in February 1996, but none of the respondents (which included the Nuclear Energy institute (NEI) and two facility licensees) questioned the clarity of the guidance. When SECY-97-079 solicited comments on interim Revision 8 of NUREG-1021, one facility licensee (of the five that responded) commented that the guidance on level of
~ difficulty needs to be clarified to enable facility licensees to satisfy the NRC's expectations in this area and to minimize the NRC examiners' subjective judgment. (Both the NRC examiners' and the facility licensees' commente Jn interim Revision 8 of NUREG-1021 are summarized in Section ll below.) . Based on the it# uation available, the staff has concluded that, in general, the guidance in interim Revision 8 of NUREG-1021 is sufficiently detailed to yield examinations of acceptable quality, difficulty, and consistency. '_ NUREG-1021 discusses and illustrates the
= guidelines for constructing questions and the psychometric principles that justify most of the changes that were made in the facility-prepared examinations. To further ensure that facility licensees understand the guidelines in NUREG-1021, the staff has conducted or participated in a number of public meetings and workshops in an effort to communicate its expectations to the facility employees who will prepare the examinations. Additional NRC and industry workshops are being planned to address examination quality and solicit industry feedback on NUREG-
~ 1021. The staff will continue to accept and review questions and comments on the NUREG and expects that additional clarifications and changes will become necessary in response to unanticipated issues.
~ Although it was not reflected in their public comments, some facility managers have since commented during meetings with the staff (e.g., a January 27 - 29,1998, Region ll public
. workshop to review test-writing techniques and other examination issues) that the NRC has raised its standard with regard to level of difficulty (particularly that of the written examinations)
. since beginning the pilot program. The staff did not intend to raise its standards during the pilot program; however, it is possible that the NRC examiners' efforts to maintain the effectiveness of the operator licensing process have raised the discriminatior, validity and level of difficulty of the examinations.
The NRC has always expected its licensing examinations to discriminate between applicants who have and have not mastered the knowledge, skills, and abilities required to perform licensed RO and SRO duties. To accomplish that objective, the NRC established a number of criteria against which its examiners are expected to evaluate test items in order to achieve an 6 L . .,. .
acceptable examination in terms of quality, consistency, level of knowledge and level of difficulty. In addition to being technically accurate, the NRC expects written examination 3 questions to be written at the highest level of knowledge appropriate for the topics selected for I the examination and the distractors in multiple choice questions to be plausible (i.e., they should appear reasonable but, nevertheless, be wrong). When the NRC or a contractor prepared an examination, the technical accuracy of the questions generally took precedence over their psychometric quality. However, the staff believes that the overall quality of the { questions has improved over time as a result of repetitive use and revision and would have i continued to improve even if the NRC had retained responsibility for developing the licensing ) examinations. Now that facility licensees are being held responsible for the technical accuracy
. of the questions, NRC examiners are able to devote more effort to ensuring that the distractors are plausible and the questions are written at the appropriate level of knowledge.
In the late 1980s, when the NRC shifted the format of the written examination from essay and - short answer to multiple choice questions, it took time to develop the NRC's bank of questions, particularly those that tested at the higher cognitive levels because they are more challenging and costly to write. By the time the NRC initiated the pilot program in FY 1996, the average written examination (based on audits performed by the staff) contained approximately 50 percent comprehension or analysis questions. To ensure that the examinations would remain effective, the staff revised interim Revision 8 of NUREG-1021 to establish a 50 percent lower limit on the higher cognitive level questions. The staff has concluded that the lack of an upper limit may have contributed to the increased variability in the level of difficulty of the examinations. The staff has also noted that some examiners may have suggested that facility licensees submit more than the minimum percentage of comprehension-level questions, thinking that the examination authors are more likely to over-estimate than under-estimate the j cognitive levels of the questions. As discussed in Section 11 below in connection with the : resolution of public comments on interim Revision 8 of NUREG-1021, the staff has clarified final Revision 8 by specifying a range of higher cognitive level questions, instead of a lower limit. With regard to the expertise of the facility personnel assigned to prepare the examinations (i.e., ' . the second possible ' explanation for the number of examination changes), the preliminary and proposed regulatory analyses for this rulemaking expressed the staff's expectation that the l training personnel at power reactor facilities already possess the basic expertise needed to develop test items for the initial licensing examinations. The staff believed that many of the problems with examination quality and level of difficulty could be attributed to the facility , licensees' lack of familiarity with the NRC's requirements and expectations and that, with ! experience, the industry would gain proficiency in preparing the examinations. However, the overall quality of the examinations submitted to the NRC during the pilot program did not improve as expected over time. Although approximately half of the 17 facility licensees that had , prepared more than one examination by the end of FY 1997 did maintain or improve the quality of their second or third examinations, the quality of the other facility licensees'second or third . examinations was lower. However, the pilot examination program has demonstrated that the NRC's review and approval process is ensuring that the quality and validity of the examinations that are administered to the license applicants, and upon which the NRC's licensing decisions are based, meet NRC standards. Although it is unclear to what extent the problems with examination quality and level of difficulty 7 '
r are c'etermined by the qualifications and training of the examination writers, the staff has asked the it dustry to address this area. Some facility licensees and NRC examiners have asserted
- that the personnel restrictions in NUREG-1021 could impede training or diminish examination
- . quality because facility licensees must choose where to assign their best-qualified personnel and whether to hire contractors to write the examinations. Although the staff still believes that the restrictions are needed to limit the potential for bias, it has relaxed the personnel restrictions
. in final Revision 8 of NUREG-1021 to allow facility licensees greater flexibility regarding which instructors can prepare the examinations (as explained in Section il below in connection with the staff's response to the public comments on interim Revision 8 of NUREG-1021 and the review of concems regarding independence and public perception in Section lil).
With regard to the level of facility resources required for preparing a licensing examination (the third possible explanation for the number of examination changes noted above), NRC examiners have seen instances where the personnel on some facility training sjaffs have been challenged to prepare the examinations without additional assistance. Moreover, NEl publicly { commented that the personnel restrictions in NUREG-1021 have made it more difficult for facility licensees with small training staffs to prepare the examinations. Therefore, some facility
. licensees may have difficulty allocating the resources necessary for their training staffs to l
. prepare good quality licensing examinations, while continuing to train the license applicants.
The fact that some facility licensees have had to hire contractors to prepare their licensing examinations may also have had an effect on the technical quality of their examinations and the efficiency with which they were prepared. (Issues regarding the facility resource burden are further discussed in the regulatory analysis (Enclosure 3), and the NEl comment regarding i personnel restrictions, including the staff's rationale for relaxing those restrictions, are discussed in Seetion ll below.) _ j
.. . As noted above, the fourth possible explanation for the number of examination changes is that facility personnel assigned to prepare the examinations may not have followed the guidance in NUREG-1021 because they may have been biased toward writing examinations that were less !
. difficult. Some NRC examiners have reported a reluctance on the part of facility personnel to make NRC-directed changes that might increase the level of difficulty of the examination and expressed concem (in their public comments on the rulemaking) that the NRC's review and approval process may not adequately compensate for the conflict of interest inherent in the revised examination method. .Two NRC examiners commented that the revised process puts
' training managers in a no-win situation and that some managers will prepare the easiest possible examination to minimize the risk of their applicants failing the examination. (A training manager made such a comment to an NRC examiner, and the region addressed the issue in a management meeting with the facility licensee.) The staff agrees that some facility personnel could prepare an examination that will maximize their applicants' chances for success despite the criteria and guidance in NUREG-1021 and their own expertise in testing and measurement techniques. The issues of independence and conflict of interest, as well as the controls that the staff has implemented to address the issues, are discussed in Section lil below in connection with the review of the vulnerabilities associated with the revised examination process and in the FRN in connection with the resolution of public comments on the rule.
In summary, the staff has concluded that, despite its concems regarding the quality of the facility-prepared examinations as submitted to the NRC, the revised examination process offers 8 e
y , , I an effective alternative to examinations prepared by the NRC or an NRC contractor. The pilot program has demonstrated that NRC examiners and their supervisors can and will ensure that facility licensees revise the examinations as necessary to arrive at acceptable levels of quality l and difficulty. Moreover, the staff will delay those examinations that do not meet NRC i standards by the scheduled administration date and, when appropriate, address significant l deficiencies in the submitted examinations as licensee performance issues in the examination i reports. g i y Pilot Process Efficiency With regard to the efficiency of the revised examination process, experience with the pilot j examinations to date suggests that the average cost to facility licensees will not differ - l significantly from the cost of the NRC or its contractors preparing all of the licensing I l examinations.' Comments from the industry reflect that it cost some facility licepsees more to .. prepare their own examinations than they paid the NRC to prepare them; other licensees, however, were able to. prepare good quality examinations at a lower cost than if the examinations wtere prepared by the NRC.
- The industry attributed the higher cost to the revised examination and administrative criteria (e.g., documenting the source of test questions) under the pilot examination process. Although the staff acknowledges that the revised criteria contribute somewhat to the increased cost, facility licensees are now controlling many of the variables that affect the quality and, consequently, the final cost of an examination. For example, facility licensees can manage the size and quality of their examination banks and, to a large extent, the training and experience of the people.they assign to write their licensing examinations. The revised examination process j
. puts facility licensees in control of developing the examinations and holds them responsible for l
. their quality, if a facility licensee submits a good quality examination, it is likely to save i resources despite the additional administrative requirements; however, if the facility licensee )
submits an examination that requires many changes, it is likely to cost more than if the NRC had prepared the examination.- 1 Comments from the NRC chief examiners who worked on the pilot examinations indicate that l the average amount of time NRC staff spent reviewing the facility-prepared examinations and i directing changes was generally consistent with the estimates developed before starting the pilot program. Although about 20 percent of the examinations reviewed in FY 1997 required significantly more effort than expected to achieve the NRC's standards, the burden was l generally offset by other examinations that required less effort to review and revise. Therefore, the staff has concluded that the original objective of the pilot examination and transition program has been achieved. The program has demonstrated that the NRC can eliminate the use of contractors in the operator licensing program (except for the generic fundamentals examination) and can conduct the initial operator licensing and requalification inspection programs with existing NRC staff.. i L l l 1 I l l E i
SECTION ll STAFF RESPONSES TO COMMENTS ON INTERIM REVISION 8 OF NUREG-1021 Public CommentsL
- As noted above, since beginning the pilot program in October 1995, the NRC staff has given the industry and public two opportunities to formally comment on the guidance in Revision 8 of NUREG-1021. The staff has also participated in a number of public meetings and workshops
- that have given the industry an attemative means to provide feedback or to raise questions
~
regarding the guidance in NUREG-1021. The staff has considered all of the comments and
- questions and made changes or clarifications,'as appropriate. The staff will continue to accept
- and review questions and comments as they arise and expects that additional clarifications and changes will become necessary in response to unanticipated issues.
~
- Of the 13 individuals and organhations that responded to the proposed rule published in the FederalRegisteron August 7,1997 (62 FR 42426),7 also commented on interim Revision 8 of NUREG-1021. Tne staff considered all of the comments and revised interim Revision 8 of NUREG-1021 as appropriate; the significant comments and the staffs responses are summarized below. A copy of final Revision 8 of NUREG-1021 will be sent to each facility licensee upon Commission approval of the final rule. Copies of NUREG-1021, Revision 8, will also be available in the NRC Public Document Room (PDR),2120 L Street, NW (Lower Level),
Washington, DC, and on the intemet at http://www.nrc. gov for a limited period of time. Comment: Seven respondents recommended changes to the restrictions that interim Revision 8 of NUREG-1021 places on facility personnel who can participate in developing the licensing
; examinations. The respondents from the nuclear industry generally believe that the restrictions
- place an undue burden on facility licensees with no apparent benefit and that the personnel ;
who played an active role.in training the license applicants should not be restricted from l preparing the licensing examinations. The industry strongly endorsed adopting the NRC's i requalification examination personnel restrictions so that any instructor could prepare the initial ;
' licensing examination as long as he or she refrained from teaching the applicants after starting i work on their licensing examination. Some of the industry respondents asserted that the guidan::e regarding supervisory involvement in the examination development process needs to be clarified.~ One NRC license examiner recommended that the restrictions be clarified to ;
prevent the same person from preparing both the audit examination that the facliity licensee i uses to confirm the applicants' readiness and the NRC licensing examination. Response: The staff acknowledges that the personnel restrictions may have increased the burden for some facility licensees with smaller training staffs, and that the quality of the training could suffer if personnel are diverted from training assignments to prepare the examinations. 1 Therefore, the NRC has revised the personnel restrictions in final Revision 8 of NUREG-1021 in'
- a manner that the staff believes will reduce the burden on facility licensees without substantially
- increasing the potential for conflicts of interest that could bias the examination process.
Specifically, final Revision 8 will allow facility instructors to prepare the written examination .
. questions without regard to the amount of time they spent training the license applicants, (as is the case on requalification examinations).but it will still preclude instructors from preparing questions related to those topics on which they provided instruction (which remains somewhat
. 10 1
more restrictive than the requalification criteria) and from instructing the applicants once they begin working on the licensing examination (as was the case in interim Revision 8 and on requalification examinations). As a result of this change, a facility licensee that has two
, instructors who equally shared the responsibility for training the license applicants, can use both instructors to prepare examination questions related to the topics they did not teach, whereas,
, they previously would both have been restricted from any examination development. Facility
. licensees may still propose forstaff consideration, on a case-by-case basis, other approaches to address the NRC's concem regarding conflict-of-interest < The vulnerability discussion
' regarding independence and public perception (in Section lil) provides additionaljustification for this change and an overview of the controls that the staff has implemented to address this issue.
With regard to facility management involvement in the examination development and review process, the staff has revised the guidance in final Revision 8 of NUREG-1021.so that it does not unnecessarily restrict management activities or authority. Comment:- One facility licensee commented that the guidance on level of difficulty for both the written examination and the operating tests needs to be clarified so that licensees can better satisfy the NRC's expectations on their first try. According to the licensee, the current guidance
' relies too heavily on the subjective judgment of the chief examiner, which fosters inconsistency.
Response: The staff agrees that assessing the level of difficulty of test items does involve some level of subjectivityi Writers of facility examinations and NRC examiners who are trained in the subject matter, measurement principles, and psychometrics, and who have general knowledge of operator and trainee performance on similar test items, can make informed
. Judgments on level of difficulty based on the guidance in NUREG-1021.
interim Revision'8 of NUREG-1021 discusses the concepts of discrimination validity and
' examination difficulty and includes a number of criteria directed at limiting the amount of variation in the difficulty of both the written examinations and the operating tests. For example,
- NUREG-1021, Appendix D," Simulator Testing Guidelines," discusses the quant;tative and qualitative attributes of simulator scenarios, and Section ES-301, " Preparing initial Operating Tests," of NUREG-1021 provides a specific target range for each of the quantitative attributes applicable to the initial licensmg examination. Similarly, NUREG-1021, Appendix B, " Written Examination Guidelines," discusses the principles of level of knowledge and level of difficulty, and ES-401, " Preparing Initial Site-Specific Written Examinations," specifes an upper limit on the number of questions based on fundamental knowledge or memory. However, it has come to the staff's attention that ES-401 does not specify an upper limit on the number of comprehension-level or analysis-level questions and that this may have contributed to variations in the level of difficulty of some examinations. Therefore, the staff has revised the criteria in interim Revision 8 of NUREG-1021 to specify ranges for memory and comprehension /
analysis-level questions. Other measures that help minimize variation in the level of examination difficulty include (1) 6 NRC and industry workshops designed to help facility licensees better understand the NRC's standards (as stated in NUREG-1021) and expectations with regard to level of difficulty, (2)
? NRC supervisory approval of every examination, and (3) regional oversight activities by the 11
Office of Nuclear _ Reactor Regulation (NRR). The NRC staff has conducted and participated in a number of meetings and workshops since beginnirig the pilot program, and additional workshops are being planned. Secondly, an NRC supervisor is required to review and approve ) 1 every examination, with particular emphasis on the changes recommended by the chief ; examiner, before the examination is administered. Moreover, NRR periodically conducts j examination reviews, which include an assessment of their level of difficulty, to ensure l . consistent regionalimplementation of the NRC's standards and expectations. j L I Comment: An NRC license examiner recommended that any question taken directly from a l bank of questions that is available to the applicants for study should be considered a memory- i level question regardless of the cognitive level at which it is written. l Response: Although this is a valid concem, making such a change' would likely increase the l burden on the facility licensees (because of the increased challenge of writing edditional . questions at the higher cognitive levels) and increase the level of difficulty of the examinations. l Therefore, the staff has made no changes to interim Revision 8 of NUREG-1021 in response to this comment. Comment: An NRC license examiner recommended that NUREG-1021 should contain a ! detailed list of minimum physical security precautions and that facility licensees should be required to list the steps they have taken to ensure examination security, including measures to control any contractors who prepare the examinations. i Response: NUREG-1021 is a guidance document intended to help NRC examiners and facility L' licensees gain a better understanding of the procedures for, and NRC management's expectations regarding, preparation of the initial operator licensing and requalification examinations. The staff has clarified the examination security expectations in final Revision 8 - ; of NUREG-1021, including the control of licensee contractors. In a separate action, the staff issued an Information Notice (IN) to advise power reactor facility licensees of the NRC's l perspective and expectations regarding the integrity of examinations developed by the facility licensees' employees and representatives, Furthermore, based on several security incidents that occurred since beginning the pilot ; examination program, the staff has concluded that applicants, licensees (operators), and facility l licensees may not understand what is meant by the term " compromise," as used in 10 CFR ' 55.49. Therefore, the staff has concluded that it would be beneficial to clarify 10 CFR 65.4g in the final rule so that facility licensees understand their responsibility for maintaining control over the examination process and establish procedures for that purpose, Comment: Two facility licensees suggested that applicants should be allowed to perform some
. or all of the five required reactivity manipulations on the simulator rather than on the actual i reactor.
I Response: The requirement for five significant control manipulations is addressed in 10 CFR 55.31(a)(5), and such a change is beyond the scope of this rulemaking. Therefore, the staff will
-address this comment separately.
12 U
b [ Comment: An NRC license examiner recommended deleting the provision in interim Revision 8 of NUREG-1021 that allows up to 30 percent of the job performance measures (JPMS) on the
- walk-through portion of an operating test to be repeated on walk-through tests given on a later
. day.
Response: The issue of repeating JPMS during successive walk-through operating tests was discussed in SECY-96-206, and the 30 percent limit was included in interim Revision 8 of NUREG-1021 because no limit had existed previously when the NRC was preparing the examinations. The staff has revised its position on this issue in favor of the approach recommended by the examiner because it will improve examination security and be consistent ' with the policy on repeating simulator scenarios. The staff believes that the impact of the - change will be minimal. Comment: One nuclear industry employee commented that requiring facility licensees to i
- prepare and submit an outline of the written examination and operating test in advance of the actual examination is an unnecessary administrative burden. He suggested that the due date for the examination itself be advanced and the requirement for a separate outline be eliminated. )
1 Response: Preparing the outline, particularly for the written portion of the examination, in ! accordance with the guidance NUREG-1021 ensures that the required knowledge and abilities are systematically sampled and evaluated on the licensing examination. Submitting the outline to the NRC for review and approval before preparing the examination itself, enables the NRC to comment and the facility licensee to make the required changes before significant heensee l resources are expended in developing the final examination materials.
)
- In response to this comment and to comments (regarding the submittal of examination
, . materials) from NRC examiners involved with the pilot examinations, the staff is changing the i guidance in interim Revision 8 of NUREG-1021 by advancing the due dates for the examination ,
outline and the proposed examination in order to provide additional time for facility licensees to i resolve any concems the NRC has with those submittals. Comment: An NRC license examiner recommended that NUREG-1021 should contain guidance to ensure that the written examination outlines are systematically developed. He argued that this will minimize the possibility that facility licensees will bias the selection of topics toward those that were emphasized during the applicants' license training program (Another NRC examiner submitted a differing professional view (DPV) regarding the potential for bias in , pare e ami a io outi . Response: The staff agrees that the specific written examination sampling guidance in interim
- Revision 8 of NUREG-1021 does not ensure that the written examination is free of biases that ,
- could affect its validity. Therefore, the staff has revised final Revision 8 of NUREG-1021 to reduce the risk of bias by ensuring that facility licensees prepare the examination outlines in a l systematic manner. ' (The panel that reviewed the DPV concluded that this was an acceptable resolution of the issue.)
13
Comment: NEl and a nuclear industry employee recommended that facility licensees be allowed to use their site-specific task lists instead of NUREG-1122, " Knowledge and Abilities [K/A] Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors," or NUREG-1123, " Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Boiling Water Reactors," to establish the content validity of the examinations. Response: The staff believes that allowing substitution of the NRC's K/A catalogs (which were
- derived from an industry job-task analysis and evaluated by a panel of subject matter experts representing the nuclear industry and the NRC) would be inappropriate and inconsistent with the regulations. Although the Part 55 rulemaking of 1987 generally endorsed the " systems approach to training," Sections 55.41(a), 55.43(a), and 55.45(a) all state that the knowledge, skills, and abilities selected for evaluation on a written examination and an operating test will be identified, in part (emphasis added), from leaming objectives derived from a systematic analysis j of licensed RO and SRO duties performed by each facility licensee. While NUttEG-1262,'- l
" Answers to Questions at Public Meetings Regardir g implementation of Title 10, Code of I
- Federal Regulations, Part 55 on Operators' Licenses," confirmed the NRC's intent that the i
. training program's leaming objectives would become the major source of the licensing examination, it also cautioned that the NRC would not be limited to those leaming objectives. ,
- Moreover, since 1986, NUREG-1021 has encouraged examiners to use the facility licensees' l leaming objectives as a source of topics for the written examination, and it has always required examiners to reference the facility licensees' training materials when developing a licensing i examination. l l
? The systematic sampling procedures for preparing the written' and walk-through examination !
' outlines per interim Revision 8 of NUREG-1021 are designed around the structure of the NRC's !
K/A catalogs and may not be compatible with the facility-specific task lists. As discussed in SECY-96-206, interim Revision 8 of NUREG-1021 contains provisions for facility licensees to j add, substitute, or delete specific knowledge and ability requirements on a case-by-case basis. ' Allowing facility licensees to substitute their entire site-specific task lists for the NRC's K/A catalogs could decrease the level of examination consistency, increase the potential for biasing
- the sample of material covered on the licensing examinations, and increase the level of effort '
for NRC examiners to review the proposed outlines to ensure adequate sampling per Part 55. The staff has. concluded that the current approach of requiring facility licensees to explain deviations from the NRC's K/A catalogs is conservative, consistent, and efficient. Therefore, no changes were made to Revision 8 of NUREG-1021 in response to this comment. < LComment: NEl recommended that the time limit for taking the written examination should be increased from 4 to 5 hours because of the increased percentage of higher cognitive-level questions on the examinations. Response: Feedback regarding the pilot examinations has confirmed that a number of applicants could not complete the examination in the 4 hours currently specified in NUREG-1021. The staff believes that increasing the time limit to 5 hours may be necessary because of the greater focus on improving the plausibility of the test question distractors (i.e., the wrong-1 . answer choices) and the required range of higher cognitive-level questions. 14 4 .e f
E, The staff believes that the' time limit can be increased to 5 hours without affecting the i discrimination validity of the examinations. The nature of the NRC licensing examination is such that allowing sufficient time to demonstrate knowledge is of primary concern. Comment: One facility licensee and a facility employee suggested that the simulator operating test administration procedure in NUREG-1021 should allow other operating assistants, such as a shift technical advisor (STA) and shift manager, to help the applicants during critical evolutions on the simulator when required by the plant's technical specifications and procedures. Response: Because the operating tests are administered one-on-one (i.e., with one NRC examiner assigned to each applicant on the crew), the number of applicants on the crew can normally not exceed the number of NRC examiners dispatched to the site. Interim Revision.8 of NUREG-1021 already makes allowances for the operating crews to be supplemented with additional surrogate personnel up to the minimum RO staffing level required by the facility
- licensee's technical specifications, but it does not address the use of STAS or additional shift managers (i.e., senior operators). Therefore, in final Revision 8 of NUREG-1021, the staff will-allow the use of STAS in accordance with the facility licensee's normal staffing practice for that position.i However, the staff does not intend to permit more than one person to fill a senior operator position during the simulator test because the principal duties of the shift manager position (i.e., assuming the role of the emergency director, performing emergency classifications,' and making protective action recommendations) are normally a part of the operating test for senior operator applicants.
Comment: An NRC license examiner suggested that the requalification examination procedures in NUREG-1021 should include limits on test item duplication between successive examinations. Response: The staff's expectations regarding this issue were previously communicated to the - NRC regional offices, and the staff agrees that the same information should be incorporated in final Revision 8 of NUREG-1021. Specifically, final Revision 8 will limit test item duplication between successive NRC requalification examinations to no more than 50 percent. Comments From NRC Staff 3 I in addition to the public comments discussed above, the staff considered a number of comments, recommendations, and questions from NRC examiners, supervisors, and managers involved in the operator licensing program. Although most of the issues related specifically to the pilot process, some applied to other aspects of the operator licensing program. A number ! of the issues were previously addressed in response to the public comments; the remaining !
. significant comments and responses are summarized below. Several additional minor changes are summarized, as appropriate, in the Executive Summary section of final Revision 8 of ,
NUREG-1021. - Comment: A senior manager suggested that NUREG-1021 should provide facility ! management with the opportunity to appeal to NRC management if the facility has significant concems regarding the staff's application of the examination criteria in NUREG-1021. 15 i I
E .c r Response: The staff agrees that such a provision is appropriate and consistent with the NRC's policy of minimizing regulatory impact when appropriate. ES-201, " Initial Operator Licensing Examination Process," of final Revision 8 of NUREG-1021 has been edited to incorporate such a practice. Comment: A regional supervisor suggested that ES-301, " Preparing initial Operating Tests," of l - NUREG-1021 should better define the NRC's expectations regarding the permissible number of closed-reference questions on the walk-through portion of the operating test.
~ '
Response: The staff agrees that the guidance in interim Revision 8 of NUREG-1021 lacks specificity and may contribute to variations in'the quality and level of difficulty of the operating tests!. Final Revision 8 of NUREG-1021 has been clarified to address the issue. t 16
SECTION 111 - VULNERABILITY REVIEW , At the Commission's direction, SECY-96-206, the rulemaking plan for amendments to the ; examination requirements in 10 CFR Part 55, discussed the pros, cons, and vulnerabilities of proceeding with the revised examination process on an industrywide basis. The SRM that later approved the rulemaking plan directed the staff to monitor the pilot program results and inform the Commission if the staff gained any insights that suggest that the revised examination process should not be pursued. In response to this directive, the staff has reviewed the vulnerabilities (i.e;,' quality and consistency, independence and public perception, examination security, NRC resources, program stability, and examiner proficiency) that were discussed in I the rulemaking plan in light of the staffs experience with the pilot process since SECY-96-206 was issued on September 25,1996. The staff has also reviewed the potential effects on reactor safety. .
- For ease of review, the staff has categorized and addressed the public comments on the proposed rulemaking with respect to the same vulnerabilities. Appropriate cross-references to the FRN for the final rule are provided to minimize the need for repetition. ;
Quality and Consistency of the Examinations J The staffs expectations regarding the quality and level of difficulty of the proposed ;
. examinations have not yet been achieved. As discussed above, the staff has considered a i number of possible reasons for the problems that have been observed, but no definitive l explanations have been identified. Additional public comments related to this vulnerability are addressed in the associated FRN.
~ To maintain acceptable uniformity of examination content, format, level of difficulty, and quality, the staff expects facility licensees to prepare the written examinations and operating tests in , accordance with the procedures and guidelines in NUREG-1021. A number of administrative l criteria and controls have been incorporated in Revision 8 of NUREG-1021 to implement the new examination development process. Those revisions include: limits on the number of test items (e.g., written questions and JPMs) that can be taken directly from the facility licensees' examination banks, the number of test items that can be developed by modifying existing bank items, and the number of newly-developed test items; limits on the repetition of test items from - previous quizzes and examinations given at the facility; guidelines regarding the cognitive level of written examination questions; and improved quality assurance checklists. Revision 8 o' NUREG-1021 discourages facility licensees from using testing methodologies that do not conform with NUREG-1021 but allows them to propose attematives for NRC review. i The significant changes required on many of the proposed examinations suggests that the guidance in interim Revision 8 of NUREG-1021 may not have been understood. However, as noted above, the staff believes that the guidance in interim Revision 8 of NUREG-1021 was sufficiently clear to enable facility licensees to prepare examinations of acceptable quality and consistency, and, therefore, has made only minor revisions to NUREG-1021 since issuing SECY-96-206. The staff has conducted and participated in a number of public meetings and
' workshops in an effort to communicate its expectations to the facility employees who will 17 f
b L prepare the examinations and to obtain feedback regarding the guidance in NUREG-1021. Additional NRC and industry workshops are being planned to address continued concerns
- regarding the quality of the submitted examinations. Moreover, as noted earlier, the staff has asked the industry to address the issue of examination quality.
The revised examination process is dependent upon the ability and judgment of the NRC chief examiners to identify weak test items and to communicate the problems to the facility representatives so that the required corrections are understood and implemented. Although a
. number of the pilot examinations submitted by the facility licensees have not met NRC
- standards for quality, level of knowledge, or level of difficulty, the NRC chief examiners have L directed the facility licensees to revise the examinations so that they would discriminate at the .
j appropriate level. The staff acknowledges that some of the facility-prepared examinations will
. be more difficult than others (yet within an acceptable range of variation) and that some of the variation can likely be traced to individual differences among the NRC examine.rs who review and approve the examinations. Such was also the case when the NRC or its contractors wrote the examinations. However, the staff believes that the variation in the level of difficulty can be maintained within acceptable limits and would be even greater if NRC examiners did not take action to ensure that the submitted examinations are consistent with NRC standards.
'~ To ensure consistent implementation of the guidance in NUREG-1021, an NRC supervisor is required to review and approve every examination, with particular emphasis on the changes J recommended by the chief examiner, before the examination,is administered. Supervisors are also required to periodically evaluate every certified examiner during the conduct of an
- operating test to ensure that the on-site examination activities are being performed as intended.
Furthermore, the NRR staff periodically reviews the licensing examinations given at selected facilities and the implementation of the operator licensing function in each region to verify the
, quality of the examination process and to ensure the consistent implementation of the NRC's
. standards and expectations. The NRR reviews include an assessment of the level of knowledge and level of difficulty of the written examinations and operating tests; each review is 1
documented in a report to the responsible regional Division Director.
' NRR has instructed regional managers and examiners not to approve or administer any
- examination that does not adhere to current NRC standards for content, format, quality, level of knowledge, and level of difficulty. If a facility licensee does not prepare appropriate licensing
- examinations, the revised process provides that the NRC, which has the responsibility to ensure that public health and safety are maintained, will develop the examinations.
Independence and Public Perception The staff implemented a number of measures during the pilot examination process to ensure that an adequate level of independence would be maintained and to minimize the potential for bias during the examination development. GL 95-06, which outlined the pilot examination process in August 1995, restricted all facility employees who played a " substantial" role in
. training the license applicants from participating in the examination development. Questions regarding the meaning of " substantial" prompted the staff to clarify the personnel restrictions in L interim Revision 8 of NUREG-1021, which incorporated lessons leamed during the first 22 pilot
' examinations and was issued for use in February 1997. Interim Revision 8 prohibited facility 18 L.n.m.:...
R instructors from participating in the examination development process if they had spent more than a specified amount of time training the applicants who would be taking the NRC . examination and prohibited any instructor from preparing written examination questions related to those topics in which the instructor had provided training. The personnel restrictions in the i GL and interim Revision 8 supplemented a related provision, which, for purposes of maintaining examination security, barred anyone who had knowledge of the contents of the NRC ! c examination from further training-related activities involving the license applicants. The latter
. provision had been adopted from the requalification examination program but did not address the separate but equally important perception that the people.who were responsible for training the applicants, if allowed to prepare tlee examination, could bias its content or level of difficulty.
The nuclear' power industry has argued t' hat the applicants' instructors should be permitted to prepare the licensing examinations, just as the instructors who perform the requalification ! training are permitted to prepare the requalification examinations required by 10 CFR 55.59(a).
~
As noted in the public comments above and in the associated FRN, the industry has indicated ! that the additional personnel restrictions implemented during the pilot examination process ) have unnecessarily increased the facility licensees' administrative burden in preparing the - examinations and forced facility licensees to divert their best-qualified instructors from their ! primary duty of training the license applicants or to use less-qualified instructors to prepare the J licensing examination, with a possible decrease in quality. ; The staff has evaluated the industry's concems in light of the other public comments related to .
. this vulnerability and the other changes that are being made in final Revision 8 of NUREG-1021. Therefore, as noted above in response to the public comments, the staff has removed the restriction that prohibits instructors from preparing any written examination questions based i solely on the amount of time they spent training the license applicants. This change should - l provide licensees with increased flexibility in managing their resources and possibly reduce their costs without adversely affecting the independence of the examination process. For , example, if a facility licensee has two instructors who equally shared the responsibility for training the license applicants, they will now be permitted to prepare the examination questions related to the topics they did not teach, whereas, they previously would both have been restricted from any examination development. As discussed above in response to the public comments on the NUREG, final Revision 8 will also include explicit guidance for systematically sampling the important, safety-related K/As from NUREG-1122 or NUREG-1123 when preparing the written examination outline. This change should reduce the chances that the written examination might be biased in favor of those topics that were emphasized during the training program and away from other important topics that received less attention. The !
operating test outline will continue to be independently developed by an individual who was not 1 involved in the applicants' license training. These measures should provide sufficient assurance that the examinations are prepared in an impartial manner. The staff maintains, for the reasons that follow, that the revised personnel restrictions are needed to limit the potential for bias in the examination development process. The personnel restrictions were an important component of the pilot examination process. The conclusions of the acceptability of the pilot process, such as the comparability of examination passing rates and test scores, are linked to the 19
1 independence of the test developer. To eliminate controls on who can write examination material at this point in the transition process could undermine the staff's conclusions l regarding the effectiveness of the pilot process. l The personnel restrictions maintain greater consistency with other professional licensing and certification examinations (e.g., medical, !egal, aircraft, and engineering), which are prepared by an independent body. l l 1 l The initial licensing examinations are considered "NRC" examinations upon approval, even if they were prepared by the facility licensee, whereas, the requalification examinations required by 10 CFR 55.59(a) are classified as " facility" examinations. Unlike the requalification examinations, the NRC's initial licensing examinations are not l a part of the facility licensee's training program. The initial examination is the cornerstone of the NRC's licensing decision. , On June 18,' 1996, the staff briefed the Commission on the results of the pilot program, i noted its concerns regarding the potential for facility employees to bias the !- ] examinations, and discussed the personnel restrictions that the staff had implemented to mitigate those concerns. The Commission acknowledged the vulnerability in this L area, including the concern that instructors could teach to the test. Eliminating the personnel restrictions could affect the public perception regarding the revised process and indicate that the NRC is not concerned about the potential for l examination bias. l With regard to the issue of public perception, The Washington Post reported, shortly after the staff disclosed its intent to revise the examination process in March 1995, that consumer advocates were concerned that allowing facility licensees to prepare the examinations might endanger reactor safety because the examinations might be less rigorous than those prepared by the NRC. However, in the three years since the original proposal, the NRC has afforded the public a number of opportunities to comment on the revised examination process, but none have been received. For example, Generic Letter 95-06 and its supplement announced and ! later summarized the results of the pilot examination program; comments were invited, but none were received. When a Federa/Registernotice (61 FR 6869) dated February 22,1996, l announced the availability of draft Revision 8 of NUREG-1021 and solicited comments, only NEl and two facility licensees responded. And, when the proposed rule was published in the Feders/ Register (62 FR 42426) on August 7,1997, the 13 respondents included NEl, five l nuclear utilities and one employee, four NRC or contract examiners, one non-power reactor ! facility licensee, and the State of Illinois, but no consumer advocates or members of the general [ public. Concerns regarding the potential for conflict ofinterest have also prompted the NRC to review the clarity of 10 CFR 55.49. The regulation encompasses not only activities like cheating and lapses in security but also activities that compromise the integrity or validity of the examination (e.g., noncompliance with the criteria designed to limit the potential for bias in the selection of
. topics to be evaluated on the written examination). Therefore, the NRC has concluded that it would be beneficial to amend 10 CFR 55.49 in the final rule to ensure that applicants, 20 t
l licensees, and facility licensees, understand the scope of the regulation. If the NRC determines that a facility licensee has biased the scope, content, or level of difficulty of an examination to l' enhance the chances that its applicants would pass the examination, the NRC will utilize its enforcement authority including, as warranted, civil penalties, orders against the individuals involved, and, charging the individuals involved with deliberate misconduct pursuant to 10 CFR 50.5.. In summary,'the pressures to conduct the examinations on schedule and the reduced . ! involvement of the NRC in the examination development process could lead to examinations j l that reflect the biases of the facility licensee and that discriminate at a lower level than is ; currently the norm. However, a number of process requirements are in place to reduce the
~
biases: the revised examination process includes restrictions on personnel, the systematic selection of topics to be tested on the written examination, independent NRC review, NRC- , directed revisions, NRC approval of the examinations, and independent NRC a.dministration of l the operating tests that should provide sufficient safeguards to reduce the possibility of biased l licensing decisions. . The pilot examination results, to date, suggest that these measures, in combination with the NRC examiners' increased focus on the psychometric quality of the written questions, will be effective at maintaining the discrimination validity of the licensing examinations. ' A' summary of the public comments related to this vulnerability and the staff's responses are included in the associated FRN. Examination Security As noted in SECY-96-206, the staff expects that the majority of facility licensees will continue to ,
- s. maintain proper examination security. Nevertheless, the revised examination development process may provide greater opportunities for examination compromise (as that term is clarified
, in an amendment to 10 CFR 55.49) than the traditional process in which fewer facility employees had knowledge of and access to the examination for a shorter period of time. Since beginning the pilot examination program, there have been a number of instances in which facility licensees have (1) failed to exercise positive control of the examinations by not locking ,
1 the room or container in which the examin'ations were being prepared or stored; (2) failed to ' control the activities of personnel involved in preparing or reviewing the examinations, as required by the signed security agreements, or to restrict unauthorized personnel from gaining access to the examinations; and (3) lost control of the examinations while printing or making copies.-
- NRC examiners are required by Revision 8 of NUREG-1021 to be attentive to examination security measures, to review the NRC's security expectations with the facility licensee at the L time the examination arrangements are confirmed, and to report any security concems to NRC management. If the NRC determines during its preparation that an examination may have been compromised, it will delay the examination until the scope of the potential compromise is determined and measures can be taken to address the integrity and validity of the examination, if the compromise is discovered after the examination has been administered, the NRC will delay the licensing action for the affected applicants until the staff can make a determination regarding the validity of the examination. if the compromise is not discovered until after the 21
k licensing action is complete, the NRC will reevaluate the licensing decision pursuant tc 10 CFR 55.61(b)(2) if it determines that the original licensing decision was based on an invalid examination. As a separate measure, the NRC will take enforcement action, when appropriate, and consult with the Office of Investigations in cases involving the possibility of personal wrongdoing. The staff has taken a number of actions since the issuance of SECY-96-206 to increase the NRC examiners' and the facility licensees' sensitivity to examination security issues and to mitigate the vulnerability in this area. Those actions include (1) a revision of NUREG-1600,
" General Statement of Policy and Procedure for NRC Enforcement Actions," to address enforcement action against the license applicants, Part 55 licensees, and Part 50 licensees that are subject to the requirements in 10 CFR 55.49; (2) a change in the final rule, based on 4
comments on the proposed rule, to clarify 10 CFR 55.49 and to require facility licensees to establish procedures to control examination security and integrity; (3) further ciprification and elaboration on the guidance in NUREG-1021 about examination security; and (4) the issuance of an IN to advise power reactor facility licensees of the NRC's perspective and expectations regarding the security of examinations developed by the facility licensees' employees and representatives. NRC Resources . When the staff developed the pilot examination process, it estimated that it would take approximately 370 examiner-hours to review and otherwise prepare for, administer, grade, and document an average operator liconsing examination. Although a number of the proposed examinations consumed more NRC resources than expected to achieve NRC's standards, the burden was generally offset by other examinations that required less effort to review and revise.
~ Therefore, the NRC staff continues to believe that the use of contractors in the operator licensing program (except for the generic fundamentals examination) can be eliminated and that the revised initial operator licensing and requalification inspection programs can be implemented with the existing NRC staff. Before initiating the pilot examination and transition program at the beginning of FY 1996, the NRC spent approximately $3 million per year for contractor assistance to prepare and administer initial examinations and assist with requalification inspections. During FY 1996, when approximately 40 percent (27/68) of the examinations were voluntarily prepared by facility licensees, the NRC spent approximately $1.6 million for contractor assistance in those areas. In FY 1997, facility participation increased to include approximately 75 percent (41/55) of the examinations, and the NRC's spending on
. contractor assistance for the licensing examinations and requalification inspections decreased to approximately $0.5 million (approximately 0 full-time equivalents (FTEs)). During FY 1997, the NRC expended approximately 16.8 staff FTEs to conduct all the site-specific initial operator licensing examinations, plus 3.8 staff FTEs for generic operator licensing activities. The licensed operatur requalification inspection program consumed approximately 4.5 additional staff FTEs (plus a minimal level of contractor support), bringing the total resources used for the
. operator licensing program in FY 1997 to approximately 28.1 FTEs. This actual resource l burden is generally consistent with the budget estimates prepared before beginning the pliot examination program and supports the staff's conclusion that the revised examination process is an efficient altomative to the traditional NRC-prepared examinations. The NRC resource burden should decrease further if and when the rule is implemented (which would cause the 22
( _ . . _ _ . _ . . . . . _
1 L number of NRC-prepared examinations to decrease) and the quality of the facility-prepared examinations improves. However, experience during the pilot program has shown that the quality of the facility-prepared examinations can very widely, making it di#icult to predict the amount of time necessary for the
' NRC to review and revise any particular examination in order to meet NRC standhrds.
Although the staff had expected the quality of the examinations to improve over time, some of the proposed examinations continue to require more changes than anticipated. The added examiner workload could increase the possibility of lower examination quality, raise the NRC's cost, and affect the staffs ability to satisfy the facility licensees' needs for licensing examinations. As noted in SECY-g7-07g, NRR has issued a memorandum to the regional administrators emphasizing the importance of (1) assigning adequate resources to carry out the operator licensing task, (2) completing a thorough review of every facility-prepared examination, and (3) not administering any examination that does not meet NRC standards for quality and level of di#iculty. Furthermore, as noted above, the staff has asked the industry to address the issue of examination quality, which should reduce the need for changes and the cost of preparing the examinations.
; The resource burden on facility licensees is addressed in the associated FRN and regulatory analysis.
Program Stability As noted in SECY-g6-206, when it started the pilot program, the sta# intended that the format, content, and level of difficulty of the facility-prepared examinations would remain consistent with previous NRC-prepared examinations so that the transition from NRC-prepared to facility-
,, prepared examinations would be imperceptible to the license applicants. Although the staff has made a number of changes in the examination criteria and guidelines in Revision 8 of NUREG-1021, it has made every effort to maintain consistency. Of the three parameters, level of difficulty is the most subjective and most complex to define and control. However, based on the overall examination results (i.e., the slight decline in the passing percentage and average written examination grades discussed above) since starting the transition process, it appears that the level of difficulty of the facility-prepared examinations, revised as directed by the NRC,
~ is consistent with those prepared by the NRC. Considering the historical fluctuation in the average examination passing rates and the other factors (e.g., training program quality and screening of applicants by facility licensees) that could be responsible for some or all of the observed decline, the staff has concluded that any increase in the level of difficulty is not
. significant.~
Examiner Proficiency < As noted in the responses to the public comments summarized in the FRN, the staff believes that by participating in the revised examination process, NRC examiners will maintain their proficiency.- An NRC examiner will review and approve every examination before it is administered to ensure that it conforms to the guidelines in NUREG-1021 regarding content, format, quality,'and levels of knowledge and Wfsulty. NRC examiners will also continue to independently administer and grade both the dynamic simulator and the plant walk-through 23
l portions of the operating tests. Because NRC examiners will be administering all of the operating tests, the revised process will enable the examiners to accrue more experience in a shorter period of time and to maintain their proficiency. The smaller pool of all NRC examiners may also improve the consistency of the operating test evaluations and the licensing decisions. New NRC license examiners will still be required to complete a standardized training program, including the development of a complete written examination and operating test, as part of their certification process. Once certified, examiners are required to mdatain their qualificate by 1 periodically conducting examinations and attending refresher training. NRR has revised the refresher training syllabus to focus more attention on reviewing, critiquing, and approving written examination questions. As directed by the Commission in the SRM of December 17, 1996, the NRR staff will ensure that each region writes at least one initial operator licensing examination per calendar year. Furthermore, as directed by the SRM, the staff intends to conduct examiner training conferences at intervals not to exceed 24 months tt le last such conference was convened in October 1997. Routine management oversight activities performed by the regions and NRR should provide additional assurance that examiner proficiency is being maintair.ed. Regional supervisors are required to review and approve the examination changes recommended by their chief examiners and they periodically evaluate every certified examiner during the conduct of an, operating test. Moreover, the NRR staff periodically reviews the licensing examinations given at selected facilities and the implementation of the operator licensing function in each region to verify the quality of the examination process and to ensure the consistent implementation of the NRC's standards and expectations. Reactor Safety As noted above and in the responses to the public comments summarized in the FRN, the staff l believes that the revised examination process will improve the staff's ability to focus on operator j performance because NRC examiners will be administering all of the operating tests. The staff ; acknowledges thatthe total number of facility operating procedures reviewed in the process of 1 examination development may decrease under the revised method. However, NRC examiners ; are still expected to review and identify discrepancies in the procedures that will be exercised ! during the walk-through JPMs and the simulator scenarios. Because only staff examiners will be administering the operating tests and reviewing all of the applicable procedures, tre revised process will enable the examiners to accrue more experience in a shorter period of time. < The staff also acknowledges that the revised examination process places an extra burden on i the facility licensees' training staffa and may divert resources from their primary training l activities so that they can prepare the licensing examinations. If a facility licensee places insufficient resources on either training or testing, the quality of its proposed licensing examinations or the passing rate on those examinations would most likely suffer. Although many of the facility-propared examinations have required numerous revisions to achieve NRC e - quality standards, the examination results, to date, aw consistent with the results on previous NRC-prepared examinations, suggesting that the quality of the facility licensees' training programs has not been affected.- 24
The NRC will continue to set the standard for operator performance by ensuring that facility licensees maintain appropriate standards for examination quality and level of difficulty. The staff will remain actively involved in the operator licensing process by reviewing and approving
' every written examination and operating test before it is administered. Unacceptable examinations have been and will continue to be revised and delayed, if necessary, until the examinations are upgraded. . The overall results of the facility-prepared examinations support the staff's conclusion that the licensing decisions that have been made on the basis of those examinations are as valid and conservative as those made using the traditional examination process. The fact that the facility . licensees are preparing the examinations with NRC review and approval, should have no negative effect on the safe operation of the plants.
6 l i 25 i.
I l i i
. l Enclosure 2 !
i l FederalRegister Notice (Including Statement of Consideration)
1 [7590-01-P] NUCLEAR REGULATORY COMMISSION 10 CFR Part 55 RIN 3150-AF62 Initial Licensed Operator Examination Requirements AGENCY: Nuclear Regulatory Commission.'
. ACTION: Final rule. -
i
SUMMARY
- The Nuclear Regulatory Commission (NRC) is amending its regulations to require j all nuclear power facility licensees to prepare, proctor, and grade the written examinations and
, prepare the operating tests that the NRC uses to evaluate the competence of individuais ;
applying for operator licenses at those plants. The amendment requires facility licensees to !
- j. ' submit, upon approval by an authorized representative of the facility licensee, each examination and test to the NRC for review and approval. The amendment preserves the NRC's authority to prepare the examinations and tests, as necessary, if the NRC has reason to question a facility licensee's ability to prepare adequate examinations. In addition, the NRC will periodically invoke this authority in order to maintain the proficiency of its own license examiners. The Commission is concerned with examination integrity; therefore,10 CFR Part 55.49 has been amended to clarify the Commission's intent that facility licensees establish procedures to control examination security and integrity.
EFFECTIVE DATE: This final rule is effective on (insert the date 180 days from date of 4
i 2
- publication in the Federal Register).
i FOR FURTHER INFORMATION CONTACT: Siegfried Guenther, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, telephone:
' 301-415-1056; e-mail: sxg@nrc. gov.
SUPPLEMENTARY INFORMATION: Background I Section 107 of the Atomic Energy Act (AEA) of 1954, as amended, requires the NRC to determine the qualifications of individuals applying for an operator's license, to prescribe uniform conditions for licensing such individuals, and to issue licenses as appropriate. Pursuant to the AEA,10 CFR Part 55 requires applicants for operators' licenses to pass an examination that satisfies the basic content requirements specified in the regulation. Although neither the J AEA nor Part 55 specifies who must prepare, proctor, or grade these examinations, the NRC has traditionally performed those tasks itself or through its contract examiners. The NRC and its contract examiners have used the guidance in NUREG-1021, " Operator Licensing j
-Examination Standards for Power Reactors," once titled " Operator Licensing Examiner I
Standards," to prepare the initial operator sconsing examinations. This document has been -I revised as experience has been acquired in preparing the examinations. The current version is designated Revision 8.' { n l 1.- Copies are available for inspection or copying for a fee from the NRC Public Document
' Room (PDR) at 2120 L Street NW, Washington, DC 20555-0001; the PDR's mailing address is ;
ts < 3
- . ; 'In accordan'ce with 10 CFR 170.12 (i), the NRC's staff and contractual costs are
- recovered from facility licensees that receive examination services. In Fiscal Year (FY) 1995, the NRC spent approximately $3 million on contractor support for the preparation and ,
administration of the initial operator licensing examinations and for support of requalification program inspections. On March 24,1995, in SECY-95-075, " Proposed Changes to the NRC l Operator Licensing Program," the staff advised the Commission of its intent to eliminate the use
~
o[ contractors by allowing facility licensees to prepare the examinations. On April 18,1995, the Commission approved the NRC staff's proposal to initiate a transition process to revise the operator licensing program and directed the NRC staff to ' carefully consider experience from pilot examinations before fully implementing the changes. ! On August 15,1995, the NRC issued Generic Letter (GL) 95-06, " Changes in the Operator i
. Licensing Program,"' outlining the revised examination development process and soliciting ;
, volunteers to participate in pilot examinations to evaluate and refine the methodology.
' Betteen October 1,1995, and April 5,1996, the NRC reviewed and approved 22 !
. operator licensing examinations, including both the written examinations and the operating :
tests, prepared by facility licensees as part of a pilot program. These examinations were !
' prepared ~using the guidance in Revision 7 (Supplement 1) of NUREG-1021' and the additional
; guidance in GL 95-06. !
' The results of the pilot examinations were discussed in SECY-96-123 " Proposed Changes to the NRC Operator Licensing Program," dated June 10,1996. Based on the results ,
of the pilot program, the NRC staff recommended that the Commission approve the Mail Stop LL-6; telephone is 202-634-3273; fax is 202-634-3343. Revision 8 of NUREG-1021 is also available for downloading from the internet at http://www.nrc. gov for a limited period of time.
E
.c 4
implementation of the new examination process on a voluntary basis until rulemaking could be completed to require all power reactor facility licensees to prepare the entire initial operator licensing examination and to proctor and grade the written portion of the examination. On July
. 23,1996, the Commission authorized the staff to continue the pilot examination process on a voluntary basis and directed the staff to develop a rulemaking plan to justify the changes that 1
L would be necessary to 1.0 CER Part 55. The Commission also directed the staff to address a number of additional items (e.g.,' pros, cons, and vulnerabilities) regarding the r'evised examination process to facilitate a Comtr:ission decision on whether to implement the revised
. process on an industrywide basis.
With Commission approval, the NRC staff resumed condu'eting pilot-style examinations on August 19,1996, and by the end of FY 1997 had reviewed, approved, and administered 42 additional examinations that were developed by facility licensees. This raised the total number of examinations completed using the pilot process to 68. ! On September 25,1996, the NRC staff forwarded the rulemaking plan and a response' to the additional items to the Commission in SECY-96-206, "Rulemaking Plan for Amendments to 10 CFR Part 55 to Change Licensed Operator Examination Requirements." SECY-96-206 addressed a number of areas (i.e., quality and consistency, independence and public _ perception, examination security, NRC resources, program stability, .4nd examiner proficiency) in which the NRC could be more vulnerable under the revised examination process and the 1 measures that the NRC has taken to manage the vulnerabilities. On December 17,1996, the Commission directed the staff to ' proceed with the proposed rulemaking. The staff forwarded j the proposed rule (SECY-97-079, " Proposed Rule - Initial Licensed Operator Examination ! Requirements") to the Commission on April 8,1997, and on June 26,1997, the Commission
\
approved publication of the proposed rule for a 75-day comment period. The proposed rule j 4 i i 1 1
i 5 was published in the Federa/ Register (62 FR 42426) on August 7,1997, and, when the public comment period expired on October 21,1997,11 comment letters had been received. Two additional comment ietters arrived after the expiration date but were also considered in the development of the final rulemaking. ! Discussion l . The pilot examinations demonstrated that the revised process, under which facility [ licensees prepare the written examinations and operating tests, is generally effective and efficient. ' From the time the pilot program began in October 1995 through the end of September 1997, the NRC staff reviewed, approved, and administered a total of 68 examinations that were s voluntarily developed by facility licensees under the pilot examination and transition program. l , Facility licensees prepared the written examinations and the operating tests, proctored
- the written examinations, and graded the written examinations using the guidance provided by the NRC in GL 95-06 during the early stages of the pilot program, and subsequently in interim Revision 8 of NUREG-1021, " Operator Licensing Examination Standards for Power Reactors."
NRC examiners thoroughly reviewed the examinations and tests to determine if they were consistent with NRC standards, directed facility licensees to make whatever changes were l l necessary to achieve NRC stan'dards if the submitted examinations and tests were deficient, and approved the examinations and tests before they.wers administered. NRC examiners ibpendently administered all of the operating tests, reviewed the written examination grading,
' and made the final licensing recommendations for approval by NRC management.
l Comments from the NRC chief examiners who evaluated the pilot examinations indicate I that the quality and le"el of difficulty of the licensee-prepared examinations (when modified as i
L i 4 l' , i 6 i directed by the NRC) are generally comparable to the examinations prepared by the NRC (i.e.,- ; by the staff or NRC contractors). The passing rate on the 68 pilot-style examinations administered through the end 'of FY 1997 was only slightly lower than the passing rate on the l power reactor licensing examinations administered during FY 1995, the last year in which all examinations were prepared by the NRC. However, considering the historical fluctuation in the average examination passing rates and the other factors (e.g., training program quality and screening of applicants by facility licensees) that could be responsible for some' or all of the observed difference, the Commission has concluded that the observed change in the passing rates is not significant. The average grades on the facility-prepared, NRC-approved written I examinations were also comparable if slightly lower than the grades on examinations prepared l by the NRC during FY_1995. These data support the conclusion that the facility-prepared l examinations are discriminating at a conservative and acceptable level and that the revised examination process is effective. Therefore, the fact that the facility licensees are preparing the examinations with NRC review and approval, should have no negative effect on the safe operation of the plants. Although the, NRC-approved examinations were comparable to NRC-prepared examinations, essentially all of the examinations prepared by facility licensees required some , changes subsequent to NRC review, and many of the examinations required significant rework. The NRC had originally believed that, with training and experience, the industry would quickly gain proficiency in preparing the examinations, but the overall quality of the examinations 0' submitted to the NRC during the pilot program did not improve as expected over time. Although l approximately half of the 17 facility licensees that had prepared more than one examination by the end of FY 1997 did maintain or improve the quality of their second or third examination submittals, the quality of the other facility licensees'second or third examinations was lower. I l l 4
]
p 1 [. V
- 7
' Consequently, the NRC has asked the industry to address the issue of examination quality and determine the need for additional training on examination development. The NRC will continue l to (1) direct facility licensees to revise the examinations as necessary to achieve an acceptable level of quality and discrimination, (2) delay those examinatior.1s that do not meet NRC standards by the scheduled administration date, (3) oversee the regionalimplementation of the operator licensing process to ensure consistency. (4) address significant deficiencies in the submitted examinations as licensee performance issues in the examination rep' orts, as j appropriate, and (5) conduct or participate in workshops, as necessary, to ensure that facility licensees understand the NRC's examination criteria.
With regard to the efficiency of the revised examination process, the pilot examination l experience to date supports the conclusion that the average industry cost will not differ ! significantly from the cost of NRC-prepared examinations. Comments from the industry reflect that the cost for some facility licensees to prepare the examinations was higher than it was for l NRC-prepared examinations; however, other licensees prepared good quality examinations at l lower cost than the NRC. The industry generally attributed the higher cost to the revised examination and administrative criteria under the pilot examination process. Although the NRC acknowledges that the revised criteria contribute somewhat to the elevated cost, many of the variables that affect the quality and, consequently, the cost of the examination will be under the facility liceneees' control and can present an opportunity for cost savings. For example, facility licensees will be able to manage the size and quality of their examination banks and the training and experience of the personnel they select to write their licensing examinations. The revised examination process puts facility licensees in control of developing the examinations and holds them responsible for their quality. If a facility licensee submits a good quality examination, it is likely to save resources despite the additional admink;trative criteria; however, if the facility ( f L
i 8 i licensee submits an examination that requires many changes, it will likely cost more than if the NRC had prepared the examination. l i Comments from the NRC chief examiners who worked on the pilot examinations ! indicate that the average amount of time spent reviewing and revising the facility-prepared examinations was generally consistent with the estimates developed before starting the pilot
. program. Although a number (approximately 20 percent in FY 1997) of the examinations l required significantly more NRC effort than originally anticipated to bring them up to the NRC's standards, the resource burden was generally offset by other examinations that required less -
effort to review and revise. Therefore, the Commission believes that the original objective of I .. the pilot examination and transition program has been achieved. The program has demonstrated that the NRC can eliminate its use of contractors in the operator licensing program (except for.the generic fundamentals examination) and can conduct the initial operator licensing and requalification inspection programs with the existing NRC staff. l l - I With regard to examination security, applicants, licensees (operators) and facility licensees, are reminded that 10 CFR 55.49 prohibits them from engaging in any activity that i i compromises the integrity of any application, tests, or examination required by 10 CFR 55. The 4 l
- j. Commission considers an examination to be " compromised" if any activity occurs that could affect the equitable and consistent administration of the examination, regardless of whether the !
I activity takes place before, during, or after the examination is administered. Based on several ! g -. security incidents that occurred since beginning the pilot examination program, the NRC has i concluded that applicants, licensees, and facility licensees may not understand that the provisions of 10 CFR 55.49 pertain to more than just those activities involving the physical administration of an examination. Therefore, the Commission has determirad that it would be beneficial to amend 10 CFR 55.49 in the final rule to clearly state the expectation that facility l L a
9 licensees establish procedures to control examination security and integrity. Revision 8 of NUREG-1021 identifies a number of security provisions that facility licensees should consider when establishing their procedures. Consistent with NUREG-1021, facility employees with specific knowledge of any NRC examination before it is given may not communicate the examination contents to unauthorized individuals and may not participate in any further instruction of the students scheduled to take j
. I the examination. Before they are given access to the examination, the facility employees are expected to sign a statement acknowledging their understanding of the restrictions and the ;
l potential consequences of noncompliance. When the examinations are complete, the same
~
employees are expected to sign a post-examination statement certifying that they have not knowingly compromised the examination. In addition to the restrictions on personnel, NUREG-1021 also discusses a number of physical security precautions, including simulator considerations and protecting and mailing the examination materials. NRC examiners are expected to be attentive to examination security measures, to review the security expectations with the facility licensee at the time the examination arrangements are confirmed, and to report any security concems to NRC management. If the NRC determines during its preparation that an examination may have been compromised, it will ! delay the examination until the scope of the potential compromise is determined and measures can be taken to address the integrity and validity of the examination. Pursuant to 10 CFR 55.51, the NRC must make a determination prior to issuing a license that the test or examination is valid, meeting the requirements of the Atomic Energy Act and the Commission's regulations. If the compromise is discovered after the examination has been administered, the NRC will delay the licensing action for the affected applicants until the staff can make a determination regarding the validity of the examination. If the compromise is not discovered
~. , i 10 until after the licensing action is complete, the NRC will reevaluate the licensing decision, if it l determines that the original licensing decision was based on an invalid examination, and take I' appropriate action' pursuant to 10 CFR 55.61(b)(2). 4 As a separate measure, the NRC will evaluate every potential examination compromise t '
- to determine if a violation of 10 CFR 55.49 has occurred. The Commission is modifying NUREG-1600, " General Statement of Policy and Procedures for NRC Enforcement Actions,"
~ (Enforcement Policy) to cover instances where a compromise occurs but measures are taken to remedy the situation that created the compromise before the examination is administered. A compromise that is discovered after an examination is given would be considered a significant regulatory concem and categorized at least at Severity Level lli. The NRC intends to utilize its full enforcement authority including, as warranted, civil penalties and orders against persons
. who (1) compromised examination integrity in violation of 10 CFR 55.49, (2) committed deliberate misconduct in violation of 10 CFR 50.5, or (3) provided incomplete or inaccurate infctmation to the NRC in violation of 10 CFR 50.9. In addition, cases involving willful violations l
_ _will be referred to the Department of Justice. ; The Commission has reviewed the vulnerabilities and costs associated with the revised L examination process and considered the measures that the NRC staff has taken to mitigate the vulnerabilities. With regard to examination quality and level of difficulty, the Commission acknowledges that the effectiveness of the revised examination process is centingent on the NRC ' staff $ review of the facility-proposed examinations to ensure that NRC standards are i achieved.' The Commission has concluded, based on the results of the pilot examination I program, that the controls implemented by the NRC -staff will provide reasonable assurance that the examinations that are administered to the license applicants will provide a valid and consistent basis upon which to make the licensing decisions. The Commission also realizes i La
11 - W
- that the frequency of examination security incidents and the risk of undetected compromises may increase. However, the Commission is confident that the compensatory measures discussed above will sufficiently control the vulnerability in this area. The Commission is aware that the staffs original expectation that facility licensees would eventually realize cost savings
~under the revised process as they gain proficiency in preparing the examinations has not yet been realized. However, the Commission has concluded that neither the increased
. vulnerabilities nor the absence of clear industry cost benefit provides sufficient ' basis for
. discontinuing the revised examination process. The Commission also finds that the revised examination process is more consistent with the NRC's other oversight programs because it requires NRC examiners to review materials prepared by the facility licensees. The revised process enables NRC examiners to focus more on the psychometric quality of examinations (e.g., the cognitive level at which the questions are written and the plausibility of the distractors or wrong answer choices) prepared by the facility licensees than on the technical accuracy of .
the examinations, which was their primary focus when the examinations were prepared by NRC contractors. This shift in the NRC examiners' focus, which is made possible by the revised examination development process, has the potential to improve the overall quality of the licensing examinations. Summary of Public Comments The 75-day public comment period began when the notice of proposed rulemaking was published in the Federal Register (62 FR 42426) on August 7,1997, and closed on October 21, 1997, The notice (FRN) requested public comment on the proposed rule, on the y
i l 12 i implementation guidance in interim Revision 8 of NUREG-1021, and on the following two questions:
- 1. _ Are there portions' of the operator exams that are common to all licensees, and would, t' arefore, be more efficiently developed by the NRC7
- 2. Is the conclusion in the regulatory _ analysis correct that it would be less costly for each licensee to prepare its own initial operator examinations to be reviewed, revised, and ~ J
. administered by the NRC, than to have one NRC contractor prepare these exa'ms for all licensed operators with the costs to be reimbursed by licensee fees?
The NRC received 13 comment letters on the proposed rule; two of the letters arrived after the comment period closed, but they were considered nonetheless. The respondents
~ included three NRC examiners, one contract examiner, five nuclear utilities and one utility y employee, one nonpower reactor facility licensee, the State of Illinois, and the Nuclear Energy institute (NEI), which submitted its comments on behalf of the nuclear power industry. Copies of the public comments are available in the NRC Public Document Room,2120 L Street, NW i
(Lower Level), Washington, DC, and on the internet at "http://ruleforum.llnl. gov /cgi-bin / j i rulemake? source =OE_ PRULE". I Seven of the respondents (three NRC examiners, one contract examiner, one utility ' employee, one nonpower facility licensee, and the State of Illinois) recommended that the rule ! change be disapproved. Five of the industry respondents (NEl and four utilities) supported the j l
. rule change; however, one utility endorsed NEl's comments but stated that it did not agree with . !
the proposed rule in its present form. NEl and two of the utilities stated that they would rather - continue with a voluntary program, but 'they would support mandatory participation with the rule change rather than retum to the previous process under which NRC contractors wrote most of ! the examinations. ] l 4
- [.
t i
y L 13 Those comments related to the two specific questions raised in the proposed rulemaking, and those that have a direct bearing on the rule are discussed below. The comments are categorized as they relate to the vulnerabilities discussed in SECY-96 206 (i.e., , quality and consistency; independence and public perception; security; NRC resources;and examiner proficiency) and reactor safety; there were no comments related to program stability. One NRC examiner, NEl, four of the utilities, and the utility employee also provided
~
specific comments and recommendations regarding the implementation guidance in interim j Revision 8 of NUREG-1021. Those comments are addressed in Enclosure 1 of the Commluion (SECY) paper associated with this rulemaking. A copy of the SECY is available in , m j the NRC Public Document Room, on the intemet at http://www.nrc. gov, or from Siegfried Guenther, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commission, at 301-415-1056 or e-mail at sxg@nrc. gov.
,' j i
, Comment: With regard to the first specific question included in the proposed rulemaking,2 of the 13 respondents (NEl and one utility) stated that all of the common material i
is already included in the generic fundamentals examination (GFE) and that the remaining elements are best covered as part of the site-specific examination. I Response
- lt appears that the current allocation of topics between the GFE and site- >
specific written examinations is generally perceived to be an efficient method of covering the topics required by 10 CFR 55.41 and 55.43. Therefore, the Commission finds no basis for changing the process to have the NRC separately develop portions of the initial examination that would be common to all facilities.' , Comment Seven of the 13 respondents (NEl, two utilities, a utility employee, and three
r s 14 examiners) directly or indirectly addressed the second specific question in their letters. NEl'and one utility stated that the revised examination criteria in interim Revision 8 of NUREG-1021 , have increased the level of effort and will result in higher licensing fees regardless of who
; prepares the examinations' However,'NEl and another utility agreed that comparing the cost of utility-prepared examinations to those prepared by the NRC is difficult, but they concluded that it should be less costly for utilities to prepare the examinations than to have the NRC prepare them under the same criteria.
NEl also stated that the relative cost of the two examination processes should not be the only factor in deciding whether to proceed with the rulemaking. NEl indicated that preparing higher cognitive level questions requires detailed plant knowledge, better provided by facility licensees, and that the revised process (which has eliminated the use of NRC contractors to administer the operating tests) will allow NRC staff to evaluate each applicant without relying on third-party observers. Two NRC examiners, one contract examiner, and a utility employee asserted that the i facility licensees' cost has gone up under the revised examination process. They cited various reasons for the increased cost, including training personnel to write the examinations and then restricting them from training the applicants, and upgrading equipment to maintain examination security. The NRC examiners based their comments on feedback from facility training 4 personnel; one examiner indicated that it took facility licensees an average of 700 hours to prepare each examination. The utility employee stated that the rule change will simply transfer
- the cost of contractors from the NRC to the utilities.
Response: The NRC acknowledges that the pilot examination administrative criteria in particular (e.g., the restrictions on facility training personnel and the need to document the source of the test items) have probably caused the cost of preparing the examinations to be
2' b 15 somewhat higher than it would have been if fbcility licensees had been allowed to prepare the examinations using the same criteria that applied to the NRC and its contractors before starting j l the pilot program. However, when the NRC first developed the revised examination process, I l with its additional administrative criteria, the NRC still believed that the cost for facility licensees j
.to prepare the examinations would be offset by the reduction in the licensing fees and that a 1 . . L. ;
cost devings could be realized as facility licensees' gained experience with the process. Many
'of the facility licensees that participated in the pilot program demonstrated that'it is possible to .
prepare a good quality examination at the same or lower cost than the NRC or its contractors I could prepare a comparable examination. The fact that a number of facility licensees did not i
. prepare acceptable examinations may be as much an indication of the licensees' inefficiency !
i 7nd inexperience as it is a symptom of deficiencies in the examination criteria. Those facility licensees thbt did not initially submit acceptable examinations, eventually paid more in fees
, 1 because of the additional effort required for the NRC to review, and the licensees' staffs to .
I rewrite, the examinations; Finally, it is possible that the magnitude of the increase in effort and { cost may be perceived to be higher than it actually is because the industry had originally l expected to save money if the NRC would have allowed facility licensees to prepare the examinations using the version of NUREG-1021 that was in effect before beginning the pilot i
- program.
With regard to the additional security costs cited by the examiners, the Commission has i stressed the importance of maintaining examination security, but the NRC has not required facility licensees to invest in additional physical security systems. However, the frequency of security incidents since beginning the pilot examination program has prompted the NRC to clarify the intent of 10 CFR 55A9 in the final rule and to include additional security guidance in final Revision 8 of NUREG-1021. This action will help ensure, among other things, that facility I
-16
' licensees understand their responsibility for maintaining control over the examination process.
The pilot examinations demonstrated that some of the people assigned by the facility to idevelop the examinations did not have sufficient expertise required to prepare good quality examination materials consistent with NRC standards, As noted earlier, the NRC has asked the industry to address the issue of examination quality and the need for additional training on
- .,mination clevelopment. The NRC acknowledges that the restrictions on the use of instructors to prepare the licensing examinations may be partially responsible f"o r limiting the availability of qualified examination preparors. Therefore, the staff has revised the personnel I restrictions in final Revision 8 of NUREG-1021 in an effort to reduce the burden on facility licensees; Specifically, instructors will be allowed to prepare the written examination questions
- without regard to the amount of time they spent training the license applicants; however, the instructors will still be precluded from preparing questions related to those topics on which they j i
provided instruction and from instructing the applicants once they begin working on the ! licensing examination (as was the ca'se in interim Revision 8). This change should provide licensees with increased flexibility in managing their resources and possibly reduce their costs without adversely affecting the independence of the examination process. 1 L - The NRC has revised the regulatory analysis in response to the public comments and flessons loamed from the pilot program. The NRC has also reevaluated the additional l
/ administrative criteria in interim Revision 8 of NUREG-1021 and considers them reasonable I l
and essential to mitigate the vulnerabilities (e.g., quality, security, and conflict of interest) of the
' new examination process and to facilitate the NRC staffs review of the proposed examinations. ,
. Comment: .Two NRC examiners with pilot examination experience reported that the quality of the simulator and walk-through tests has decreased significantly and that, in most l
1 u ..
17 cases, the quality and difficulty of the submitted examinations have been below NRC standards. All four examiners cited various reasons why the quality and difficulty of the facility-prepared examinations might be lower than examinations prepared by the NRC or its contract examiners including: (1) the facility licensees' tendency to narrow the scope of the operating test to those procedures that the facility thinks are important (and emphasized in the training program) and (2) the belief that most facility training personnel do not have the expertise to develop valid test items. Two NRC examiners stated that the quality of the examinations has not' improved during the pilot program and is not likely to improve because there is nothing to prevent licensees from using different people to develop successive examinations. A utility employee asserted that the utilities' limited contact with the process by preparing an examination once every 18 to 24 months will not foster consistency or develop skilled examination writers. Two NRC examiners stated that the elimination of NRC contract examiners who ; participated in examinations across the four NRC regions will be detrimental to examination consistency. One NRC examiner stated that the guidance in interim Revision 8 of NUREG-1021 is not sufficiently prescriptive to ensure nationwide consistency in the level of knowledge tested and the level of difficulty of the examinations and that several specific changes should be included in NUREG-1021 to address his concerns. The State of Illinois asserted that the quality and consistency of the written examination questions can be maintained because the NRC can change and approve the questions before they are used. However, the State also recommended that the NRC should compile the examination questions and proctor the examinations (refer to the conflict-of-interest discussion below). According to NEl, the recent facility-prepared examinations were of higher quality than the examinations prepared by the NRC before the pilot program started. Many of the NRC-
~
q 18'
- prepared examinations had to be revised in response to the facility licensees' technical reviews.
Response: Essentially all of the facility-prepared examinations required some changes and many required significant changes to make them conform to the NRC's standards for quality and level of difficulty. . According to the questionnaires, completed by the NRC chief examiners responsible for the' pilot examinatior.a, the average facility-prepared written examination required approximately 10 to 20 changes, which is consistent with the number of J changes often required on examinations prepared by NRC contract examiners.' Most NRC chief examiners judged the final examinations, with the NRC's changes incorporated, to be comparable to recent NRC-prepared examinations in terms of quality and level of difficulty. Moreover, the fact that the passing rate on the facility-prepared examinations is generally consistent with the historical passing rate on examinations prepared by the NRC suggests that the NRC-approved examinations have discriminated at an acceptable level and that they have
. provided an adequate basis for licensing the applicants at those facilities.
Although the NRC expected that the proposed examination quality would improve as facility' licensees gained experience and familiarity with the NRC's requirements and expectations, the overall quality of examinations submitted to the NRC during the transition process did not improve appreciably over time. Although approximately half of the 17 facility licensees that had prepared more than one examination by the end of FY 1997 did maintain or improve the quality of their second or third examination submittals, the quality of the other .; l facility licensees'second or third examinations was lower. Although it is unclear to what extent the problems'with proposed examination quality and difficulty have been caused by a lack of 2 sufficient expertise on the part of the examination writers, the NRC has asked the industry to
- address this issue. Furthermore, the NRC staff has conducted and participated in a number of
. public meetings and workshops in an effort to communicate its expectations to the facility I
9 j i 1 19-employees who will be preparing the examinations. Additional NRC and industry workshops will be conducted to address examination quality and solicit industry feedback. ; In SEqY-96-206, the NRC staff discussed the issues of examination quality and consistency and.how they might be affected when a large number of facility employees assume I the role that had been filled by a smaller number of experienced NRC and contract examiners.
' The NRC staffs comprehensive examination reviews versus the examination criteria in .
NUREG-1021, in combination with supervisory reviews and the examination ov'ersight activities conducted by the Office of Nuclear Reactor Regulation, should mitigate the vulnerability in this area. Moreover, the industry and staff initiatives to improve the expertise of the examination 1 writers should eventually enhance examination quality'and consistency. I Comment: All four examiners who submitted comments, a nonpower reactor facility l licensee, and the State of lilinois asserted that allowing the facility licensees to prepare the operator licensing examinations decreases'the level of independence and creates a conflict of interest for facility personnel having responsibility for training and licensing the operators. Their ; letters maintained that the new process makes it possible for the utilities to " teach the examination," to test applicants only on what was taught, or to avoid testing in areas with known difficulties.' One NRC examiner noted that the new process places training managers in a no- j win situation because if applicants fail the examination, the managers look like poor trainers, and if the examination is too easy, the NRC gives them a bad report. He and another NRC i
. examiner reported, based on their experience during the pilot examinations, that some facility personnel openly admit that they would develop the easiest possible examination to ensure that all their applicants would pass. *
- One NRC examiner noted that the NRC review and approval process cannot adequately L
20 . compensate for the conflict-of-interest problems inherent in this approach and recommended a change to interim Revision 8 of NUREG-1021 that would limit the licensees' latitude in selecting topics for the examination outline. The State of Illinois suggested that the NRC should compile the questions and proctor the examination to maintain more of the checks and balances that existed under the old process. The nonpower reactor facility licensee noted that most professional licensing examinations are developed by independent agencies, and that this fosters a s'ense of professionalism in the license applicants. j q Resoonse The NRC agrees that the revised examination process decreases the level 1 of independence in the licensing process and may create a potential conflict of interest for facility personnel involved in preparing the examination. As discussed in SECY-96-206, the NRC included a number of measures in Revision 8 of NUREG-1021, including restrictions on
~
which facility employees can develop the examination and a comprehensive review of every examination by an NRC examiner, to ensure that an adequate level of independence is maintained and to minimize the potential for bias in the examination development process. Although the NRC will no longer prohibit instructors from participating in the examination development based solely on the amount of time they spent training the license applicants (as discussed above in response to comments concoming the industry burden under the revised examination process), this change is not expected to substantially inemase the potential for bias in the operator licensing process because instructors will still be prohibited from writing questions'related to the topics in which they provided instruction. Moreover, as a separate i measure, the NRC has amended final Revision 8 of NUREG-1021 to include an expectation that facility licensees will use an objective, systematic process for preparing the written examination outline.' This process enhancement should further limit the potential for bias in the
i 3-i x , , i 21
- selection of topics to be evaluated on the written examination. I The Commission has concluded that the amended personnel restrictions, in combination !
with the systematic selection of topics for the written examination and the continued 1 L independent selection of topics for the operating test, will establish a sufficient level of - independence to address the vulnerability associated with the conflict of interest inherent in the i 1 revised examination process. - However, if the NRC determines that a facility licensee has j intentionally biased the scope,' content, or level of difficulty of an examination (i$e., compromised its integrity contrary to 10 CFR 55.49) to enhance the chances that its applicants would pass the examination, the NRC will utilize its enforcement authority including, as l warranted, civil penalties, orders against the individuals involved, and, charging the individuals involved with deliberate misconduct pursuant to 10 CFR 50.5. Concerns regarding the potential for conflict of interest and the frequency of security
, incidents since beginning the pilot examination program have prompted the NRC to review the
, clarity of 10 CFR 55.4g. The regulation encompasses not only activities like cheating and
- lapses in security but also activities that compromise the integrity or validity of the examination itself (e.g., noncompliance with the criteria designed to limit the potential for bias in the selection )
I of topics to be evaluated on the written examination). Therefore, the NRC has concluded that it would be beneficial to amend 10 CFR 55.49 to clearly state the expectation that facility licensees establish procedures to control examination security and integrity. l Comment: Three NRC examiners and the State of Illinois asserted that the revised i i examination process increases the threat to examination security. One examiner noted that the -i examination is onsite for a longer period of time, thereby proportionally increasing the risk of 1
. being compromised. Another examiner cited the fact that a number of examination reports !
i
)
1 h 22 have documented problems with security. Resoonse: As discussed in SECY-96-206 and in the proposed rulemaking, the Commission is aware of the vulnerability in this area because several security incidents have occurred since beginning the pilot examination program. Therefore, based on the comments l received and the experience with security incidents, the NRC has clarified 10 CFR 55.49 in the j final rule to ensure that applicants, licensees, and facility licensees understand the scope and 1 . I intent of the regulation. The NRC has also strengthened the discussion of examination security l in final Revision 8 of NUREG-1021 and modified NUREG-1600," General Statement of Policy
]
and Procedures for NRC Enforcement Actions," to address enforcement action against parties i subject to the requirements in 10 CFR 55.49. NRC examiners are expected to review the l NRC's physical security guidelines and the facility licensee's specific plans for ensuring l examination security at the time the examination arrangements are confirmed with the i designated facility contact. Furthermore, the NRC has issued an information notice to advise power reactor facility licensees of the NRC's perspective and expectations regarding the integrity of examinations developed by the facility licensees' employees and representatives, and it has asked NEl to take the initiat;ve in developing a model for securing examinations. I As a separate action, the NRC will delay any examination that may have been compromised until the scope of the potential compromise is determined and measures can be taken to address the integrity and validity of the examination. If the compromise is discovered after the examination has been administered, the NRC will delay the licensing action for the affected applicants until the staff can make a determination regarding the impact that the compromise has had on the examination process. If the compromise is not discovered until after the licensing action is complete, the NRC will reevaluate the licensing decision pursuant to
n 23 i
.10 CFR 55.61(b)(2) if it determines that the original licensing decision was based on an invalid . i examination.
- Comment: One'NRC examiner disagreed with the conclusion in the proposed rulemaking that the facility-prepared examination process is an efficient use of NRC resources when compared to the NRC-prepared or contractor-prepared examinations. He noted that, in
- most cases, the' quality and difficulty of the proposed examinations have been below NRC standards (as discussed above) and that it has taken a significant effort on the part of the NRC chief examiner to achieve an accepta'ble product. l An NRC contract examiner asserted that NRC cost-saving is a poor reason for changing !
the rule, since the utilities pay for the examinations anyway. He noted that the pilot examination process has led to a loss of certified examiners and contends that those NRC examiners who
. are left will become more dissatisfied with their jobs and will leave because they will be required to travel more to compensate for the loss of contractors.
Response The NRC acknowledges that many of the facility-prepared examinations (about 20 percent in FY 1997) required significantly more NRC examiner time than desired or planned in order to achieve NRC quality standards.' However, questionnaires filled out by NRC chief examiners for the pilot examinations indicate that the average amount of time spent on reviewing and upgrading the examinations is generally consistent with the estimates developed
. before starting the pilot program (i.e. approximately 170 examiner-hours). As noted in SECY-97-079, the NRC has issued a memorandum to the regional administrators emphasizing the importance of (1) assigning adequate resources to carry out the operator licensing task, (2) completing a thorough review of every facility-prepared examination, and (3) not administering I
any examination that fails to meet NRC standards for quality and level of difficulty.
p. ?l. i 24
' Furthermore, all the time that NRC examiners spend reviewing an examination and bringing it
'up to standards is ultimately billed to the facility licensee.
. The Commission acknowledges that facility licensees bear the cost of preparing the licensing examinations whether or not the NRC performs this function. However, this comment does not take into account the NRC's intemal accounting procedures. As a result of this process change, the NRC has been able to reprogram the funds that were saved to perform other functions.
- The NRC's budget cuts have necessitated agency-wide downsizing, which can be expected to result in increased travel for many NRC employees, not just the operator licensing
- examiners. The number of NRC full-time equivalent (FTE) license examiners has remained essentially constant throughout the pilot program and, aside frem normal attrition and staff tumover, the loss of certified examiners has been limited to NRC contractors.
.e Comment Two NRC examiners expressed concem that examinerproficiencywill decrease as a result of implementing the revised examination process. One of the examiners stated that examination reviewers will not maintain the same base of knowledge as examination writers maintained and that they will lose their familianty with plant operating procedures.
Resnonse: The Commission has concluded that the revised examination process affords sufficient NRC staff involvement that NRC examiners will maintain an acceptable level of proficiency. ' An NRC examiner will review and approve every examination before it is administered to ensure that it conforms to the criteria specified in NUREG-1021 for content, format, quality, and level of knowledge and difficulty. NRC examiners will also continue to independently administer and grade both the dynamic simulator and the plant walk-through portions of the operating tests. Because NRC examiners will be administering all of the
-i 25 operating tests, the Commission believes that the revised process will enable the examiners to accrue more experience in a shorter period of time and to maintain their proficiency. New NRC license examiners will still be required to complete a standardized training program, including
- the 'develcpment of a complete written examination and operating test, as part of their qualification process. Moreover, the'NRC will ensure that the in-house capability to prepare the examinations is maintained by (1) requiring each region to write at least one initial operator licensing examination per calendar year, (2) requiring a regional supervisor to r'eview and approve every examination anC the Office of Nuclear Reactor Regulation to conduct periodic examination reviews, (3) conducting examiner refresher training, and (4) convening an operator licensing examiners' training conference at intervals not to exceed 24 months.
Comment: A utility employee asserted that the revised examination process will not
, enhance the competency of the operators or reactor safety because the facilities' training resources will be diverted from their primary purpose (i.e., training the applicants) as much as six months before the ' examination date. Three NRC examiners also took issue with the -
conclusion in the proposed rulemaking that the NRC staffs focus on operator performance and its core of experience will improve under the pilot examination process because contractors will no longer be used to administer the operating tests. Two of the examiners asserted that the reduction in the amount of procedural research by examiners will result in the identification and correction of fewer procedural problems. Two of the examiners also statad that the contract examiners help maintain examination consistency across the NRC regions and that their contribution to the operator licensing program goes beyond simple task completion. Response The Commission expects that those training departments that cannot readily and safely absorb the examination development work will use the funds that they were
m 26
- previously paying to the NRC through the fee recovery program to secure the additional personnel to do the extra work; if a facility licensee' places insufficient resources on either training or testing, the' quality of its proposed licensing examinations or the passing rate on th'ose examinations would most likely suffer. Although many of the facility-prepared examinations have required significant changes to achieve NRC quality standards, the examination results, to date, are generally consistent with the results on previous NRC-prepared examinations, suggesting that the quality 'of the facility licensees' training programs
' has not been affected. Therefore, the fact that facility licensees will be preparing the examinations is' not expected to have a negative effect on reactor safety.
The NRC acknowledges that the contract examiners identified procedural and training
' problems'in addition to their primary responsibility for preparing and administering the licensing examinations, and that they helped maintain examination consistency by workino on
' examinati9ns in t sh of ti,a NRC's regions. As noted in connection with the discussion of examination quality, the Commission realizes that the revised examination process increases the' possibility of inconsistency, but it believes that the examination criteria in final Revision 8 of NUREG-1021, in combination with the NRC's examination oversight programs, will minimize i
. such inconsistencies so that they remain within acceptable limits.
When the NRC initiated the pilot program, its goal was to eliminate the need for NRC contract examiners without compromising the existing levels of reactor safety. Because NRC examiners will be administering all of the operating tests, the revised process will enable the NRC examiners to accrue more experience in a shorter period of time and may improve the consistency of the operating test evaluations and the licensing decisions. Although the total number of procedures reviewed in the process of developing examinations may be fewer under the revised method, NRC examiners will still be expected to review and identify discrepancies in l
,a 27 the procedures that will be exercised during the walk-through portion of the operating test and during the simulator scenarios.
l i Other Comments. Since beginning the p!!ot examination program, the Commission has sought to obtain
~
up-to-date insights regarding the effectiveness of the revised examination process based on the staff's growing body of experience in reviewing the facility-prepared examinations. Many of the staff comments received have paralleled the public comments and require no further attention in this notice. ' However, one recommendation to amend the actual wording of the proposed regulation is considered worthy of discussion and incorporation. Specifically, it was j
' recommended that the rule should indicate that a key manager would be responsible for , submitting the examination because that individual would be in a position to ensure that the facility licensee's operations and training departments apply sufficient resources to prepare a quality examination. The Commission finds that the recommendation is consistent with normal l
NRC practice and the analogous regulatory requirement in $55.31(a)(3), which requires "...an authorized representative of the facility licensee by which the applicant will be employed..." to ]
- submit a written request that examinations be administered to the applicant. Therefore, the wording of $55.40(a)(2) has been amended to require an authorized representative of the facility licensee to approve the written examinations and operating tests before they are
' submitted to the NRC for review and approval.
o 6 J { l 28 l l l Availability of Guidance Document for Preparing Operator Licensing Examinations l l As a consequence of preparing and administering the initial operator licensing examinations over a number of years, the NRC has developed a substantial body of guidance I to aid its examiners. That guidance has been published in various versions of NUREG-1021, the latest version of which (final Revision 8) incorporates lessons learned since interim Revision 8 was published in February 1997 as well as refinements prompted by the com'ments submitted in response to the FRN of August 7,1997 (62 FR 42426), which solicited public comments in conjunction with the proposed rulemaking. A copy of final Revision 8 of NUREG-1021 will be mailed to each facility licensee. Copies may be inspected and/or copied for a fee at the NRC's Public Document Room,2120 L Street NW (Lower Level), Washington, DC. Final Revision 8 of NUREG-1021 is also electronically available for downloading from the intemet at "http://www.nrc. gov." The NRC plans to prepare, administer, and grade initial operator licensing examinations
~ at least four times a year, using NUREG-1021 as guidance. Licensees will also be expected to use the guidance in NUREG-1021 to prepare the licensing examinations. The NRC staff will review and approve any attematives that do not conform with this guidance. The NRC will not l approve any attemative that would compromise its statutory responsibility of prescribing uniform conditions for the operator licensing examinations. Examples of unacceptable attematives include, but are not limited to, the use of essay questions in place of multiple choice quest ons
, and the administration of open book examinations. l
i . e.!
-29 I Final Rule 9
This regulation adds a new section, 955.40, " implementation," to Subpart E of 10 CFR
'Part 55, which requires power reactor facility licensees to prepare the written examinations and
- operating tests, to submit the examinations that have been approved by an authorized representative of the facility licensee to the NRC for review and approval, and t'o proctor and i
- grade the written examinations. These requirements are contained in $$55.40(a)(1),'(2), and
)
(3), respectively. Each power reactor facility licensee is expected to prepare and submit the proposed i examinations (including the written examination, the walk-through, and the dynamic simulator tests) to the NRC consistent with the guidance in NUREG-1021. An authorized representative of the facility licensee shall approve the written examinations and operating tests before they I
. are submitted to the NRC. The NRC staff will review the entire examination and direct
, whatever changes are necessary to ensure that adequate levels of quality, difficulty, and 1 ;
j consistency are maintained. After the NRC staff reviews and approves an examination, the l facility licensee will proctor and grade the written portion consistent with the guidance in
. NUREG-1021. The NRC staff will continue to independently administer and grade the operating tests, review and approve the writbn examination results, and make the final i
licensing decisions. The facility licensee will not conduct parallel' operator evaluations during the dynamic simulator or the walk-through tests. Pursuant to the requirements in $55.40(b), the NRC staff maintains the authority to j prope.re the examinations and tests and to proctor and grade the site-specific written examinations. This allows the NRC to maintain its staff proficiency to perform these activities. '
30 Also, if the NRC has reason to question a licensee's ability to prepare an acceptable examination, $55.40 (b) gives the NRC authority to prepare and administer the examinations and tests. Paragraph (c) of 55.40 reasserts that the NRC will continue to prepare and administer the written examinations and operating tests at non-power reactor facilities. The NRC has taken this position because the non power reactor community does not have an accreditation process for training and qualification or the resources to prepare the examinati5ns. This regulation also amends $55.49 because the NRC has determined, since the _ proposed rule was published, that applicants, licensees, and facility licensees may be interpreting $55.49 too narrowly by limiting it to actual cases of cheating. The amendment clarifies that the. regulation pertains to all activities that could affect the equitable and consistent administration of the examination, including activities prior to, during, and after the examination is administered. The amendment also states the NRC's expectation that facility licensees will establish procedures to control examination security and integrity. I Environmental impact: Categorical Exclusion The NRC has determined that this rule is the type of action described as a categorical
- exclusion in 10 CFR 51.22(c)(1). Therefore, neither an environmental impact statement r.or an environmental assessment has been prepared for this regulation.
p 1'
31 Paperwork Reduction Act Statement This final rule amends information collection requirements that are subject to the . 7 Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq). These requirements were approved by the Office of Management and Budget (OMB), approval number 3150-0101. The additional I public. reporting burden for this collection of information is estimated to average 500 hours per response, including the time for reviewing instructions, searching existing data ' sources, gathering and maintaining the data needed, and completing and reviewing the collection of information (i.e., preparing the examinations). Send comments on any aspect of this collection of information, including suggestions for reducing the burden, to the information and Records
- Management Branch (T-6F33), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by intemet electronic mail to bjs1@nrc. gov, and to the Desk Officer, Office of Informition and Regulatory Affairs, NEOB-10202, (3150-0101), Office of Management and l Budget, Washington, DC 20503.
Public Protection Notification l l The NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number. ! i Regulatory Analysis L The Commission has prepared a regulatory analysis on this regulation. The analysis < 4 examines the costs and benefits of the altematives considered by the Commission. The
32 , regulatory analysis is available for inspection in the NRC Public Document Room, 2120 L Street NW (Lower Level), Washington, DC. Single copies of the analysis may be obtained from ) Siegfried Guenther, Office of Nuclear Reactor Reguiation, U.S.. Nuclear Regulatory l Commission, at 301-415.1056 or by e-mail at sxg@nrc. gov. , a 1 Regulatory Fjexibility Certification I In accordance with the Regulatory Flexibility Act of 1980, (5 U.S.C. 605(b)), the Commission certifies that this rule does not have a significant economic impact on a substantial , number of small entities. This rule affects only the licensing and operation of nuclear power i plants. The companies that own these plants do not fall within the scope of the definition of i "small entities" described in the Regulatory Flexibility Act or the Small Business Size Standards stated in regulations issued by the Small Business Administration at 13 CFR Part 121. l Backfit Analysis The pertinent part of 10 CFR 50.109(a)(1) defines backfitting as "the modification of or addition to ... the procedures or organization required to ... operate a facility; any of which may result from a new or amended provision in the Commission's rules or the imposition of a regulatory staff position interpreting the Commission's rules that is either new or different from a previously applicable staff position...." Part 55 addresses the qualifications and requirements for operators' licenses and changes are not per se subject to the backfit rule in Part 50. Changes to these requirements could be Pncluded within the backfit definition of " procedures or organization required to ... operate a facility;" however, in this case, the shift of responsibility
, -4 1
L 33 from the NRC staff (or its contractors) to the facility licensee for developing and administering l^~ .. .. i the initial written operator license examination, developing the initial operating tests, and establishing procedures to control examination security and integrity would not constitute a
" modification of the procedures required to operate a facility" within the scope of the backfit rule.
; Therefore, no backfit analysis is necessary, i This rule does not affect the basic procedures for operator license qualification, i.e., the required training programs, the required testing, the content and format of the exams, the grading of the exams, or the basis for issuing an operator license. The shift in responsibility for
. preparation of the initiallicensing examination does not affect the content or format of the !
examination.. The rule is designed to ensure that the format, content, and quality of the initial
. written examination'will not be modified. The rule requires the NRC to provide oversight of facility licensees' development and administration of initial written examinations and the development of the operating tests. The NRC also retains its discretion to determine whether to I administer the initial written ex' amination itself, as well as continuing to determine whether to
- grant or deny an application for a reactor operator or senior reactor operator license, and consider candidates' appeals.
)
i
~ The licensee's organizational structure required to operate the facility will not be i modified. All reactor licensees have a training component as part of their organizational
. structure, and this rule does not alter that organizational structure. Although, the rule could have an "effect" on the licensee's organization, it does not require any modification to the i
organizational structure. i
E ] L 34 Enforcement Policy in conjunction with this final rule, the Commission is separately publishing modifications 4
' to NUREG-1600, " General Statement'of Policy and Procedure for NRC Enforcement Actions,"
1 l to address enforcement action against parties' subject to the requirements in 10 CFR 55.49 l (i.e., Part 55 license applicants / licensees and Part 50 licensees). l List of Subjects in 10 CFR Part 55 Criminal penalties, Manpower trainlag programs, Nuclear power plants and reketors, Reporting and recordkeeping requirements. l For the reasons given in the preamble and under the authority of the Atomic Energy Act of 1954, as amended; the Energy Reorganization Act of 1974, as amended; and 5 U.S.C. 553; the NRC adopts the following amendments to 10 CFR Part 55. l
' PART 55 - OPERATORS' LICENSES I i
i
- 1. The authority citation for Part 55 continues to read as follows:
L ' AUTHORITY: Sees. 107,161,182, 68 Stat. 939, 948, 953 , as amended, sec. 234, 83 i
- Stat. 444, as amended (42 U.S.C. 2137, 2201, 2232, 2282); secs. 201, as amended, 202, 88 Stat.1242, as amended, 1244 (42 U.S.C. 5841, 5842).
l i. l.,-
l 1 I l 35-
~
Sections 55.41,55.43,55.45, and 55.59 also issued under sec. 306, Pub. L. 97-425, 96 Stat. 2262 (42 U.S.C.10226). Section 55.61 also issued under secs. 186,187,68 Stat. 955 (42 j i U.S.C. 2236, 2237). i
- 2. In 955.8, paragraph (b) is revised to read as follows: l l- 1
$15.8 Information Collection Reauirements: OMB Acoroval. j l
. . . .. . l l
(b) The approved information collection requirements contained in this part appear in !
$$55.31,55.40,55.45,55.53, and 55.59. ;
- 3. A new $55.40 is added to read as follows:
$55.40 Implementation 1
(a) Power reactor facility licensees shall- . (1) Prepare the required site-specific written examinations and operating tests; (2) Upon approval by an authorized representative of the facility licensee, submit the written examinations and operating tests to the Commission for review and approval; and' l (3) Pr6ctor and grade the NRC-approved site-specific written examinations.
'(b) in lieu of requiring a specific power reactor facility licensee to prepare the i
examinations and tests or to proctor and grade the site-specific written examinations, the Commission may elect to perform those tasks. J ! l (c) The . Commission will prepare and administer the written examinations and operating l tests at non-power reactor facilities. ' l
h i 36
- 4. $55.49 is revised to read as follows:
,655.49 Inteority of examinations and tests.-
Applicants, licensees,'and facility licensees shall not engage in any activity that l compromises the integrity of any application, test, or examination required by this part, and facility licensees shall establish,' implement, and maintain procedures to prevent compromises. l An examination or test is considered compromised if any activity, regardless of intent, affected, .
- j. or but for detection wobid have affected, the equitable and consistent administr'ation of the test l 'or examination. This includes activities related to the preparation and certification of the license ,
. applications and all activities related to the preparation, administration, and grading of the :
j
. required tests and examinations. I 4
. Dated at Rockville, Maryland, this - day of .'1998.
For the Nuclear Regulatory Commission. i. John C. Hoyle, Secretary of the Commission. I 1
.( .
I I
,( ' Enclosure 3 Regulatory Analysis 1 i a l
y REGULATORY ANALYSIS FOR RULEMAKING ON REQUIREMENTS FOR INITIAL LICENSED OPERATOR EXAMINATIONS
- 1. Statement of Problem and Objective Section 107 of the Atomic Energy Act of 1954 (AEA), as amended, requires the NRC to determine the qualifications of individuals applying for an operator license, to prescribe uniform conditions for licensing such individuals, and to issue licenses as appropriate. l To implement this statutory mandate, operator license applicants are required by 10 CFR Part 55, " Operators' Licenses," to pass a written examination and an operating
. test. The written examination and operating test must satisfy the basic content requirements that are specified in the regulation. Although neither the AEA nor Part 55 specifies who must prepare, administer, and grade these examinations, the NRC has traditionally performed those tasks itself or through its contract examine.rs. Because this has been a costly process in terms of NRC personnel and contractual support, the NRC staff has evaluated an attemative approach - one that will require nuclear power plant licensees to prepare the examinations and submit them to the NRC for review and approval. This approach has been tested and assessed through a voluntary pilot program and has been deemed by the NRC staff to be feasible. The monitoring and assessment of this voluntary pilot program has demonstrated that examinations prepared by facility licensees and modified as directed by the NRC are comparable in terms of their quality and discrimination validity to those prepared by the NRC and its 4 contract examiners under the existing process; therefore, the safe operation of the nuclear power plants is not compromised by using the altemative approach. Thus, the NRC is amending 10 CFR Part 55 to require nuclear power plant licensees to prepare these examinations and has published a final version of Revision 8 of NUREG-1021,
" Operator Licensing Examinatior *tandards for Power Reactors," as guidance. This action will allow the NRC to eliminate the need for contractor support for the operator licensing program (with the exception of the generic fundamentals examination). The fiscal year (FY) 1998 and FY 1999 budgets are consistent with this proposal and already reflect the elimination of contract examiner support made possible under the voluntary !
pilot and transition program. The NRC staff's primary objective in shifting responsibility for preparing the initial .
. operator licensing examinations to the power reactor facility licensees is to reduce the l amount of NRC resources used in this area. This change in policy is part of the NRC's ;
continuing effort to stoamline the functions of the Federa1 Government consistent with ! the Administration's initiativos and to accommodate NRC resource reductions. Given the limited resources available, the NRC has judged that the operator licensing contractor funds could be better spent on other work rather than paying for examinations that the facility licensees are generally qualified and motivated to prepare. Pursuant to the provisions of the AEA, the NRC will ensure that the quality of the operator licensing examinations and the effectiveness of the operator licensing program are maintained. These changes are not intended to affect the format, content, scope, quality and level of difficulty of the examinations, thereby minimizing the impact of the rule change on the operator license applicants. 1
l
- 2. DwAground 10 CFR 55.31(a)(3) requires the applicant for an operator's license to submit a written request from an authorized representative of the facility licensee that the written examination and the operating test be administered to the applicant. Furthermore, i 10 CFR 55.33(a)(2) states that the Commission will approve an initial application for a license if it finds that the applicant has passed the requisite written examination and operating test in accordance with 5555.41 and 55.45 for reactor operators, or 9955.43 and 55.45 for senior operators. These written examinations and operating tests ;
determine whether the applicant for an operator's license has learned to operate a ] facility competently and safely, and additionally, in the case of a senior operator, ! whether the applicant has learned to direct the licensed activities of licensed operators l competently and safely. ! As stated above, the NRC or its contract examiners have traditionally pr'epared, administered, and graded the written examinations and operating tests. Before beginning the pilot program, the NRC spent approximately 53 million per year to retain contractor support for the operator licensing programs (including initial licensing examinations and requalification inspections). In accordance with 10 CFR 170.12 (i), the NRC staff and contractual costs are recovered from the facility licencees that receive examination services. This rule will change the current practice in which the NRC prepares and proctors the initial examination for reactor operators and senior operators and, instead, will require each power reactor facility licensee,to prepare the entire examination and proctor and grade the written portion of the initial examination. This action does not modify the procedures or organization required to operate the plant, and, therefore, does not ! constitute a backfit pursuant to 10 CFR 50.109.
- 3. Identification and Analysis of Three Alternative Approaches 3.1 Alternative 1 -Take No Action As discussed above, the budget for FYs 1998 and 1999 is consistent with this action and reflects the elimination of contract support. If 10 CFR Part 55 is not amended, it may require actions such as (1) the restoration of contractor support for the operator licensing program, (2) an increase in the direct examiner resources in each regional office to satisfy the demand for initial licensing examinations and inspections, (3) the implementation of other measures (e.g., changing the format or length of the examinations) that will enable the existing NRC staff to conduct the examination and inspection programs, or (4) scheduling the examinations based on the NRC resources available (which may require the staff to prioritize its activities based on the facility licensees' needs).
2
3.2' Alternative 2 - Provide Regulatory Guidance p This altemative was rejected because the NRC staff considers implementation of the new process on a voluntary basis alone unworkable over the long term. If the NRC
; does not require facility licensees to prepare the initial operator licensing examinations, there will be no guarantee that each licensee would_ elect to prepare these examinations.
With the elimination of contractor support, the NRC staff may no longer have sufficient i examiner resources to prepare examinations consistent with the scheduling needs of I facility licenseesi This resource problem is further compounded by the unpredictable nature of the examination workload and by other unanticipated demands on the examiner work force. 3.3 Alternative 3 - Amend 10 CFR Part 65 , This alternative requires every power reactor facility licensee to prepare the initial i operator licensing examinations and to proctor and grade the written portion of the i examination. This enables the NRC to eliminate the use of contractors in the operator licensing program (with the exception of the generic fundamentals examinations). i Before initiating the pilot examination and transition program at the beginning of FY 1996, the NRC spent approximately $3 million per year on contractor support for _ j licensing examinations and requalification inspections. Under this attemative, the NRC ! staif will perform those tasks that were previously performed by contract examiners, including examination administration and inspections of licensee requalification ; programs.' The funds will be spent on other, high priority NRC work, rather than paying for contractors to prepare examinations that the facility licensees are generally qualified l L to prepare. As discussed above, the budget for FYs 1998 and 1999 is consistent with ! this action and reflects the elimination of contract support.
- 4. Regulatory impact - Qualitative Coats and Benefits l Facihty Licensees i
Before beginning the pilot examination program, the NRC depended on NRC employees l and contractors to prepare and administer the initial operator licensing examinations < required by 10 CFR Part 55. NRC contractors also assisted in the inspection of requalification programs administered by facility licensees. In accordanca with 10 CFR 170.12 (i), the cost of NRC time spent and any related contractual costs were
- (and are) billed directly to the facility licensees that receive (d) the examination services.
Under the revised examination process, each power reactor facility licensee will assume
- i . . responsibility for preparing the site-specific initial operator licensing examinations at its -
1 facilities. This will allow the NRC to discontinue the use of contract examiners for both the initiallicensing and requalification inspection programs. Facility licensees will be expected to prepare the proposed examinations (including the written examination, the walk-through, and the dynamic simulator tests) based on the guidance in NUREG-1021 and to submit the examinations to the NRC for review and approval. Facility licensees 3
will have the option to hire a contractor to prepare the license examination (as the NRC I often did before starting the pilot process). This rule change gives facility licensees more control over the cost of their examination services because it puts them in a position to manage the quality of the product that is submitted to the NRC. The higher the quality of the examination that the facility licensee submits, the lower the resulting NRC charges. If the NRC or one of its contractors writes an examination, the facility licensee is responsible for the entire cost of preparing the examination. The NRC will thoroughly review the examinations prepared by the facility licensees ! consistent with the guidance in NUREG-1021 and will direct the facility training staffs to ! make whatever changes are necessary to achieve a product that meets the NRC's standards for quality and level of difficulty Each facility licensee will be, billed only for
)
the time that the NRC staff spends to review the examination prepared by the facility !
-licensee, to direct the revisions that are necessary for the examination to meet NRC standards, and to administer and grade the operating tests. The NRC will not approve any examination that fails to meet its standards or that compromises the NRC's statutory responsibility for prescribing uniform conditions for the licensing of operators pursuant to the AEA. Examples of unacceptable deviations include, but may not be limited to, the use of essay questions in lieu of multiple choice questions and the j administration of open-book examinations. '
After the NRC reviews and approves an examination, the facility licensee will proctor l and grade the written portion according to the guidance in NUREG-1021. The NRC staff will continue to administer and grade the operating tests, review and approve the ; written examination results recommended by the facility licensee, and make the final licensing decisions. 1 Based on lessons learned and feedback from the pilot examinations completed at the l time, the proposed regulatory analysis predicted that the average time ' spent by a facility - licensee to prepare the written examination and operating tests would be approximately i 600 to 800 staff-hours. Because a portion of that time (about 200 hours) would be spent i reviewing and assisting with the administration of NRC-developed examinations under the traditional examination orocess,200 hours should be subtracted from the total. The resulting average burden of approximately 400 to 600 staff-hours was somewhat higher than the 400 hours that the NRC staff or its contract examiners typically take to prepare an examination. The extra burden was generally attributable to the facility licensees' unfamiliarity with specific NRC examination expectations and to the additional administrative tasks, such as documenting the source of the examination questions, required to maintain examination integrity. The NRC staff believed that the facility training staffs already had the basic knowledge, skills, and abilities necessary to evaluate operator performance and develop test items for the initial licensing examination and expected that the burden would diminish as facility training staffs gained familiarity and experience with the NRC's specific examination requirements and expectations. Moreover, the NRC staff expected that the facility employees' more detailed knowledge of their facility and easier access to the reference materials required 4
I i I-l to prepare the examinations would eventually enable them to prepare acceptable quality examinations in less time than the NRC or a contractor could. In an effort to quantify the potential cost savings, the proposed regulatory analysis l assumed that the fucility licensees would eventual!y prepare the examinations in the
. same amount of time as the NRC allotted its contract examiners to perform the task (i.e., approximately 400 hours), and that the time would be equally distributed between l
contractors and the licensee's own in-house staff at a cost of $120 per hour and $60 per , I hour, respectively. This translated into an industrywide burden of $2.16 million, l assuming 60 site-specific licensing examinations per year, and was roughly the same as i the cost would be if the NRC were to prepare the examinations relying equally on in-house and contractor staff efforts. The NRC staff had no basis for estimating the cost savings that might be gained if facility licensees availed themselves of the l aforementioned efficiencies, but a 10 percent reduction in the burden wpuid result in a small industrywide savings of about $220,000 per year. l The proposed rulemaking that was published in the Federal Register (62 FR 42426) on August 7,1997, specifically requested public comments on the NRC staff's conclusion in i the proposed regulatory analysis that it would eventually be less costly for each facility j l licensee to prepare its own initial operator examinations rather than to have an NRC ! l contractor prepare the examinations for all licensed operators with the costs reimbursed f ) by licensee fees. The proposed rulemaking also invited public comments on interim ) Revision 8 of NUREG-1021, which defines the NRC's examination criteria and l expectations. Seven respondents directly or indirectly addressed the resource question in their comment letters, and seven respondents submitted specific comments and i recommendations related to NUREG-1021. ! The Nuclear Energy institute (NEI) and one utility stated that the revised examination criteria in interim Revision 8 of NUREG-1021 have increased the level of effort and will l . result in higher licensing fees regardless of who prepares the examinations. However, l NEl and another utility agreed that comparing the cost of utility-prepared examinations to those prepared by the NRC is difficult. They concluded that it should be less costly l for utilities to prepare the examinations than to have the NRC prepare them, provided l the preparers are using exactly the same criteria. NEl also stated that the relative cost of the two examination processes should not be the only factor in deciding whether to proceed with the rulemaking. NEl indicated that preparing higher-cognitive level questions requires detailed plant knowledge better known to licensees and that the , revised process will allow NRC staff to concentrate on directly evaluating each applicant l l without relying on third-party observers. l Two NRC examiners, one contract examiner, and a utility employee asserted that the facility licensees' cost has increased under the revised examination process. They cited various reasons for the increased cost, including training personnel to write the examinations and then restricting them from training the applicants and upgrading equipment to maintain examination security. The NRC examiners based their opinions l on infarmal feedback from facility training personnel. One examiner indicated that it was ; taking facility licensees an average of 700 hours to prepare each examination. The i 5 4 I 1
utility employee stated that the rule will simply transfer the cost of contractors from the NRC to the utilities. The NRC staff acknowledges that the revised examination and administrative criteria in interim Revision 8 of NUREG-1021 have probably caused the cost of the examinations to be somewhat higher than it would have been if facility licensees had been allowed to prepare the examinations using the same criteria that applied to the NRC and its contractors before starting the pilot program. The staff has reevaluated the revised criteria in light of the public comments and lessons leamed during the pilot examination program and concluded that the criteria remain necessary to mitigate the vulnerabilities (e.g., quality, security, and conflict of interest) inherent in the revised examination process and to facilitate the NRC staff's review of the proposed examinations. The staff has incorporated several minor changes (e.g., allowing instructors to prepare written examination questions on subjects they did not teach, without regard to the amount of time they spent training the applicants) and clarifications in the final version of Revision 8 of NUREG-1021. However, the changes are not expected to significantly affect the cost of preparing an examination. Despite the revised examination and administrative criteria, many of the facility licensees that participated in the pilot program demonstrated that it is possible to prepare an acceptable quality examination at the same or lower cost than the NRC or its ' contractors could prepare a comparable examination. However, approximately 40 percent of proposed pilot examinations have not met NRC standards for quality and have required significant changes. The fact that the lower quality examinations take longer to review and require significant rework by the facility licensees has driven up their total cost and caused the staff to question the validity of its original expectation that the industry may realize additional cost reductions as it gains experience with the NRC's examination requirements. As noted earlier, the proposed regulatory analysis assumed that the training personnel at power reactor facilities already have the basic expertise necessary to develop test items fer the initiallicensing examination. During the second half of the 1980s, the industry increased its emphasis in the training area, and all power reactor licensees established formal training programs that were based on a systems approach to training (SAT) and were accredited by the National Academy for Nuclear Training. Moreover, pursuant to 10 CFR 50.120 and 55.4, SAT-based training programs must evaluate the trainee's mastery of training objectives, and NRC inspections of licensee requalification programs for licensed operators confirmed that training staffs generally possessed the skills needed to evaluate the trainees' knowledge. However, the public comments on the proposed rule, comments from NRC examiners involved with the pilot examinations, and the number of significant changes required on ! many of the pilot examinations have raised concems about the capability of some of the facilities to write the licensing examinations. The NRC staff realizes that the restrictions on the use of instructors to prepare the licensing examinations may have prompted some facility licensees having smaller training staffs to use less-qualified personnel to prepare the examinations. And although the NRC continues to believe that personnel
. 6
)
restrictions are necessary to address the potential for conflicts of interest in the examination development process, it has reconsidered the scope of the restrictions in
. interim Revision 8 of NUREG-1021 in an effort to reduce the potential burden on facility licensees. Consequently, final Revision 8 will allow facility instructors to prepare the written examination questions without regard to the amount of time they spent training the license applicants. However, final Revision 8 will still preclude instructors from preparing questions related to those topics on which they provided instruction and from instructing the applicants once they begin working on the licensing examination (as was the case in interim Revision 8).
It is unclear to what extent the examination quality has been affected by the training and 1 qualification of the examination writers, but the NRC staff has asked the industry to l address this issue and to determine the need for additional standards for training on testing and measurement principles and examination development techniques. The cost of such an initiative is uncertain at this time, but the benefits would be realized throughout the facility licensees' training programs and not just in the initial operator licensing program. With regard to the additional security costs cited in the public comments, it is important to note that although the NRC has stressed the seriousness of maintaining examination security, the staff has not required that facility licensees invest in additional physical security systems. The NRC's expectations regarding the facility licensees' responsibility for maintaining examination security were outlined in draft Revision 8 of NUREG-1021, which was issued for public comment in February 1996, and subsequently clarified in j interim Revision 8 of NUREG-1021, which was issued for use in February 1997. The ; security incidents that occurred during the pilot program and the public comments on the proposed rule prompted the NRC to include additional security guidelines in final Revision 8 of NUREG-1021; however, the guidelines are instructional in nature and do not require any security system improvements. The security incidents that occurred during the pilot program has also prompted the NRC to conclude that applicants, licensees, and facility licensees may not understand the provisions of $55.49, " Integrity of Examinations and Tests." Therefore, the NRC has determined that it would be l beneficial to amend $55.49 in the final rule to clearly state the expectation that facility licensees establish and maintain procedures to control examination security and integrity. The NRC staff believes that most facility licensees have already established examination security policies and practices, so the additional cost of documenting those policies and practices in formal procedures should not be significant. Establishing , formal security procedures should raise facility employees' awareness of this important l issue and could avert future security incidents that might require examinations (or ! portions of the examinations) to be delayed and replaced at significant expense to the i affected facility licensees. As noted in the proposed regulatory analysis, the revised examination process will eliminate the need for fscility licensees to duplicate and ship multiple sets of reference materials because NRC contract examiners (who would have required a separate copy of the reference material) will not be used to prepare the examinations. Feedback from the industry in response to the NRC staff's solicitation of public comments on the draft 7 1 i
p revision of NUREG-1021 (refer to 61 FR 6869, of February 22,1996) indicated that facility licensees had been spending an additional 80 to 160 hours to prepare and ship the reference materials under the existing examination process. Under the revised l process, the facility licensees will generally submit only materials that are needed for the NRC chief examiner to verify the accuracy of the examination questions. This is a reduction of resources, but it has not been quantified in this analysis. l In summary, before beginning the pilot program, the NRC relied on its own staff and l contractors to write the site-specific operator licensing examinations, and the cost was l billed to the facility licensees in accordance with 10 CFR 170.12 (i). To appropriately l prepare and administer such examinations, the NRC staff and contractors had to learn ! the details of the operation of each specific plant. In effect, this necessitated that, to prepare the examinations,' the examining staff duplicated technical expertise already present at each site. This rule will eliminate such inefficiency by placing the
. responsibility for preparing the examinations upon each power reactor facility licensee.
Efficiency should be gained if the NRC focuses its efforts on maintaining the appropriate scope, depth, and quality of the examinations, leaving the preparation of the detailed examination materials to the facility licensees. The NRC expects that the initial period of inefficiency will continue until facility licensees leam the process for preparing the examinations, expand and irhprove their examination question banks, and upgrade the capabilities of their training staffs, as necessary. Experience during the pilot examination program has demonstrated that the quality of the examinations and the efficiency of the process will be slow to improve. It may take considerable time and effort to develop a bank of written examination questions i ! appropriate for use on the initial licensing examination, but such a bank should improve l the quality of future examinations and should be more efficient. The NRC still expects that with experience and training most facility licensees will be able to prepare acceptable quality examinations that require less NRC review resources and provide larger potential savings to the licensees. . Those facility licensees that submit acceptable , quality examinations are likely to save resources despite the additional administrative criteria that the NRC considers necessary under the revised examination process. However, those facility licensees that submit examinations that require many changes because they do not meet NRC quality standards, will likely and up with a more costly examination than if it had been prepared by the NRC. _ l Operator License Applicants
. To the extent possible, the format, content, quality, and level of difficulty of the examinations should remain unchanged, thereby minimizing the impact of the rule change on the operator license applicants. NRC examiners will continue to review, to direct revisions where necessary, and approve every written examination and operating test before it is administered. The NRC acknowledges that some of the facility-prepared examinations will be more difficult than others and that some of the variation can likely be traced to individual differences among the NRC examiners who review and approve t the examinations. Such was also the case when the NRC or its contractors wrote the examinations. However, the NRC believes that the variation in the level of difficulty can 8
l l L
i-be maintained within acceptable limits and would be even greater if NRC examiners do l- not take action to ensure that the submitted examinations are consistent with NRC ' standards. The average reactor operator (RO) and senior operator (SRO) passing rates on both the l written and operating portions of the 68 facility-prepared examinations administered
- through the end of FY 1997 were only slightly lower than the corresponding passing i rates on the NRC-prepared examinations administered during FY 1995, the last year in l which all of the examinations were prepared by the NRC or its contractors. However, the average passing rates tend to fluctuate, and the pilot examination results were comparable to the range of passing rates for FYs 1990 through 1994. The average passing rates for ROs on the facility-prepared operating tests during FYs.1996 and 1997 were consistent with the passing rates on the tests prepared by the NRC during those same years. Although the passing rate on the facility-prepared SRO op.erating tests j during FY 1997 was slightly lower than in FY 1995, it was the same as the passing rate l
~ on the NRC-prepared operating tests during FY 1996.
The NRC staff believes that the lower passing rate on the facility-prepared written l examinations for reactor operators and senior operators may be a reflection of the NRC
- . examiners' increased concentration on the psychometric quality (e.g., the cognitive level at which the questions are written and the plausibility of the distractors or wrong answer choices) in addition to the technical quality of the examinations. Although the NRC did not intend for the level of difficulty or the failure rate of the examinations to increase, it is possible that the NRC examiners' efforts to achieve NRC standards regarding the cognitive level of the questions and the plausibility of the distractors have affected the l- discrimination validity of the examinations. Consequently, those marginal applicants who rnay have passed an examination on which two of the distractors could often be eliminated as implausible are now having more difficulty. However, considering the historical fluctuation in the average examination passing rates and the other factors (e.g., training program quality and screening of applicants by facility licensees) that could be responsible for some or all of the observed decline, the staff has concluded that any increase in the level of difficulty is not significant.
If the NRC decides to prepare the examination in lieu of accepting an examination
' prepared by the facility licensee, the NRC examiners will use the same procedures and guidance (i.e., NUREG-1021) that the facility licensees would use to prepare the examinations.
NRC Staff When the staff developed the pilot examination process, it estimated that it would take l ~approximately 370 examiner-hours to review and otherwise prepare for, administer, I grade, and document an average operator licensing examination. Although a number of the proposed examinations consumed more NRC resources than expected to achieve NRC's standards, the resource burden was generally offset by other examinations that required less effort to review and revise. Therefore, the NRC staff continues to believe that the use of contractors in the operator licensing program (except for the generic l l 1 L
l l ! l l fundamentals examination) can be eliminated and that the revised initial operator ! licensing and requalification inspection programs can be implemented with the existing NRC staff. Before initiating the pilot examination and transition process at the beginning of FY 1996, the NRC spent approximately $3 million per year for contractor assistance l to prepare'and administer initial examinations and assist with requalification inspections. l' During FY 1996, when approximately 40 percent (27/68) of the examinations were ! voluntarily prepared by facility licensees, the NRC spent approximately $1.6 million for I contractor assistance in those areas. In FY 1997, facility participation increased to l include approximately 75 percent (41/55) of the examinations, and the NRC's spending on contractor assistance for the licensing examinations and requalification inspections decreased to approximately $0.5 million (approximately 3 full-time equivalents (FTEs)). l During FY 1997, the NRC expended approximately 16.8 staff FTEs to conduct all the l site-specific initial operator licensing examinations, plus 3.8 staff FTEs for generic operator licensing activities. The licensed operator requalification inspection program consumed approximately 4.5 additional staff FTEs (plus a minimal level of contractor support), bringing the total resources used for the operator licensing program in FY 1997 to approximately 28.1 FTEs. This actual resource burden is generally consistent l with the budget estimates prepared before beginning the pilot examination program and 1-supports the staff's conclusion that the revised examination process is an efficient attemative to the traditional NRC-prepared examinations. The NRC resource burden j should decrease further if and when the rule is implemented (which would cause the ! number of NRC-prepared examinations to decrease) and the quality of the facility-prepared examinations improves. I , However, experience during the pilot program has shown that the quality of the facility-l prepared examinations can vary widely, making it difficult to predict the amount of time l necessary for the NRC to review and revise any particular examination so as to meet l NRC standards. Although the staff had expected the quality of the examinations to l improve over time, some of the proposed examinations continue to require more l changes than anticipated. The added examiner workload could increase the possibility I of lower examination quality, raise the NRC's cost, and affect the staff's ability to satisfy the facility licensees' needs for licensing examinations. As noted in SECY-97-079, the Office of Nuclear Reactor Regulation (NRR) has issued a memorandum to the regional administrators emphasizing the importance of (1) assigning adequate resources to carry out the operator licensing task, (2) completing a thorough review of every facility-prepared examination, and (3) not administering any examination that does not meet NRC standards for quality and level of difficulty. 1
- 5. Decision Rationale The proposed amendments to 10 CFR Part 55 will require all power reactor facility licensees to prepare the entire initial operator licensing examinations and to proctor and grade the written portion of the examinations. The qualitative assessment of costs and benefits discussed above, leads the NRC to conclude that the overall impact of the rulemaking would not significantly increase licensee costs and could eventually result in a savings to licensees as they become more familiar with the NRC examination guidelines. Any improvements in efficiency would likely be due, in part, to the facility 10 I
e 1 employees' better understanding of the plant design and operating characteristics and their ready access to the reference materials required to prepare and validate the examinations. The apparent need of some facilities to improve their examination development capabilities in order to achieve the expected standards for examination quality may delay the realization of cost savings for some facility licensees and may limit the overallindustry cost benefit. Facility licensees that do not prepare examinations that meet the NRC's quality standards will likely derive no cost benefit from this action because of the additional effort required to review and rewrite the examinations. The voluntary pilot program has demonstrated that the revised examination process is effective as well as efficient. The pilot examinations, revised as directed by the NRC, were comparable in terms of their quality to examinations prepared by the NRC and its contract examiners. The overall passing rate and average written examination grades on the 68 pilot examinations administered during FYs 1996 and 1997 w,ere the same as, or slightly lower than, those on the power reactor licensing examinations during FY 1995, when all the examinations were prepared by the NRC or its contractors. Therefore, the safe operation of the facilities was in no way compromised. The NRC staff believes that its ability to focus on operator performance and its core of experience may improve if the pilot process is implemented on an industrywide basis because every applicant will be directly observed by an NRC. employee. Before beginning the transition process, contract examiners administered about half of the operating tests and collected the observations that formed the basis for the NRC's - licensing actions. The revised process will enable the NRC examiners to accrue more experience in a shorter period of time and may improve the consistency of the operating test evaluations and the licensing decisions. Although the total number of procedures , reviewed in the process of examination development may decrease under the revised j method, NRC examiners are still expected to review and identify discrepancies in the procedures that will be exercised during the plant walk-through test and the dynamic simulator scenarios. Alternative 3 was selected as the preferred attemative because it has the potential to save the facility licensees money as they oecome proficient in preparing the examinations, has negligible impact upon operator license applicants, provides a substantial cost savings to the NRC, and has the potential for an improvement in the staff's experience and ability to focus on operator performance. 6.- Implementation Schedule The rule will be implemented 180 days after the date on which it is published in the Federa/ Register. No effect on other schedules is anticipated. l 11
I Attachment 2 l
" Response to CRGR Charter Questions" 2PT55FNL CRG
CRGR CHARTER REVIEW PACKAGE PROPOSED ACTION: Issue a final rule that requires power reactor facility licensees to prepare the initial operator licensing written examinations and, upon i approval by an authorized representative of the facility licensee, to ! submit them to the NRC for review and approval. Facility licensees will also be required to proctor and grade the written examinations, as
. well as establish, implement and maintain procedures to prevent compromises of the integrity of any application, test or examination required by 10 CFR 55. The process is described in detail in Revision 8 of NUREG-1021," Operator Licensing Examination Standards for l Power Reactors," and, with Commission approval, will be implemented on an industrywide basis six months after the rule and the revised NUREG are published. j CATEGORY: - 2 RESPONSE TO REQUIREMENTS FOR CONTENT OF PACKAGE SUBMITTED FOR CRGR REVIEW (Ref: CRGR Charter, Rev. 6, Section IV.B, " Contents of Packages submitted to CRGR")
.lV.B.(l) The proposed generic requirement or staff position as it is proposed to be j sent out to licensees. Where the objective or intended result of a proposed {
generic requirement or staff position can be achieved by setting a readily quantifiable standard that has an unambiguous relationship to a readily measurable quantity and is enforceable, the proposed requirement should merely specify the objective or result to be attained, rather than prescribing to the licensee how the objective or result is to be attained. By publishing the final rule in the Federal Register and issuing Revision 8 of NUREG-1021, " Operator Licensing Examination Standards for Power Reactors," (Attachment 3), the staff will inform power reactor facility licensees of a change in the operator licensing process such that facility licensees will be required to prepare the initial operator licensing examinations at their facility. Facility licensees will be expected to develop and submit their proposed examinations (including the written examination, the walk-through, and the dynamic simulator tests) in accordance with the instructions contained in NUREG-1021, the same NUREG that the NRC would use if it were to develop the examinations. An authorized representative of the facility licensee will be required to approve the examinations before they are submitted to the NRC. Pursuant to Section 107 of the Atomic Energy Act, which requires the Commission to prescribe uniform conditions for licensing operators, facility licensees will have to obtain approval from the NRC if they wish to use attemative testing methodologies. After the NRC has reviewed and approved the examinations, the facility licensees will administer and grade the written examinations as prescribed in NUREG-1021. The NRC will continue to administer and grade the operating tests, review and approve the written examination results recommended by the facility licensee, and make the finallicensing decisions. mwseno . l IV.B.(ii) Draft staff papers or other underlying staff documents supporting the ; requirements or staff.positions. (A copy of all materials referenced in the i document shall be made available upon request to the CRGR staff. Any l
- Committee member may request CRGR staff to obtain a copy of any reference material for his or her use.) ,
SECY-95-075, " Proposed Changes to the NRC Operator Licensing Program,"
. described the staffs intent to revise the process for examining applicants for reactor operator (RO) or senior reactor operator (SRO) licenses at power reactor i facilities.
. l The SRM of April 18,1995, advised the staff that the Commission did not object to the initiation of a transition process to revise the operator licensing program. It
'also directed the staff to carefully consider the experience from the proposed pilot
.- examinations, to keep the Commission informed of progress with the process, and to obtain formal Commission approval before fully implementing the revised program.
Generic Letter 95-06, " Changes in the Operator Licensing Program," notified licensees of the NRC's intent to change the examination development process and solicited volunteers to participate in a pilot program that would evaluate and refine the new process.' ; Federal Register notice 61 FR 6869, " Operator Licensing Examination Standards Draft Revision; Notice of Availability and Request for Comment," was published on February 22,1996, soliciting public and industry comments on draft Revision 8 of NUREG-1021, " Operator Licensing Examination Standards for Power Reactors," , which reflected the new examination development process and incorporated !
' lessons leamed during the pilot examination program. !
SECY-96-123, " Proposed Changes to the NRC Operator Licensing Program," dated June 10,1996, discussed the results of the pilot examinations and recommended that 10 CFR 55 be amended to require power reactor facility licensees to prepare the initial operator licensing examinations. (The staff had briefed the CRGR on the pilot program on May 1,1996, and was subsequently informed that it could not implement the proposed changes with a GL, and that it
- would have to change the regulation to require facility licensees to prepare the licensing examinations.)
The SRM of July 23,1996, authorized the staff to continue the pilot examination process on a voluntary basis and directed the staff to develop a detailed rulemaking plan to justify the changes that may be necessary to 10 CFR 55. The Commission also directed the staff to address the pros, cons, and vulnerabilities associated with the revised examination process in order to facilitate a Commission decision on whether to implement the revised process on an industrywide basis. SECY-96 206, "Rulemaking Plan for Amendments to 10 CFR Part 55 to Change Licensed Operator Examination Requirements," of September 25,1996, forwarded the requested rulemaking plan and a discussion of the pros, cons, and
. mww.cno .
vulnerabilities of the pilot process to the Commission. The paper also discussed the industry's comments on the draft version of Revision 8 of NUREG-1021 and the additional changes that had been made in the document (i.e., interim Revision
- 8) since it was issued for public comment in Februsry 1996.'
An SRM of December 17,1996, directed the staff to implement interim Revision 8 of NUREG-1021 on a voluntary basis and to proceed with the proposed rulemaking. Generic Letter 95-06, Supplement 1, " Changes in the Operator Licensing Program," issued January 31,1997, notified licensees of the results of the pilot program described in GL 95-06 and of NRC's decision to change the operator licensing process so that facility licensees may voluntarily prepare the operating tests and prepare, administer, and grade the written examinations that the NRC
. will review, approve, and use to determine the competence of operator license applicants at power reactor facilities.
Federa/ Register notice 62 FR 8462, " Operator Licensing Examination Standards Interim Revision; Notice of Availability," was published on February 25,1997, making available a revised. NUREG-1021 which incorporated the examination development process described in Generic Letter 95-06, " Changes in the Operator Licensing Program," dated August 15,1995, and permitted power reactor facility licensees to continue preparing their initial operator licensing examinations on a voluntary basis pending an amendment to 10 CFR Part 55 that will require facility participation. This Interim Revision 8 incorporated lessons leamed during a pilot examination program conducted from October 1995 to April 1996 and industry recommendations submitted in response to the NRC's request for public comments published in the Federel Register on February 22,1996 (61 FR 6869). SECY-97-079, " Proposed Rule - Initial Licensed Operator Examination Requirements," of April 8,1997, forwarded the proposed rule to the Commission. An SRM of June 26,1997, approved the publication of the rule in the Federal Registerfor a 75-day comment period. The Federal Refster notice (62 FR 42426) of August 8,1997, published the proposed rule enJ invited public comments on interim Revision 8 of NUREG-1021 because facility licensees will be expected to use it to prepare the licensing examinations. IV.B.(lii) . Each proposed requirement or sta# position shall contain the sponsoring office's position as to whether the proposal would increase requirements or sta# positions, implement existing mquirements or sta# positions, or would relax or reduce existing requirements or sta# positions.
. The staff believes that the proposed changes in the exam development and administration portion of the overall examination process do not increase or reduce existing requirements for licensing operators at nuclear potrar facilities. '
The Atomic Energy Act (AEA) requires the NRC to determine the qualifications of m.sc .. . .. .
Individuals applying for an operator license and to issue licenses as appropriate. Operator license applicants are required by 10 CFR 55 to pass an examination satisfying basic content requirements specified in the regulation. However, Part 55 does not state who will write, administer, or grade the examination. The NRC has traditionally accomplished those tasks using NRC regional or contractor employees.' To ensure uniformity (as required by the AEA), those employees are required to comply with NUREG-1021, which contains specific requirements and
. guidelines for the development, administration, and grading of operator licensing (and requalification) examinations, as well as instructions for administering the operatorlicensing process.
The NRC will continue to review and approve every examination before it is administered to ensure that it meets NRC standards for content, format, quality, level of knowledge, and level of difficulty as specified in NUREG-1021. Facility licensees will proctor and grade the written examinations in accordance with NRC procedures, but NRC examinera will continue to independently administer and grade the operating tests. The NRC will review and approve the written examination grading results, including any recommended answer key changes and question deletions, provided by the facility licensee. The NRC will continue to make all licensing decisions and administer the appeal process for applicants who fail the examination and are denied a license. The specific procedural changes required to implement the new examination process are administrative in nature and limited to those required to maintain the security, integrity, quality, and consistency of the examinations. The basic structure and conter;t of the examinations are specified in Part 55 and will not be changed.
~
IV.B.(iv) The proposed method of implementation with the concurrence (and any comments) of OGC on the method proposed. The concurrence of affected program offices or an explanation of any nonconcurrences. Upon approval by the Commission, the final rule will be published in the Federal Registerand the revised examination process will be implemented by issuir.g final Revision 8 of NUREG-1021. The Office of the General Counsel has no legal objection to the staff's position contained in the regulation or the NUREG. No other program offices are affected. IV.B.(v) Regulatory analyses generally conforming to the directives and guidance of
. NUREGIBR-0058 and NUREG/BR 0184, as applicable. (This does not apply
. for backfits that ensure compliance or ensure, define, or redefine adequate protection. In such cases, for power reactors, a documented evaluation is required as discussed in IV.B.(ix).)
A qualitative assessment of costs and benefits associated with the proposed amendments to 10 CFR Part 55 requiring all power reactor facility licensees to prepare the entire initial operator licensing examinations and to proctor and grade the written portion of the examinations was performed (Ref. Enclosure 3 of Attachment 1). Based on the assessment, the NRC concluded that the overall impact of the rulemaking would not significantly increase licensee costs and could wnw acao 4-in- -.. T- .- : - ,
o 1 (' ; o )' ever*Jally result in a savings to licensees as they become more familiar with the NRC examination guidelines. Improvements in efficiency would likely result due to the facility employees' better understanding of the plant design and operating ! characteristics as well as their ready access to the reference materials required to prepare and validate the examinations. For some facility licensees, added efforts to achieve the expected standards for examination quality may delay the realization of cost savings and may limit the overallindustry cost benefit. Facility licensees that do not prepare examinations that meet the NRC's quality standards
.willlikely derive no cost benefit from this action because of the additional effort required to review and rewrite the examinations.
The revised process has the potential to improve the overall quality of the licensing examinations as it enables NRC examiners to focus more on the psychometric quality of examinations (e.g., the cognitive level at which the questions are written ! and the plausibility of the distractors or wrong answer choices) prepared by the ; facility licensees than on the technical accuracy of the examinations, which was their primary focus when the examinations were prepared by NRC contractors. Additionally, the revised process requires NRC staff examiners to conduct all l operating tests without the use of contractors. IV.B.(vi) Identification of the category of reactor plants to which the generic requirement or staff position is to apply (that is, whether it is to apply to new plants only, new OLs only, OLs after a certain date, OLs before a certain , date, all OLs, license renewals, all plants under construction, all plants, all l
' light water reactors, all PWRs only, some vendor types, some vintage types
, such as BWR 6 and 4, jet pump and nonjet pump plants, etc.).
The amended regulation will apply to all power reactor facility licensees. IV.B.(vil) For power reactor backfits other than compliance or adequate protection backfits, a backfit analysis as defined in 10 CFR 50.109. The backfit analysis shall include, for each category of reactor plants, an evaluation which demonstrates how action should be prioritized and scheduled in light of other ongoing regulatory activities. The backfit analysis shall document for consideration information available concerning any of the following factors as may be appropriate and any other information relevant and material to the proposed action. No backfit analysis is necessary. The pertinent part of 10 CFR 50.109(a)(1) defines backfitting as "the modification of or addition to ... the procedures or organization required to ... operate a facility; any of which may result from a new or amended provision in the Commission's rules or the imposition of a regulatory staff position interpreting the Commission's rules that is either new or different from a previously applicable staff position...." Part 55 addresses the qualifications and requirements for operators' licenses and changes are not per se subject to the backfit rule in Part 50. Changes to there requirements could be included within the backfit definition of" procedures or organization required to ... operate a facility;" however, in this case, the shift of responsibility from the NRC staff (or its contractors) to the facility licensee for developing and administering the initial written operator license examination and developing the initial operating tests, and mw teno establishing procedures to control examination security and integrity would not constitute a " modification of the procedures required to operate a facility" within the scope of the backfit rule.- Therefore, no backfit analysis is necessary. This rule does not affect the basic procedures for operator license qualification, i.e., the required training programs, the required testing, the content and format of the exams, the grading of the exams, or the basis for issuing an operator license. The rule is intended to ensure that the format, content, and quality of the initial examination will not be modified. To that end, the rule requires the NRC to review and approve the written examinations and operating tests developed by the
. licensees. The NRC also retains its discretion to determine whether to administer
~ the initial written examination itself, as well as continuing to determine whether to grant or deny an application for a reactor operator or senior reactor operator license, and consider applicants' appeals.
The licensee's organizational structure required to operate the facility will not be 3 modified. All reactor licensees have a training component as part of their organizational structure, and this rule does not alter that organizational structure. Although, the rule could have an "effect" on the licensee's organization, it does not require any modification to the organizational structure. IV.B.(vii)(a) Statement of the specific objectives that the proposed action is designed to achieve: The specific objective of the final rule is to change the operator licensing process
, to require power reactor facility licensees to prepare the initial operator licensing examinations at their facility. Facility licensees will be expected to develop and '
. submit their proposed examinations (including the written examination, the walk-through, and the dynamic simulator tests) for NRC review and approval, in accordance with the instructions contained in Revision 8 of NUREG-1021,
" Operator Licensing Examination Standards for Power Reactors," the same NUREG that the NRC would use if it were to develop the examinations. An authorized representative of the facility licensee will be required to approve the examinations before they are submitted to the NRC.
The proposed change to the rule was initiated because of a substantial reduction in the resources allocated to the operator licensing program, in particular, the elimination of contractor support for site-specific operator licensing activities. The resulting changes in the Examination Standards (NUREG-1021) are primarily
, administrative in nature and do not substantially alter the format or content of the license examinations. The changes are not intended or expected to substantially c ,
alter the overall protection of the public health and safety or the common defense and security. The proposed changes were originally described in Generic Letter 95-06,
" Changes in the Operator Licensing Program," and tested during pilot examinations conducted at 22 volunteer facilities during the period from October 1, 1995, through April 5,1996. Lessons lemmed from the pilot examinations have been incorporated in the revised NUREG. The proposed revision also formally implements Revision 1 of NUREGs-1122 and 1123, the Knowledge and Abilities 4
m w neno 4
p L. Catalogs for pressurized and boiling water reactors, respectively, which were recently reorganized and updated. Additionally, the revision supersedes l NUREG/BR-0122, " Examiners' Handbook for Developing Operator Licensing l Written Examinations," and incorporates other minor improvements and l clarifications that were recommended by industry groups and NRC staff. , NUREG-1021 was reformatted to more clearly identify the various organizational responsibilities and is being reissued in its entirety. The significant changes that
' have been made to each Examination Standard are outlined in the Executive Summary of Attachment 3.
IV.B.(vii)(b) General description of the activity that would be required by the licensee or applicant in order to complete the action: Under the proposed change to Part 55, facility licensees will assume primary responsibility for preparing the initial operator licensing examinations at their facility. The amendment requires facility licensees to submit, upon approval by an authorized representative of the facility licensee, each examination and test to the NRC for review and approval. Facility licensees will develop their proposed l examinations (including the written examination, the walk-through, and the l dynamic simulator tests) in accordance with the instructions contained in NUREG-1021, the same NUREG that the NRC would use if it were to develop the j examinations. Any licensee request (s) for attemative testing methodologies will be subject to a ! review pursuant to Section 107 of the Atomic Energy Act, which requires the Commission to prescribe uniform conditions for licensing operators. After the NRC has reviewed and approved the examinations, the facility licensees , will administer and grade the written examinations as prescribed in NUREG-1021. j The NRC will continue to administer and grade the operating tests, review and approve the written examination results recommended by the facility licensee, and ! make the finallicensing decisions. l
- IV.B.(vil)(c) Potential change in the risk to the public from the accidental offsite release of radioactive material: i The staff does not expect that the proposed changes to Part 55 and to NUREG-1021 will increase or decrease the risk to the public from the accidental offsite release of radioactive material. !
IV.B.(vil)(d) Potential impact on radiological exposure of facility employees and other onsite workers: The proposed changes to Part 55 and to NUREG-1021 will have no significant impact on the radiation exposure of facility employees and other onsite workers. I' I o m m.cao - i
IV.B.(vii)(e) installation and continuing costs associated with the action, including the cost of facility downtime or the cost of construction delay: The NRC has depended on NRC employees and contractors to draft and administer the initial operator licensing examinations required by 10 CFR Part 55. In accordance with 10 CFR 170.12(i), the cost of NRC time spent and any related contractual costs are billed directly to the facility licensees that receive the examination services. Under the proposed change, facility licensees will assume responsibility for developing the examination materials, thereby allowing the NRC to discontinue the use of contract examiners, resulting in a commensurate reduction in Part 55 review fees billed to the facility licensees. The charges that will be passed on to each facility licensee pursuant to 10 CFR 170.12(i) will be based on NRC time spent to: a) review and approve the proposed examinations; b) administer and grade the operating tests; c) review and approve the written examination results recommended by the facility licensee; and d) make the final licensing decisions. It is expected that the amount of NRC time spent in examination review will be inversely proportional to the quality of the proposed examinations provided by the facility licensee. This change will give facility licensees more control over the cost of their examination services because they will be in a position to manage the quality of the product that is submitted to the NRC. Although several of the pilot examinations did take more NRC staff time than expected to review, the staff believes that cost reductions may be realized as facility licensees gain experience with the NRC examination requirements and the quality of the proposed examinations improves. The staff expects that facility licensees will eventually be able to develop quality examinations in less time than the NRC or a contractor because the facility employees have more detailed knowledge of their facility and easier access to the reference materials required to prepare the examinations. Although several of the pilot examinations required more time to write the examinations than usually required by experienced NRC examiners or contractors, the industry believes that was due to a lack of familiarity with NRC examination requirements and expects that efficiency will improve once facility licensees gain experience in writing the examinations. So far, average quality has not improved. It should be noted that facility licensees will have the option of retaining the services of a contractor to draft the license examinations as the NRC has typically done. The cost of using this option is expected to be comparable to the contractor service fees that the NRC has passed on to facility licensees. In summary, the staff views this process change as resource neutral, and possibly resulting in a resource savings to licensees over time. The proposed change is not expected to affect facility downtime or the cost of construction. IV.B.(vil)(f) The potential safety impact of changes in plant or operational complexity, including the relationehip to proposed and existing regulatory requirements and staff positions: The proposed revision does not affect either plant or operational complexity and is m o m eno , F'-
not expected to significantly impact plant safety. The Atomic Energy Act (AEA) requires the NRC to determine the qualifications of
! individuals applying for an operator license and to issue licenses as appropriate.
Operator license applicants are required by Part 55 to pass an examination satisfying basic content requirements specified in the regulation. However, Part 55 does not state who will write, administer, or grade the examination. The NRC has historically accomplished those tasks using NRC regional examiners or contractor employees. To ensure uniformity (as required by the AEA), those employees are required to comply with NUREG-1021, which contains specific requirements and guidelines for the development, administration, and grading of operator licensing (and requalification) examinations, as well as instructions for administering the I
. operator licensing process. 1 Although facility licensees will assume responsibility for developing the initial
. operator licensing examinations, the NRC will continue to review and approve every examination before it is administered to ensure that it meets NRC standards for content, format, quality, level of knowledge, and level of difficulty as specified in NUREG-1021, Facility licensees will administer and grade the written examinations in accordance with NRC procedures, but NRC examiners will continue to independently administer and grade the operating tests. The NRC will review and approve the written examination grading results, including any recommended answer key changes and question deletions, provided by the facility licensee. The NRC will continue to make all licensing decisions and administer the I appeal process for applicants who fail the examination and are denied a license.
From the time the pilot program began in October 1995 through the end of ,
- September 1997, the NRC staff reviewed, approved, and administered a total of !
68 examinations that were voluntarily developed by facility licensees in accordance j with the NRC's examination criteria and guidance. ! The NRC reviews showed that although the quality of the proposed examinations varied widely, with essentially all of the examinations requiring some changes and I significant revisions to achieve NRC standards for quality and level of difficulty, the ! exam results were consistent with prior results. The NRC staff originally expected l that examination quality would improve as the industry became familiar with NRC ] examination criteria and expectations and gained experience in preparing the ! examinations. However, some of the proposed examinations continue to require more changes than anticipated. Therefore, the NRC staff has concluded that J examination quality and level of difficulty will require continued attention. I l Facility licensees have been advised that the NRC expects the technical quality of examinations to be high and the number of post-examination changes to be low. i Similarly, regional examiners and management have been instructed not to i approve or administer any examination that does not meet NRC standards for content, formal, quality, level of knowledge, or level of difficulty. The quality assurance checklists in NUREG-1021 have been enhanced in an effort to promote examination consistency and the detection of unacceptable examination materials by NRC examiners. Any examination that is found to be unacceptable will be ; s- delayed or canceled, if necessary, until facility licensees and the NRC can apply I momeno - t
the resources required to upgrade or replace the examination. IV.B.(vil)(g) The estimated resource burden on the NRC associated with the proposed action and the availability of such resources: The voluntary pilot program has resulted in a significant budget reduction for the NRC. Contractor support in this area has historically required approximately $3M per year. NRR has eliminated contractor support for the operator licensing ; program (with the exception of the generic fundamentals examination). ' IV.B.(vil)(h) The potential impact of differences in facility type, design or age on the 1 relevancy and practicality of the proposed action: l The proposed revision will apply to all power reactor facilities equally, regardless of 3 facility type, design, or age. The proposed action is not applicable to non-power reactor facilities. IV.B.(vii)(1) Whether the proposed action is interim or final, and if interim, the justification for imposing the proposed action on an interim basis: The proposed revision is considered to be final, but future improvements in the examination procedures are possible. IV.B.(vil)(j) For both rulemaking actions and proposed generic correspondence, staff evaluation of comments received as a result of the noti. e and comment ]
- process. 1 The 75-day public comment period began when the notice of proposed rulemaking was published in the Federa/ Register (62 FR 42426) on August 7,1997, and closed on October 21,1997. The notice (FRN) requested public comment on the l proposed rule, on the implementation guidance in interim Revision 8 of NUREG-1021, and on the following two questions
- 1
)
- 1. Are there portions of the operator exams that are common to alllicensees, and
. would, therefore, be more efficiently developed by the NRC7
- 2. Is the conclusion in the regulatory analysis correct that it would be less costly for each licens2e to prepare its own initial operator examinations to be reviewed, revised, and administered by the NRC, than to have one NRC contractor prepare these exams for all licensed operators with the costs to be reimbursed by licensee fees?
The NRC received 13 comment letters on the proposed rule; two of the letters arrived after the comment period closed, but they were considered nonetheless. The respondents included three NRC examiners, one contract examiner, five
' nuclear utilities and one utility employee, one non-power reactor facility licensee, the State of Illinois, and the Nuclear Energy institute (NEI), which submitted its
( comments on behalf of the nuclear power industry. l Seven of the respondents (three NRC examiners, one contract examiner, one 1 2PTssmLcRo o
utility employee, one non-power facility licensee, and the State of Illinois) recommended that the rule change be disapproved. Five of the industry respondents (NEl and four utilities) supported the rule change; however, one utility endorsed NEl's comments but stated that it did not agree with the proposed rule in , its present form. NEl and two of the utilities stated that they would rather continue with a voluntary program, but they would support mandatory participation with the { rule change rather than return to the previous process under which NRC contractors wrote most of the examinations. Those comments related to the two specific quettions raised in the proposed rulemaking, and those that have a direct bearing on the rule are discussed in Enclosure 2 of Attachment 1. The comments are categorized as they relate to the vulnerabilities discussed in SECY-96-206 (i.e., quality and consistency; independence and public perception; security; resources; examiner proficiency; and reactor safety). Copies of the commenters letters are provided in Attachment 4. One NRC examiner, NEl, four of the utilities, and the utility employee also provided specific comments and recommendations regarding the . implementation guidance in interim Revision 8 of NUREG-1021. Those are discussed in Enclosure 1 of Attachment 1. IV.B.(vil)(k) How the action should be prioritized and scheduled in light of other ongoing regulatory activities. The following information may be appropriate in this regard:
- 1. The proposed priority or schedule.
- 2. A summary of the current backlog of existing requirements awaiting j implementation. -
- 3. An assessment of whether implementation of existing requirements 1 should be deferred as a result, and '
- 4. Any other information that may be considered appropriate with regard to priority, schedule or cumulative impact. For example, could implementation be delayed pending publin comment?
3 This action has allowed the staff to discontinue the use of contractor support for the development and administration of site-specific initial operator licensing examinations. Before initiating the pilot examination and trans! tion process at the beginning of Fiscal Year (FY) 1996, the NRC spent approximately $3 million per
. year on contractor assistance for initial examinations and requalification inspections. During FY 1996, when approximately 40 percent of the examinations were voluntarily prepared by facility licensees, the NRC spent approximately $1.6 million for contractor assistance (to prepare and administer initial examinations and ,
assist with requalification inspections). In FY 1997, facility participation increased to include approximately 75 percent of the examinations, and the NRC's spending on contractor assistance for the licensing examinations and requalification inspections decreased to approximately $0.5 million. The FY 1998 and FY 1999 budgets are consistent with this proposal and reflect the elimination of contractor support for the operator licensing program (with the m ,.c o- o
i exception of the generic fundamentals examination). If the Commission decides not to amend 10 CFR Part 55 as recommended by the NRC staff, other measures will be needed to ensure that the facility licensees' requirements for licensed operators are satisfied. Such measures might include (1) examination length or format changes that will enable the existing staff to conduct the examination and inspection programs (but may raise concerns regarding program stability and examination validity), (2) scheduling the examinations based on the NRC resources available (which may require the staff to prioritize its activities based on the facility licensees' needs), (3) an increase in direct examiner resources in each regional office to. satisfy the demand for initial licensing examinations, or (4) the restoration of contractor support for the operator licensing program. The uncertainty regarding the industry's continued voluntary support for the program, combined with the fact that contractor support for the site-specific examinations and requalification inspections has already been eliminated, may force the stcff to delay examinations, divert examiners from other planned activities, or reprogram other resources to meet its obligations if the
- implementation date for the rule is delayed significantly.
The staff is recommending that the Commission:
- 1. Anorove the notice of final rulemaking for publication in the Federal Register.
- 2. Certify that this rule, if promulgated, would not have a significant economic impact on a substantial number of small entities to satisfy the requirements of the Regulatory Flexibility Act,5 U.S.C. 605(b).
- 3. Determine that the backfit rule,10 CFR 50.109, does not apply to this rule.
- 4. Determine that neither an environmental impact statement nor an environmental assessment has been prepared because this proposed rule is eligible for a categorical exclusion as defined in 10 CFR 51.22(c)(1). I IV.B.(viii) For each power reactor backfit analyzed pursuant to 10 CFR 50.109(a)(2) (i.e.,
not adequate protection backfits and not compliance backfits), the proposing OfNce Director's determination, together with the rationale for the determination based on the consideration of paragraphs (l) through (vil) above, that: (a) There is a substantial increase in the overall protection of public health and safety or the common defense and security to be derived from the { proposal; and l l (b) The direct and indirect costs of implementation, for the facilities affected, are Justified in view of this increased protection. (a) As discussed in paragraphs (vii), (vii)(a) and (vil)(f), the proposed change is neither intended nor expected to alter the overall level of protection of public { health and safety or the common defense and security. l I i m m eno ' 12-I , l
(b) As discussed in paragraph (vii)(e), NRR Views this process change as - resource neutral, and possibly resulting in a resource savings to licensees ; i over time. j IV.B.(ix) For adequate protection or compliance backfits affecting power reactors, evaluated pursuant to 10 CFR 50.109(a)(4) (a) a documented evaluation consisting of: (1) the objectives of the modification (2) the reasons for the modification (3) If the compliance exception is invoked, the requirement (s) (e.g., Commission regulation, license condition, order) or written licensee commitment (s), for which compliance is sought. (4) If an adequate protection exemption is invoked, the basis for concluding that the matter to be addressed involves adequate protection, and why current requirements (e.g., Commission regulation, license condition, order) or written licensee commitments do not provide adequate protection. l l (b) in addition, for actions that were immediately effective (and therefore , issued without prior CRGR review as discussed in Ill.C) the evaluation ! shall document the safety sigrtificance and appropriateness of the l action taken and (if applicable) consideration of how costs contributed to selecting the solution among various acceptable altematives. 4 The proposed action does not involve an adequate protection or compliance backfit issue. 4 IV.B.(x) For each evaluation conducted for proposed relaxations or decreases in current requirements or staff pos7tions, the proposing Office Director's determination, together with the rationale for the determination based on the considerations or paragraphs (1) Phrough (vil) above, that: (a) The public health and safety and the common defense and security would be adequately protected if the proposed reduction in requirements or positions were implemented, and (b) The cost savings attributed to the action would be substantial enough to justify taking the action. As noted in paragraph (iii), the proposed action does not relax or reduce existing requirements or staff positions. IV.B.(xi) For each request for information from power reactor licensees under 10 CFR 50.54(f) (which is not subject to exception as discussed in ill.A) an evaluation that includes at least the following elements: (a) A problem statement that describes the need for the information in terms of potential safety benefit. 2n e teno (
E (b) The licensee actions required and the cost to develop a response to the information request. (c) ' An anticipated schedule for NRC use of the information. (d)' A statement affirming that the request does not impose new requirements on the licensee, other than for the requested information. (e) The proposing Office Director's determination that the burden to be imposed on the respondents is justifiedin view of the potential safety significance of the issue to be addressed in the requested information. The proposed action does not contain a request for information under 10 CFR 50.54(f). IV.B.(xil) For each proposed power reactor backfit analyzed pursuant to 10 CFR 50.109 (a)(2) (i.e., not adequate protection or compliance backfits), an i assessment of how the proposed action relates to the Commission's Safety { Goal Policy Statement. ' For the reasons stated earlier under IV.B.(vii), no backfit analysis is necessary. Nevertheless, the proposed action is consistent with the Commission's qualitative j safety goals. The staff does not expect that the proposed revisions to the operator i licensing program will alter the current risk to the life and health of any individual member of the public or to contribute to other societal risks. The licensing examinations will continue to be prepared in accordance with the established format and procedures in NUREG-1021. The NRC will remain actively engaged in the examination process to ensure that acceptable levels of quality, effectiveness, objectivity, and independence are maintained. NRC examiners will: , a) Review the examinations in detail to verify that they conform with the applicable guidelines; b) direct Facility licensees to make whatever changes are necessary to bring the examinations into conformance; c) continue to administer and grade the 1 operating tests; and d) review the written examinations after they are graded by the facility licensee. The NRC will continue to issue or deny operator licenses based upon the qualifications and competence of the license applicants. i m w w.cao l Attachment 3
~
NUREG-1021 l
"O3erator Licensing Examination Standarcs for Power Reactors" l l
Revision 8 1 I l t
n i
\UREG-:.02:.
Revision 8 l l L Operator Licensing Examination l Stancarcs for Power Reactors i-l r Manuscript Completed: Month 1998
.Date-Published: Month 1998 Division of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation U.S. ' Nuclear Regulatory Commission Washington, DC 20555-0001
r-ABSTRACT l l NUREG-1021, " Operator Licensing Examination Standards for Power Reactors," establishes o ' the policies, procedures, and practices for examining licensees and applicants for reactor operator and senior reactor operator licenses at power reactor facilities pursuant to Title 10, Part 55, of the Code of FederalRegulations (10 CFR Part 55). The related guidance that was previously published in the " Examiners' Handbook for Developing Operator Licensing Written Examinations (NUREG/BR-0122, Rev. 5, dated March 1990) has been incorporated herein. NUREG/BR-0122 is no longer in effect. These examination standards are intended to assist NRC examiners and facility licensees to
- better understand the processes associated with initial and requalification examinations. The standards also ensure the equitable and consistent administration of examinations for all applicants. These standards are for guidance purposes and are not a substitute for the operator licensing regulations (i.e.,10 CFR Part 55), and they are subject to revision or other changes in intemal operator licensing policy..
i . This htf.m revision p;; ;.F; S;::::i llsr.;;s .. r.- .. ...; r bf.b: :;i;%; ::ca;b;
< :s;7.b; ten; n ; abntri r:: pg.dbslmplements t an amendment to 10 CFR Part 55 that well-requires facility pertepeteenlicensees to prepare the entire operator licensing examination and to proctor and grade the written portion'of the examination. The NRC expects to prepare a limited number of examinations to maintain the proficiency ofitUixaiminers and, as necessary, to ensure quality. T.t NCO lc. tad; t; n;bl: sir. .mr.O ;; 0.:. ;';itbn duf.r.; ce rubm;kb; p.e x: :nd b Ixx ; tra: Cs:::r. O b enjur.;;bn ..l10; Sn;' reb.
Fu :sa .lr._. . .. r. .r.. . ., .. . i. . .This revision will become effective 60 days after its publication is noted in the FederalRegister. The corporate notification letters for specific examinations issued after the effective date will give facility licensees at least 120 days of advance notice that the examinations will be administered in accordance with the revised procedures. I:-"fi llar.:x: 0% sbr.ts.ed 2 p.;pr; 2e:r ;smb;tbra be%r; te d;t ;f 1; l;d;;;.'.";gl7.Or. ret:s (ICN) ;;'e ;-a-red 2 pre;- ;:e Se esmbdbn; ts;d en m;
;;:fx.s br;b = 0; ;:::: :s.r.bf: . ;uf fx.x b 0;nsb L:.".x 05 00, "Obsgs ln 1; Opsda Lbs.dr., "re,; .;.," d;td ." ugust 15,1005. I;-:::::/ l:snxx Se abr.tr ;':;- S; d:::;"24FCN se04:M:f 5 p;:;: ; 2; =;mb;;bre L;;;d en 0; gs:ds.x hs;b.
r . NUREG-1021 iii Revision 8 l i
1
)
t l l 1 Intentionally Blank . J NUREG-1021 .sv Revision 8
TABLE OF CONTENTS ABSTRACT TABLE OF CONTENTS EXECUTIVE
SUMMARY
ABBREVIATIONS ' ES-101 ' Purpose and Format of Operator Licensing Examination Standards ES-102 - Regulations and Publications Applicable to Operator Licensing ES-201 Initial Operator Licensing Examination Process
' ES-202 Preparing and Reviewing Operator License Applications ES-203 .
[ Deleted) . ES-204 < Processing Waivers Requested by Reactor Operator and Senior Reactor OperatorApplicants . ES-205 - Procedure for Adininistering the Generic Fundamentais Examination Program 4
. ES-301 Preparing initial Operating Tests'.:
. - ES-302 Administering Operating Tests to initial License Applicants ES-303 Documenting and Grading initial Operating Tests ES 304 'O;bted)
ES-401 ' Preparing initial Site-Specific Written Examinations ES-402 Administerfng initial Written Examinations ES-403 Grading Initial Site-Specific Written Examinations
, l .
ES '^'. ,
, [0;b ;d]
- ES-501 Initial Post-Examination Activities ES-502 Processing Requests for Administrative Reviews and Hearings After initial License Denial NUREG-1021 v Revision 8 L
TABLE OF CONTENTS ES-601 Conducting NRC Requalification Examinations ES-602 Requalification Written Examinations ES-603. Requalification Walk-Through Examinations
- ES-604 - - Dynamic Simulator Requalification Examinations ES-605 . License Maintenance, License Renewal Applications, and Requests for Administrative Reviews and Hearings ES-701 Administration of initial Examinations ':- c Senior Operatom Limited to Fuel -
Handling ES-702 Administration of Requalification Examinations for Senior Reactor Operators Limited to Fuel Handling APPENDIX A Overview of Generic Examination Concepts APPENDIX B Written Examination Guidelines ~ . _. APPENDIX C Job Performance Measure Guidelines._ , APPENDlX D Simulator Testing Guidelines APPENDIX E' Policies and Guidelines for Taking NRC Examinations APPENDIX F Glossary b W ' rP r
.e
'/ NUREG-1021 vi Revision 8
m-( EXECUTIVE
SUMMARY
J Title 10, Part 55, of the Code of Federal Regulations (10 CFR Part 55) requires applicants for reactor operator (RO) and senior reactor operator (SRO) licenses to pass a written examination and an operating test that are developed and administered in accordance with 10 CFR 55.41 and 55.45 or 55.43 and 55.45, respectively. Although license examiners from the U.S. Nuclear Regulatory Commission (NRC) have historically prepared all of the licensing examinations using facility-provided reference materials. lJ;=this, th r;;2 bm de n;; Archbh , the
- NRC has now amended Part 55 by adding a new section ($55.40) that requires facility -
licensees from-to developing and submitting, upon approval by an authorized representative of the facility licensee, proposed examinations for NRC review and approval. 0; lJRO b; :cen thd Oe::.;i l beans ..;;; th; spr:b; 0; pap;a dre'; n;.T.htma; 2.d ps;;de en E-::;^:i': ici Si bF.b: Op; ; O rl M .d r.;. R.; lJR O i x x thi ;;..L d b.. On 'he au- : n - of7.;lh r.ndep s tr c ;;;".uth'; = m hth pm ,.. n ndth .r ";;'?.; p h s;mb:^.::x 'hd ;;.; ;dm:n?::: ;d b r rider.; ';;b 0;xib : ;^^;; (Ol ) 05 00, "Oh;r.,;; b 0; Opstr Lumb; ";,sm," d O .'. ;;;:t 15,1^^5. "L.tarm;;;, th; ::COspC thd th; %d::y 'i:. xx:' ,cz' : %.T 9b-i^y ;;Ph ph: ;y;'..T; and pasdua; (attic; O 'hd c' TOCO lb;s; se.T.hm) .;;ll enhen .; 2.; 2..%nt =l:d:;/ ;' O.; n;mhth; er.d th;
;O.b ey ch;;; d;;; bpm;..
l _ _ , . . . . , ____________.u___2m2_._,__ m ,e____,__ _..___,_2 ,_ , _ _ . ._ m m _ _ 2 4
'T?- - -
SR, . a n"' " . , -' z : : ';. ~ "" '. ._,-_....s.....-_...-.........72,"."J'".....'.".,",*L""..".'.'.:.";,K1".',"..A"."..'.L".._"..'.'._""'"
= he-Fy t; d;;ip th; semb:th; ;. hen drmed nnnr i. In lieu of requiring a specific ;
facility licensee to prapere the examinations and tests or to proctor and grade the site-specific j wntten examinations, the NRC may elect to perform those tasks in order to maintain the ! proficiency of its examiners or if it has reason to question a licensee's ability to prepare acceptable examinations. y ( ,= Facility licensees are expected to develop and submit their proposed examinations based on , the guidelines and instructions contained herein. Section'107 of the Atomic Energy Act of I
. 1954, as amended, requires the Commission to prescribe uniform licensing conditions for operators.- Therefore, the NRC discourages facility licensees from using testing methodologies that do not conform to the policies, procedures, and practices defined in this NUREG. l Nevertheless, facility licensees may propose deviatione4 rem-altematives to specific guidance in l NUREG-1021, and the NRC will review and rule on the acceptability of the 1 devietsenealtematives.'
(- ~ m ;A The NRC will make a reasonable attempt to administer all license examinations on the dates requested by the facility licensees. At times, however, resource limitations may compel the staff to prioritize its examination review and development activities based on need and safety considerations. Facility. licensees are strongly encouraged to schedule their initiallicense examinationsand to resolve any ap)licant eligibility questions with their NRC regional office before commencing a license train ng class. ! For Revision 8, NUREG-1021 was reorganized to more clearly identify the various organizational responsibilities. It incorporates the methodology and lessons leamed from the pilot examination program described in GL 95-00, n ;..:: ;; ;0er lmpanm;C ;nd 1." ^': n x c.;;nf:I by bf1:^. j ,;z;:, ;h..sd sprees, e .d !JnO semba er.d managers and changes made in response to the public comments on interim Revision 8 solicited in connection with the regulatory amendment noted above. This revision also formally implements Revision 1 of NUREGs-1122 and 1123, the " Knowledge and Abilities Catalogs" for pressurized and boiling water reactors, respectively. NUREG-1021 vii Revision 8 I.
r T EXECUTIVE
SUMMARY
In addition, Revision 8 of NUREG-1021 supersedes Revision 5 of NUREG/BR-0122,
" Examiners' Handbook for Developing Operator Licensing Written Examinations," dated March 11990.V This document reflects the following significant changes from . Revision 7:
- Abbreviations This list has been a dde d to provide a central location for defining the acronyms and abbrMations used throughout this NUREG.
3 ES-101 No significant changes.
. ES-102. N: ;;;n;'s.;nt 2;ng;;.NUREG-1560, " Individual Plant Examination Progmm:
Perspectives on Reactor Safety and Plant Performance," and NUREG-1600,
" General Statement of Policy and Procedure for NRC Enforcement Actions," have been added to the list of documents applicable to,the operator licensing program.
- ES-201 Resources permitting, the NRC will give each facility licensee an opportunity for one initial operator licensing examination per reactor type or site per fiscal year.
- Additional examinations may be scheduled as the workload permits. Each NRC
. region will prepare at least one examination per calendar year.
~
Facility licensees generally conduct the following activities based on the guidance in NUREG-1021: .'
. N. t, .
Observe various examination security and integrity criteria, including restrictions on which personnel can participate in developing the examinations, physical security expectations.and considerations, and limits on the use of examination banks.
~ . ,_ .
At least 60' days before the scheduled examination review date, prepare and submit for NRC review and comment an integrated examination outline, in accordance with ES-301, ES-401, and the associated quality assurance checklist. At least 30 days before the scheduled examination review date, prepare and submit for NRC review and comment the complete examination, in u : accordance with ES-301 and ES-401, along with a statement indicating
-the source of each test item proposed for use on the examination.
2y Make exa'mination changes as agreed upon with the NRC. Facility licensees shall designate a point of contact to work with the NRC chief examiner, ;; w;" ;; . An authorized .m;n;;;.;;nt spresentative of the facility -
, heensee shall review en& approve the submittals before sending them to the NRC for review and comment. The standard also discusses the restrictions on the activibes of instructors and supervisors who have knowledge of the examination
; contents.
NUREi-1021 viii Revision 8 e
s Fi ; j EXECUTIVE
SUMMARY
ES-201 Instructors are no longer restricted from preparing written examination questions based solely on the amount of time they spent training the applicants. The amount of reference material requested from the facility licensee will be
,, ' adjusted based on the NRC's level of involvement in the examination development t
process; ES-202 The eligibility criteria for senior reactor operators limited to fuel handling (LSRfis) have been moved from ES-701 to ES-202. To make 'the standard conform with 10 CFR 55 and current practice, ES-20L now includes a provision requiring that facility licensees submit a written request to have a license examination administered to an applicant.
^
If more than six months pass sincs an applicant's medical examination, the facility must certify that the applicant has not developed any condition reportable under 10 CFR 55.25. The regions will verify that an applicant's name does not nppear on the " Restricted Individual; List" before accepting the application. _ The requirement for five significantienctivity manipulations ~ has been clarified.
/ .
+
;7 ES-204 ' The provision for LSROs to be licensed at more than one site has been moved from ES-701 to ES-204. ; - ~
A y , e, % The regions may, under certain cir'cumstances, waive the requirement for an examination for applicants that were previously licensed at the same facility. i 7: The regions may, under the circumstances specified in 10 CFR 55.31(a)(5), accept
~
an application and adminster an examination before the applicant completes the
, required control manipulabons.
~
The regioris may waive ttia r'equirement for a new medical examination for up to
~
two years from the date of the last examination if the facility licensee certifies that
- the~'applicaht has not developed any medical condition reportable under 10 CFR 155.25. T;[_.(g ES-205' Facility lice should notify the Headquarters Operator Licensing Branch (OLB)
If they must modify their previously submitted registration letter for the generic fundamentals examination by adding or deleting a person. ES-301 Dominant accident sequences, as determined by the facility licensee's probabilistic .
, , risk assessment or individual plant examination, should be considered for sampling during the operating test, w; . , . l A site-specific task list may be used to supplement or override, on a case-by-case basis, selected individual items in the NRC's knowledge and abilities catalog.
NUREG-1021 ix Revision 8 1
)
1
EXECUTIVE
SUMMARY
ES-301 The instructions for developing the operating test outline and the final test items have been separated to facilitate their sequential preparation, review, and approval. Generic guidelines (i.e., those that apply to both initial and requalification examinations) for developing the walk-through and dynamic simulator tests have been relocated to Appendix C, " Job Performance Measure Guidelines," and Appendix D, " Simulator Testing Guidelines," respectively. No more than 80 percent of any applicant's walk-through test may be taken directly from the facility licensee's item bank without significant modification, and no more than 30 percent of the walk-through may be repeated from the last NRC license examination at the facility. Closed-reference questions should not exceed 30 percent the total questions on either walk-through test category. A quality assurance checklist has been included as an attachment to this standard to highlight various criteria and promote consistency. Each applicant's dynamic simulator test shall include at least one new or significantly modified scenario that the applicant has not had the opportunity to rehearse or practice. A quality assurance checklist has been included as an attachment to this standard to promote consistency by highlighting and suggesting target ranges for various criteria, including simulator critical tasks. The target ranges are based on a study of simulator scenarios used during past initial operator licensing examinations.
~
No dynamic simulator scenarios or job performance measures will be repeated on succe{epays, gyg .gygyg.-{.g.g j.. nm.mm.m
. . . _ . . . . . . . . . . ~ . . , . . . - .
m _ _ . m_ _ . . . . . . ~ . , . . . . ~ . ~ , . . ES-302 NRC examiners may use additional surrogate operators to augment the simulator crews if the technical specifications require the facility licensee to operate with more than two ROs in the control room. A shift technical advisor (STA) may also be used consistent with facility operating practice. SRO-upgrade applicants in an RO~ p6sition do not have to be monitored individually by an NRC examiner during the simulatoftest.
- m. .
The facility licer.*ee and NRC chief examiner should confirm that the simulator instructors staten, programmers' tools, and extemal interconnections do not compromise the integrity of the operating test. Appendix D briefly describes a number of vulnerabilities. The practice of conducting an exit briefing with the facility licensee after the
. , operating tests are complete has been adopted as policy.
NUREG-1021 x Revision 8 l
EXECUTIVE
SUMMARY
ES-302 - The operating test briefing for the applicants has been moved to Appendix E,
" Policies and Guidelines for Taking NRC Examinations."
The regions may delay the operating tests for up to 30 days after the written examinations if necessary to achieve a quality product.
- ES-303 The' simulator operating test grading guidelines for errors having serious safety consequences (including critical tasks) have been clarified. Missing a critical task does not necessarily mean that an applicant will fall the simulator test, nor does success on every critical task prevent the examiner from recommending a failure if the applicant had other deficiencies that, in the aggregate, justify the failure based
- on the competency evaluations.-
The simulator operating test documentation requirements have been increased; examiners must now briefly describe the error that the applicant made to justify a
- grade of"2" on any rating factor.
, The applicants' responses to prescripted JPM follow-upjuostions will be evaluated based prima.rily on safety-significance. .-
.. :R .
ES-401 - This standard now includes instructions,'an example method, and forms for use in systematically developing tho' written exarninstion outline. References to
- NUREG/BR-0122, ." Examiners' Handbook for Developing Operator Licensing Written Examinations,".have been deleted. .
y y References for guidance in developing multiple choice test items have been changed from NUREG/BR-0122 to Appendix B, " Written Examination Guidelines." J .. . .1 ~
. Facility licensees shall submit an outline at least 60 days before the examination review date, followed by the " ready-to-use" examination at least 30 days before the examination review date.t Rg%;
. A site-specific task list may be used to supplement or override, on a case-by-case
-basis,-selected individual items in the NRC's knowledge and abilities catalog; a site-
[ ' specific task _ list may not be used in place of the entire catalog.
%6#
This standarii riow" includes several new criteria developed to ensure the integrity of examinations developed by facility licensees. These criteria include simits on the c# number of questions that can be taken directly from the facility licensee's item bank or can be repeated from earlier quizzes and examinations. y;.x
+.
^'
*' ' 'In'en effort to maintain examination quality and consistency, eHeest-50 to 60
' percent of the questens on the examination shall testbe written at the
" . comprehension / analysis level.
~ NUREG-1021.. xi - Revision 8 l
1
\
EXECUTIVE
SUMMARY
ES-401 As a final check for technical accuracy, facility licensees should consider administering the examination to one or more previously uninvolved licensed personnel (under security agreements). ES-402 Facility licensees will generally administer the written examinations after they are approved by the NRC. The region may delay an examination for up to 30 days after the operating tests if necessary to achieve a quality product. The facility licensees will document for subsequent review by the NRC any questions posed by and answers provided to the license applicants during the examination. If NRC examiners are on site, they may periodically monitor the administration process. The guidelines for briefing the applicants'who will take the examination have been moved to Appendix E. - l The time permitted to take the written examination has been extended from four to five hours. ES-403 Facility licensees should collect and consider any questions and comments made
. by the applicants after the examinations are administered. Facility licensees that prepare and administer the written examinations will grade the examinations, review the grading, evaluate the applicants' performance, and submit the results to the NRC for review and approval. The facility licensee shalljustify all recommended question deletions and changes to the answer key.
The discussion of examination grading quality assurance has been moved to ES-501. "
.. ' . C^ .
ES-501 This standard summarizes the documentation that facility licensees are expected to
. provide to the NRC if they develop and administer (in the case of the written) the licenseq.examinations.g< ,
In' addition, the standard now summarizes the post-examination quality assurance review process that was previously contained in ES-403. _ . :--4 3
'If a facility hoensee recommends deleting or changing five percent or more of the questions on a~ written examination that it developed, it may be asked to explain why the changes were necessary.
C
~
The regional offices willmay delay issuing the licenses for applibants who pass the written examination with insufficient margin to ensure that the licensing decision will be sustained if additional questions are deleted or changed upon appeal.
,y TheI' examination report shall address any significant problems that the region or
~ '
facility licensee encountered in developing the examination. NUREG-1021 xii Revision 8 9
EXECUTIVE
SUMMARY
- ES-501 The record keeping requirements, including the submittal of proposed examinations to the public document room (PDR), have been revised to reflect the new examination process.
ES-502 For those denials that are sustained by the regional office on preliminary review, the Chief of the Operator Licensing Branch will determine whether to convene a panel or evaluate the appeal intemally. consist of three certified examiners, on will e of which Appeal panels,chairperson. be designated when required, will nor ES-601 The NRC will continue to monitor licensees for indications of undue stress during - requalification examinations, however the stress feedback forms have been eliminated. ES-602 References to NUREG/BR-0122 have been changed because the guidance for . developing multiple choice test questions is now in Appendix B. Because the guidelines previously documented in' Attachment 1 to ES-602,
" Policies and Guidelines for Taking NRC.. Written Examinations," are generally the same for initial and requalification examinations, they have been consolidated in Appendix E. ."
No more than 50 percent of the' questions on the exarnination may be duplicated from any past examination or combination of ~esaminations during the current requalification training cycli. m .s y ES-603. Attachment 1 to ES403,"" Guidelines for the' Development and Use of Altemate ! Path JPMs"; Attachment 3, " Walk-Through Evaluation Guidelines"; Form ES-603-1, "JPM Quality Checklist"; and Fonn ES-603-2, "JPM Worksheet," have been moved j to Appendix C because they apply to both initial and requalification examinations. Attachment 2 to ES403, " Briefing Checklist - System Walk-Through," has been moved to Appendix E.O r ' gg i y'm ; ES-604 Attsiment 1 to ES404,'" Critical Task Methodology,".and Attachment 3, j
" Quantitative and Qualitative Scenario Attributes," have been moved to Appendix D i because they apply to both initial and requalification examinations. Attachment 2,
" Dynamic Simulator Briefing Checklist," has been moved to Appendix E.
e j ES-605 The policy on standing proficiency watches and renewing inactive licenses has been clarified. The regions may, under certain circumstances, authorize an operator to temporarily
' suspend participation in the facility licensee's requalification training program .
. , y ES-701 ~ . The eligibility criteria for LSROs have been moved to ES-202, and the provision for LSROs to be licensed at more than one site is now discussed in ES-204.
NUREG-1021 xiii Revision 8 l l 4 h
EXECUTIVE
SUMMARY
ES-701 The standard has also been edited to clarify the differences between the full-scope SRO and the LSRO examinations. The number of systems tested in Category B of the operating test has been decreased from six to five, and the requirement to test a normal evolution during each of the two discussion scenarios in Category C has been eliminated. The number of subject areas to be evaluated with questions when it is not practical to conduct or simulate a job performance measure has been decreased. ES-702 This standard has been edited to clarify the differences between the full-scope and
. .the LSRO requalification examinations.
Whenever possible, the facility licensee should include an LSRO on the requalification examination team. - Appendix A This new appendix discusses the generic examination concepts that play a role in the operator licensing process. It includes much of the information that was previously contained in NUREG/BR-0122 as well as discussions of new topics that have a bearing on the level of difficulty of an examination. Appendix B This new opendix inM the guidance for developing written test questions that was p aviously contained in NUREG/BR-0122.:lt focuses primarily on multiple-choice quustions, the only type currently permitted on the initial operator licensing examination, and includes examples to illustrate various psychometric concepts. W ..M .:l Appendix C This new appendix summarizes the guidelines concerning job performance
. measurec that apply to both initial and requalification examinations. Much of this
, ~ information.was previously contained in Attachments to ES-603. There are no significant policy changes. .
_7 Appendix D This new appendix summartzes the dynamic simulator scenario guidelines that apply to both.the initial and requalification examination programs. Much of the
. Information was previously contained in ES-301 and Attachments to ES-604, i' %!,ph .
Appendix D also. describes a number of simulator security vulnerabilities (related to features of the instructor's station, programmers' tools, and extemal 7 interconnections) that NRC examiners and facility personnel should consider when preparing and administering operating tests. Appendix E This new' appendix summarizes all of the policies and guidelines applicable to
. , . . examinees who will be taking an initial or requalification examination. The
.information was previously contained in ES-302, ES-402, ES-602, ES-603, and ES-
" 604; there are no significant policy changes.
The policy on examining senior reactor operator upgrade applicants on the control
- boards has been added to the simulator test briefing list.
NUREG-1021 xiv Revision 8 l 9
F l' EXECUTIVE
SUMMARY
Appendix F This new appendix provides a central location for defining terms used throughout this NUREG. ! 1 Any reference to the plant's technical specifications includes the plant's other technical requirements documents, whether stated or not. I l l i i 'l y l NUREG-1021 xv Rev ii s on 8 4 e_%>g,+- La
7 l i L ABBREVIATIONS
' AC . attemating current . ,
ADS' automatic depressurization system 1AFW. auxiliary feedwater ! ANSI /ANS . American National Standards Institute /American Nuclear Society { AO- auxiliary operator ' AOP: abnormal operating procedure l APRM - average power range monitor
.ARP . alarm (or annunciator) response procedure ATWS[T] anticipated transient without scram [ trip) I t :B&W Babcock and Wilcox - I BWR boiling water reactor :
i_ CAL- - confirmatory action letter . CCW- - component cooling water { CFR Code of FederalRegulations i CRD- control rod drive L CRT criterion-referenced test ~ CT critical task i CTMT. ' containment - CVCS chemical and volume control system DAS dominant accident sequence DC '
. direct current l
.DHR decay heat removal _
.DRCH. Division of Reactor Controls and Human Factors EAL- ' emergency action level i ECA - emergency contingency action (procedure) ;
ECCS. emergency core cooling system - 1 ECP. estimated critical position .
- l. .EDG emergency diesel generator i EHC electrohydraulic control l l EOP. emergency operating procedure I
EPIP Jemergency plan _ implementing procedure
-EQB examination question bank 1
ES-examinshon standard ESF engineered safety feature
-FHE'? fuel _ handling equipment FRP functional recovery procedure i FSAR final safety analysis report GFE l generic fundamentals examination GL ~ ,
~genericletter -
GUI . graphic user interface
-HP.' u
- health physics HPCI " " ~ high pressure coolant injection HPCS' high pressure core spray HVAC.- heating, ventilation, and air conditioning IC instrumentation and control NUREG-1021 xvi . 3 vision 8
ABBREVIATIONS INPO Institute of Nuclear Power Operations IP. inspection procedure IPE ! individual plant examination j IRM intermediate range monitor JPM job performance measure i JTA job task analysis K/A knowledge and ability _ . KSA knowledge, skill, and ability LCO limiting condition for operation LER licensee event report LOCA loss of coolant accident LPCI low pressure coolant injection . LPCS low pressure core spray LPRM local power range monitor LSRO limited senior reactor operator MIP master inspection plan MSIV main steam isolation valve ' NEl Nuclear Energy Institute NRC Nuclear Regulator Comrnission NOP normal operating procedure NRR Office of Nuclear Reactor Regulation -
~..
NRT l norm-referenced test NWPA ! Nuclear Waste PolicyAct(of1982) ; OJT on-the-job training ! OLA operatorlicensing assistant OLB Operstor Licensing Branch-OLTS operatorlicensing tracking system OMB Office of Management and Budget PCIS primary containment isolation system ) i PDR public document room - ' PORV power-operated relief valve PPR plant performance review PRA- probabikstic risk assessment PWR pressurized water reactor QA.,- quality essurance RBM rod block monitor RCA radiologically controlled area RCIC reactor core isolation cooling RG: Regulatory Guide RHR: .f~ residual heat removal RMCS- reactor manual control system : RO ~ reactor operator ROI report on interaction RM radiation monitor RPIS rod position indication system NUREG-1021 xvii Revision 8
ABBREVIATIONS RPS reactor protection system RPV reactor pressure vessel RWST refueling water storage tank
'S satisfactory SALP systematic assessment of licensee performance SAT systems approach to training SGTS standby gas treatment system SD standard deviation SGTR ~steam generator tube rupture SI safety injection SLC standby liquid control .SME subject matter expert - .
SRO senior reactor operator SRP Standard Review Plan SRV safety relief valve STA shift technical advisor TDAFW(P) turbine-driven AFW(pump) _. TS technical specification (or other technical requirements document) U. unsatisfactory . , UPS uninterruptible power supply . W/T walk-through .
~
k NUREG-1021 xviii Revision 8
ES-101 l PURPOSE AND FORMAT OF OPERATOR LICENSING EXAMINATION STANDARDS )
- (-
' A.' . PURPOSE-Title 10, Part 55, of the Code of Federal Regulations (10 CFR Part 55) requires that applicants for reactor operator (RO) and senior reactor operator (SRO) licenses pass written examinations
. and operating tests (both initially and for requalification). Moreover, the regulations mandate -
~ that the license examinations must be developed and administered in accordance with 10 CFR 55.41 and 55.45 for ROs, or 10 CFR 55.43 and 55.45 for SROs.
The " Operator Licensing Examination Standards for Power Reactors" (NUREG-1021) establish .
~ the procedures and practices for administering the required initial and requalification written examinations and operating tests. These standards describe the provisions of.the act and.
regulations on which the program is based. They also ensure the equitable and consistent administration of examinations to all applica'nts and licensed operators at all facilities subject to the regulations. B. ' FORMAT- _[. Each standard explains the rules, procedures, and practices for a particular aspect of the propram. For ease of reference, each examination standard (ES) is assigned a three-digit numoer, and related standards are grouped together in the sense that standards beginning with the same digit apply to related aspects of the program, as follows: ES-1xx - General - ES-2xx - Initial pre-examination activities ES-3xx - Initial operating tests '
; ES-4xx - Initial written exam ~mations ES-5xx - Initial post-examination activities
. ES-6xx - Requalification examinations ES-7xx - Fuel handling examinations ^
;+
, + ,
..~-
]
2y l l y # l j
- i
+ .. 7
*. , f NUREG-1021 - .1 of 1 Revision 8 f
ES-102 REGULATIONS AND PUBLICATIONS APPLICABLE TO OPERATOR LICENSING
'A. PURPOSE This standard lists the U.S. statutes and the regulations of the U.S. Nuclear Regulatory Commission (NRC) that establish the requirements for conducting operator licensing examinations, it also identifiss the regulatory guides and NUREG reports that establish the procedures for implementing the regulations and administering the examinations, as well as standards of the American National Standards institute /American Nuclear Society (ANSI /ANS)-
that may provide additional guidance. Regulatory guides (RGs), NUREG reports, and industry standards are not requirements, except as specified in Commission orders or as committed to by the facility licensee. The appropriate revisions should be consulted as referenced in the facility's FSAR or approved training program.
- The following paragraphs summarize the latest revisions of these documents.
B. STATUTES .
- 1. Atomic Enemy Act of1954 .
Section 107 of the Atomic Enemy Act of 1954 (42 U.S.C.'2I37), as amended, requires that the NRC prescribe uniform conditions forlicensing individuals as operators of production and utilization facilities, determining the qualifications of these individuals, i and issuing licenses to such individuals.' '
- 2. . . Nuclear Waste Policy Act of1982 -
[ Section 306 of the Nuclear Waste Policy Act of 1982 (42 U.S.C.10226, 96 Stat. 2201 at 2262- 2263) directs the NRC to establish requirements goveming (1) simulator training for applicants for open es licenses and for operator requalification training programs, (2) NRC administration of requalification examinations, and (3) operating tests at civilian nuclear power plant simulators. C. REGULATIONSi ~
- 1. 10 CFR Part 2. Rules'of Practice The regulatior$s in 10 CFR Part 2 govem the conduct of all proceedings under the Atomic Enemy Act of 1954, as amended, and the Enemy Roomanization Act of 1974 with regard to (a) granting, suspending, revoking, amending, or taking other action with
, respect to any license; (b) imposing civil penalties; and (c) public rulemaking.
10 CFR 2.103(b)(2) establishes the applicant's right to demand a review of a proposed license denial, and defines the applicant's appeal and hearing rights. NUREG-1021 1 of 6 Revision 8
ES-102 Subpart G, " Rules of General Applicability," govems all adjudications initiated by the issuance of an order to show cause, an order designating the time and place of a hearing requested by a person charged with a violation, and a notice of hearing. Subpart L, " Informal Hearing Procedures for Adjudications in Materials and Operator Licensing Proceedings," govems proceedings for the issuance, renewal, or licensee-initiated amendment of an operator or senior operator license.
- 2. 10 CFR Part 9. Public Records The regulations in 10 CFR Part 9 prescribe the rules goveming the NRC's public records that relate to any proceeding subject to 10 CFR Part 2. ,
Subparts A and B describe and implement the requirements for balancing the public's rights to information under the Freedom ofinformation Act and the NRC's responsibility to protect personal information under the Privacy Act.' Subparts C and D implement the provisions of the Sunshine Act, conceming the opening of Commission meetings to public observation. They also describe the procedures goveming the production of agency records, information, or testimony in response to subpoenas or demands of courts or otherjudicial authorities in State and : Federal proceedings. ,.3 , p+. u
- 3. ,10 CFR Part 20. Standards for Protentian Anninst Radiation The regulations in 10 CFR Part 20 establish standards for protection against radiation j hazards arising from licensed activities. Some of the material is appropriate for l inclusion'in the examinations administered to candidates for RO or SRO licenses.
.ji
- 4. 10 CFR Phrt 50. I leansirAnf irrwinetian and Utilitation F=eilitiae
.~, .- !
.. h ma ; .C ;
10 CFR'50.34(b)(8) requires that the final safety analysis report (FSAR) include a description of the operator requalification program. That description forms the basis for the inspection, audit, and approval of requalification programs.
' ' 10 CFR 50.54(i-1) requires facility licensees to implement an operator requalification program that meets the requirements of 10 CFR 55.59(c) within 3 months after receiving a facility operating license. Notwithstanding the provisions of 10 CFR 50.59, the j , : licensee mayhot decrease the scope of its approved requalification program without l authortzstion from the Commission. !
.. ~f l
' 10 CFR 50.54(k)- (m) contain regulations restricting control manipulations to licensed . j operators. These regulations are conditions of all facility licenses issued under 10 CFR }
Part 50. ! NUREG-1021. 2 of 6 Revision 8 ;
D j ES-102 l 10 CFR 50.74 requires facility licensees to notify the Commission within 30 days if there is a change in the status of a licensed RO or SRO.
- 5. 10 CFR Part 55. Ooerators' Licenset
-10 CFR Part 55 is the implementing regulation that establishes the requirements and the regulatory basis for licensing and requalifying ROs and SROs.
D. REGULATORY GUIDES i 1. Reaulatorv Guide 1.8. " Qualification and Trainina of Personnel for Nuetaar PowdE
. Plants." Rev. 2. Anril 1987 -
i L Section C.1 of this RG currently endorses, with exception, ANSI /ANS 3.1-1981, !
"American National Standard for Selection, Qualification, and Training of Personnel for l
Nuclear Power Plants"(effective March 31,1988). The NRC is currently reviewing, and j is expected to endorse, with exception, the 1993 version of ANSI /ANS 3.1. Section C.2 endorses, with exception, ANSI /ANS N18.1-1971, "American National Standard for Selection and Training of Nuclear Power Plant Personnel."
. 2. Reaulatorv Guide 1.33. "Onamv Assurance Prooram Raants,T,sats -Ooerations" Appendix A to this RG contains a list of typical proceb[res for pressurized water i reactors and boiling water reactors. .
l
~
- 3. Raoulatorv Guide 1.114:"Gnidance on 8taina an Ooerator at the Controls of a Nuclear Power Plant" This RG describes' a' method acceptable to the NRC staff for complying with the i
Commission's regulations in 10 CFR 50.54(k)-(m), which require the presence of an RO at the controls of a nuclear power unit and an SRO in the control room from which the nuclear power unit is being operated.
- K 3, t i
- 4. Reautatory Guide 1.134;aumdiea! Evain=+ inn of Licanand Personnel for Noelaar Pc:: - !
Plants." Rev. 2. April 1987 i This RG currently endorses ANSI /ANS 3.4-1983, " Medical Certification and Monitoring
, of Personnel Requiring Operator Licenses for Nuclear Power Plants." The RG is being revised to endorse the 1996 version of the standard.
5.5 ' R =N Guida 1.149. "Nnetaar Power Plant Simulation Facilities for Use in Ooerator .
~~ License N.TJasticas." Rev. 2. Anril 1996 This RG currently endorses, with exception, ANSI /ANS 3.5-1993, " Nuclear Power Plant l- Simulators for Use in Operator Training and Examination."
i l l NUREG-1021 3 of 6 Revision 8 l l
ES-102 E: ' NUREG REPORTS
- 1. NUREG-0660. Vol.1. "NRC Action Plan Develooed as a Result of the TMI-2 Accident."
May1980 item I.A.4.2 of this document describes the guidelines for long-term simulator upgrades.
- 2. NUREG-0737. " Clarification of TMI Action Plan Reauirements? November 1980 This document clarifies the following r.ction plan items which are intended to upgrade the training, licensing, education, and experience of operators on the basis of experience gained from the accident at ihree Mile Island, Unit 2: .
Item I.A.2.1, "Immediate Upgrading of RO and SRO Training and Qualifications" Item 1.A.2.3, " Administration of Training Programs" Item 1.A.3.1, " Revised Scope and Crit 5ria for Licensing Exams" ltem 11.B.4, " Training for lhlitigating' Core Damage"
- _.: w . . .a :
- 3. NUREG-0800. " Standard Review Plan for the Review of Safety Analysis Reoorts for Nurimar Power Plants. LWR Mitian." Juiv 1981 : ,
- .~
Section 13.2, " Reactor Operator Training," describes the training and licensing of operators and identifies information to be submitted by applicants for construction permits and operating licenses.
- 4. NUREG-1122. "Knowledae and Abilities Cataloa for Norlaar Power Plant Ooerators:
Pressurized Water Rame+ars." Rev.1. Auanet 1995
;; . . vfl This document provides the basis fsc developing content-valid licensing examinations for operators at pressurized water reactors (PWRs). It contains knowledge and ability y (K/A) statements that have been rated for their importance to ensuring that the plant is
" operated in a manner' consistent with the health and safety of plant personnel and the public. .
j
- 5. . NUREG-1123f"Knowledoe and Abilities Catalan for Nuelaar Power Plant Ooerators:
Boilina Wa+=r R== dea" Rev.1. Anauet 1995
. This document provides the basis for developing content-valid' licensing examinations
' ' for operators at boiling water reactors (BWRs). It contains K/A statements that have been rated for their importance to ensuring that the plant is operated in a manner consistent with the health and safety of plant personnel and the public.
NUREG-1021 4 of 6 Revision 8
t, ES-102 l 6. NUREG-1291. "BWR and PWR Off-Normal Event Descriotions." November 1987 i The reactor event descriptions in this document provide a reliable, performance-based L source of information that examiners may use to design simulator scenarios that will be a valid test of an applicant's ability to safely and competently perform alllicensed duties and responsibilities.'
- 7. NUREG-1560. " individual Plant Examination Proaram: Persoectives on Reactor Safetv and Plant Performance" l
This report provides perspectives gained by reviewing 75 individual plant examination l (IPE) submittals pertaining to 108 nuclear power plant units. Chapter 13, " Operational Perspectives," is of particular interest because it identifies a number of important human actions that should be considered for evaluation on BWR and PWR licensing and requalification examinations.
- 8. NUREG-1600. " General Statement of Poliev and Procedure for NRC Enforcement Actions" This report addresses the NRC's expectations regarding compliance with 10 CFR 55.49,
" Integrity of Examinations and Tests," and possible enforcement actions against parties subject to that regulation (i.e., Part 55 license holders _and applicants and Part 50 licensees).
- 9. NUREG/BR-0122. " Examiners' Handbook for Develonino Ooerator Licensina Written Examinations." Rev. 5. March 1990 : =-c i
This document, which presented a procedure for systematically constructing content- ! valid licensing examinations for nuclear power plant operators, has been incorporated I into the examination standards in NUREG-1021. Rev. 8. It may be used for historical perspective, but is no longer used for developing examinations. F. INDUSTRY STANDARDS i
- 1. ANSI /ANS 3.1. "A' merican National Standard for Selection. Qualification and Trainina of Personnel for NMaar Power Plants" l This standard provides criteria for selecting and training nuclear power plant employees performing a variety of functions at various levels of responsibility (e.g., managers, supervisors, operators, and technicians). RG 1.8, Revision 2, endorsed, with exceptions, the 1981 version of the standard; the 1987 version was never endorsed by
'the NRC; the 1993 version is currently under review and is expected to receive a qualified endorsement from the NRC.
1 NUREG-1021 5 of 6 Revision 8
ES-102
- 2. ANS 3.2 (ANSI N18.7-1976). " Administrative Controls and QA for the Ooerational Phase of Nuclear Power Plants" This standard provides guidance and recommendations for administrative rules of prac-tice and related subjects and for preparing procedures and audit programs. See RG 1.33.'
- 3. ANSI /ANS 3.41996. " Medical Certification and Monitorina of Personnel Reauirina Ooerator Licenses for Nuclear Power Plants" This standard is the basic document covering the general health and disqualifying conditions applicable to license applicants and licensed personnel. RG.1.134 currently endorses the 1983 version of the standard but is being revised to recognize the newer document.
- 4. ANSl/ANS 3.5-1993. "Nucimar Power Plant Simulators for Use in Ooerator Trainino" This standard establishes the minimum functio'nal requirements and capabilities for nuclear power plant simulators for use in operator training. Revision 2 of RG 1.149 endorses this standard, with exceptions.
~
, W A
- s4 v 1
't 4
w 1 1 l l NUREG-1021 6 of 6 Revision 8 l
I i ES-201 INITIAL OPERATOR LICENSING EXAMINATION PROCESS l 1
/A'. -PURPOSE _
This standard describes the activities that must be completed to prepare for initial operator
- licensing examinations (including written examinations and operating tests) at power reactor facilities. It incfudes instructions for scheduling and coordinating examination development, i assigning NRC examiners and facility personnel, maintaining examination security, and obtaining reference and examination materials from the facility licensee.
B.' BACKGROUND Title 10, Part 55, of the Code of FederalRegulations (10 CFR Part 55) requires 1 hat applicants for reactor operator (RO) and senior reactor operator (SRO) licenses pass a written . examination and an operating test. The regulation requiresfpower reactor facility licensees to: (1) prepare the site-specific written examinations and operating tests, (2) upon approval by an . authorized representative of the facility licensee, to submit the examinations and tests to the l NRC for review and approval, and (3) to proctor and gfade the writte6'ex^aminations. Idoreover, ; the regulation requires that the license examinations be developed and administered in accordance with 10 CFR 55.41 and 55.45 for Ros or 10 CFR 55.43 and 55.45 for SROs.- l Llc: ;; ;:-:nT. re ;2fm;; _. 2.; N"0 m _ ,_ _.. . . _ __. ar..s .g . _ . . . .g y,...- _ _ ....... _ ,_ __ _ __3 _h;=._.'.2di:i'!y pr:;: .d ::: (i.; llc n;;r.; e,;.m: .;'J;. ; m s..___.._._. _ m,__m_.___ m____.._..____m.__,___mm.,__,... )
......._____g ._g_ ___.....sgy......ir..
~
.. 7 llc ::::: fr.cr. wr ;...; ;nd :dm'" Jag ::::~m rat ;rs; f; "O r;;;;;; ;r.d ;ppr;;;'. _ ; N".O i:::en
_____._m_2.d <__m,__mm._______.,- f;;.:..i ::..:::: S
..;;; h;;; 2.; ;:,:pe-fx ^; r :;:c c. ::::m:r.;;;;n; 'h;'. pr;;;d; ir.
____g-._ ____.3 .s .ww.. _g._. s ..__.. ..g. . .m. v. ,i.r_m..<____im_<_m,__m__..______,m..__ _ ____ ... _.._. s..r. . . . . . i
.. ___.a_____i_____...sm.__m. . _i. __; ic_ ___..u_ _,
_ _ _ _ , - _ . . . . . _ . _ _ _ - . . '_...._..r.......,..,..-____..i.L_.
,_,._i _ . _ _ m _ im _ _
.~ , _ . . _ . . . , . . . . . ~ . . .
5 2.0 w;.;
, _m_m ed.T.:r22:.;d _
ln..._ m .__. ::::.d __i ,r.s w',". .. 0;r.;;;;n L.;;;.7 (O'.) % %. "Ohengs m_.m
- n 'he, _Operder
.w x u_,___:
..._____m_m,-,_._m.,~....,.r.,......~......,.
v_. ,~_
~_ _ x _ _
,_,..a.m_.
. , _ _ . _ _ _ _ _ _ _ . .._ ._......_..., ,..=r
....,. .... . r_--__. .s. _ ...... .
,m,n,..._____
e-em am) ;f: ;;.'.;.an 2.6 ; .:.^." ^;;':fii ;' ?; n=T.:.- ; ;;d th; ;O;;; ray ;' ^h;;r
~
d;;;;pT.; .;. .(..Q ;nf E;r< 1 ., <4
- m. ,
m., um Facility licensees .. r . r . are expected to develop and submit the examinations in accordance with the instructions contained herein. The NRC retains the authority to develop the' esaminations on a case-by-case basis if it loses confidence that a facility licensee will devulop" examinations upon which the NRC can base its licensing decisions. The NRC will also prepar; ::': ':I ;:aT. r.:^2:.; n ; rendem hn!. at least one examination per
- u. _ _
region per calendafyear to certify new examiners, as required, and to maintain examiner g wwmt.' lQ ,. a' 'g~
- The NRC'sj6'alis to provide each facility with an opportunity for at least one initial operator licensing examination per fiscal year. Additional examinations may be scheduled on a case-by-case be' sis if NRC resources are available. Examinations for fewer than three applicants should be scheduled only under extenuating circumstances such as a shortage of licensed ROs or SROs at the facility. If a facility licensee has fewer than three license applicants, the examinations may be delayed until more applicants are trained.
NUREG-1021 1 of 26 Revision 8
1 ES-201
- Other pre-examination activities, such as submitting and reviewing license applications and eligibility waivers and administering the generic fundamentals examination program, are addressed in ES-202, ES-204, and ES-205. Specific instructions for developing, administering, and grading the written examinations and operating tests are found in ES-401 through ES-403 - and ES-301 through ES-303, respectively. Post-examination administrative activities, including management review of the examination results and preparation of examination reports, are discussed in ES-501. Cross-references to each of these star.dards have been made where appropriate.
C. RESPONSIBILIT!ES
~
Facility licensees and NRC staff should use Form ESdO1-1, " Examination Prepa' ration - Checklist," to track the examination preparations. As noted on the form, the target due dates can be adjusted as necessary to accommodate a given situation. The NRC chief examiner will
- initial the items as they are completed and ensure that the original form is retained for the k
master examination file (refer to ES-501). ,
- 1. Facility Licensee i ^
a.
/ ,n i The facility licensee shelHiexpected to apprise its NRC regional office of changes in its examination requirements., 3.
,m u
";; S: ':j ::xxxE 'd 7:d:l.d in w.. .; t; the !?CO'; ;nr =l :di.. r.:;;. ;re;
';ter x"::fn; ;--_= z::f :p.;Lr l:sr.;'n; n;;d; ;r.d r.e;; j 5 7 R0 r;;;;n;.l 4
; ". s :1 5 e s s..lrz t::7. r;g. c;b;.r.O d.;r.;; ;;;n:'.;;r.;:j 're.T. ths; ; mod in 5 repr.n. l acility licensees are strongly encouraged to schedule their examinations with their NRC regional office before commencing an initial license training class.:' ' ',w
, .. m
< :2k If the NRC determines that's facility is unable to develop acceptable examinations, the examinations could be delayed significantly until sufficient NRC' resources can be scheduled to develop and conduct the examinations, or until the facility licensee can develop an acceptable examination.
gf
- b. In accordance with 10 CFR 55.4g, facility licensees and applicants shall not engage in any activity that compromises the integrity of any application, test, or examination required by 10 CFR Part 55. Attachment 1 of this ES summarizes several examination security and integrity considerations. NUREG-1600,
"; " General Statement of Policy and Procedures for NRC Enforcement Actions,"
, '- ^ Jaddresses possible. enforcement actions against parties subject to the requirements in the regulation (i.e., Part 55 license applicants and licensees and Part 50 licensees).
- c. All facility and contractor personnel involved with an examination are subject to NUREG-1021 2 of 26 Revision 8
y ES-201 1 I
~
the restrictions stated in Section D of this ES. Any questions regarding those l l restrictions should be resolved with the NRC chief examiner before granting an individual access to the licensing examination. The facility licensee shall designate a point of contact to work with the NRC chief examiner and assign additional personnel as required to ensure that the examinations are developed, reviewed, administered, and graded in accordance with the applicable examination standards. The facility licensee may use contractors or other outside assistance to develop the examinations, but the j licensee bears full responsibility for the product, including conformance with the examination criteria and maintenance of examination security and integrity.
. d. The facility contact shall submit the required reference materials,' examination l outlines, and examinations, as applicable, based on the level of facility I participation. Form ES-201-1 specifies target due dates for the various l materials; the actual dates may be adjusted with p' rior agreement from the NRC !
regional office. ' All examination-specific materials (i.e., the~ examination outlines and examinations) shall be controlled and protected as sensitive information (refer to Attachment 1). , i
~
- e. The examination outlines and the examinatiorirIshall be prepared in accordance j with the guidelines in ES-301, ES-401, and ES-701, as applicable. The !
proposed outlines and examinations shall cover all portions of the license i examination (written, dynamic simulator, and walk-through) at all license levels relevant to the applicants (RO, SRO, and limited SRO) to be tested. An ;#.;ted %C:i;;pren-O!.;; (i.e., A knowledgeable facility supervisor or manager h;;lng ;#.di t: ;;::% = b;h;lf ;f % 5::::j llxsn) ; hall independently review and approve ;ll :x:- la;On ;p;2; m;%2%. the examiriation outline (s) and the proposed examination (s) before they are
~
. submitted to the NRC regional office Sr m.is; ;nd ;.ppw;; per item (f) below.
Sup;r;;;e; 2.4 ;;;;;;;d h c=;; m:22:; but did not par %cm :n;;ca:Mna:
;;$;O; b- t ;;;na :;;::xat m;y me =;:nd appr;;;; t ;x;m;=On o
in condisting this review, the etdhoneed-facility aprenrOO;; eupervisor shall use Forms ES-201-2, " Examination Outline Quality Assurance Checklist"; ES-7301-3, " Operating Test Quality Assurance Checklist"; ES-301-4, " Simulator
~ Scenario Quality Assurance Checklist"; and ES-401-6, " Written Examination Quality Assurance Checklist."
- f. Pursuant to 10 CFR 55.40(a)(2), an authorized representative of the facility licensee shall approve the examination outline (s) and the proposed NUREG-1021 3 of 26 Revision 8
ES-201 examination (s) before they are submitted to the NRC regional office for review and approval. The outline (s) and examination (s) should be forwarded to the regional office with a cover letter signed by the facility representative.
- g. In its examination submittal to the NRC, the facility licensee shall indicate the source of each test item proposed for use on the written examination and the operating tests. The following information shall be included for each item:
State the source of each item (e.g., is the item taken directly, without changes, from the facility licensee's bank, another facility's bank, the
- NRC's bank, or an old NRC exam; is the item a modified version of a i bank item; or is the item new?). Items that the facility licensee has obtained from another bank and deposited in its own bank may be treated as " bank" items provided they have an equal chance of being selected for use on the examination. Items from another bank may be treated as new items if they have not been made available for review and study by the license applicants and there is no basis (e.g., historical precedent or reciprocal arrangements with the other facility licensee) for the applicants to predict their use on the examination.
x. 3, _5' For those items that are taken directly from the facility licensee's bank, state if and when the item was used on the last two NRC license examinations at the facility or to evaluate the applicants' performance during their current license training class. For those' Items that re derived by modifying existing bank items, note the changes that were made or submit a copy of the item from which it originated! ' -
.s . . M.,A
- h. : The facility licensee shall make its simulation facility available, as necessary, for NRC examiners to prepare for and administer the operating tests. The NRC will
. - take. reasonable efforts to minimize the impact on other training activities.
i~ . da Before d=Mg or administering an initial licensing examination, facility m licensees,are encouraged to review the simulator examination security considerations in Appendix D to NUREG-1021 for applicability to their facility. Because facility licensees are more familiar than the NRC examiners with the
+" unique capabilities, limitations, and vulnerabilities of their simulators, it is
' expected that the licensees will take responsibility for determining and f ^ > 773 implementing whatever measures might be'necessary to ensure the integrity of s "~p the operating tests.
.. ~
- i. The facility licensee shall meet with the NRC in the regional office or at the facility, as necessary and appropriate, to review the examinations and discuss potential changes.
NUREG-1021 4 of 26 Revision 8
I ES-201 If the examination was prepared by the NRC, the facility reviewers should make their comments and recommendations on a copy of the written examination (s) , and opebting test (s) provided to them by the NRC examiner. Simple editorial !
. changes that do not change the intent of the question require no justification, however, every substantive change (e.g., deleting a question, replacing a L distractor, or revising an answer) must be supported by approved facility j reference material before the NRC will change the examination. I i
If the facility licensee has significant disagre$ments with the content or dimculty . ) of the NRC-prepared examination or the changes that the NRC has directed the
~
facility licensee to make in its proposed examination, the facility licensee is - 1 encouraged to communicate those concems to the NRC. If the t.4RC chief examiner is not responsive to the stated concems, the facility licensee should I contact regional management, and, if necessary, the Chief of the Operator Licensing Branch, Division of Reactor Controis and Human Factors, Office of Nuclear Reactor Regulation, for resolution'. J. If the facility licensee developed the eiaminations, it will generally make any necessary changes as agreed upon.with the NRC; however, the NRC retains i final authority to approve the examinations.- t
.. e k.
i f .
- .- =.w In accordance with ES-202, the facility licensee ~shall submit the license
~
applications along with a letter requesting that licensing examinations be i administered. , ': / ' T g n ,. 1+ f
- 2. NRC Reaional Marinaament. Sunarvialan. and Deslanrrr
- , f l 3 .~
a." ' The regional. omee'shall schedule the NRC's initial operator licensing
. examinations and shall arrange for the development, administration, and grading of those examinations as discussed below. The regional office shall periodically review each facility. licensee's examination requirements and shall negotiate with
./the facility licensee's training representatives as necessary to schedule specific examinatioridates = i:t: n ;z::Te t: Pe dds :::txd en Pa adbr;l
.' eneminsbem sehedule- consistent with operational safety requirements and NRC resource spallability. Each regional office shall plan to prepare at least one complete examination per calendar year.
a
- b. Approximately six months before each anticipated examination date, the regional
.~ ! /* - office should contact the facility licensee and confirm the examination date(s)
" ~.'and the expected number of applicants to be examined. The regional office
. 'should use that information to estimate the required number of NRC examiners and to make preliminary work assignments.
- c. The regional office should contact the facility licensee by telephone at least four months before the scheduled examinations to reconfirm the expected number of NUREG-1021 5 of 26 Revision 8
ES-201 applicants and the examination dates, and to make other preliminary arrangements for developing the examinations. The person who contacts the facility licensee shall discuss the following exa'mination arrangements: the guidelines for ensuring examination integrity and security (refer to Attachment 1) the requirement for an authorized representative of the facility licensee to approve the examination outlines and examinations before they are submitted to the NRC for review - the need to have the examination outlines delivered to the NRC at least 60 days! before the scheduled ewommaten-review date the need to have the referer:,:e materials necessary for tije NRC to develop the examination (if applicable; refer to Attachment 2) delivered to the regional office at least 60, but preferably go, days before the scheduled eneminsterweview date - the guidelines for developing, administering, and grading the written examinations, as applicable (ES-401 ES-402, and ES 403, respectively) the need to have the simulator available and the guidelines for developing and administering the operating tests (ES-301 and ES-302, respectively) the need to have the examinabons and the supporting reference materials (refer to Attachment 2) delivered to the regional office at least 30 days before the scheduled eneminehen-review date . {j the requirements (refer to 10 CFR 55.'31) and guidelines (refer to ES-202) for submitting the license l applications
' The regional office may negotiate earlier due dates with the facility contact but should refraid from advancing'ttsdates if it is unlikely that the review will begin promptlyinfter the material arrives in the regional office. The region should also keep the fa'cilit9' contact informed of the dates by which the region expects to provide its com64ntiTegarding the licensee's submittals.
, , mmn .
y%mp
- d. The' regional office shall normally issue a letter confirming the vert >al
^
" ~ arrangements no later than 120 days before the examination begins. The letter should ts addsised to the person at the highest level of corporate management who is responsible for plant operations (e.g., Vice President of Nuclear Operations). 'At+=chment 3 is an example of such a letter; the exact wording may S be modified as necessary to reflect the situation.
y i e. Approximately four months before the scheduled examination, the regional office 7,'will assign the required number of examiners to develop, prepare for, and
. administer the examination as arranged with the facility licensee. The regional office will also designate a chief examiner to coordinate the examination project with the facility licensee and other examiners assigned to the examination.
When making assignments, the region should consider each examiner's certification status, other examination commitments, possible conflicts of interest . NUREG-1021 6 of 26 Revision 8
ES-201 (as discussed in Section D of this ES) and general availability. The region should try to assign a sufficient number of examiners so that no examiner will have to administer more than four operating tests per week.
- f. The regional office will evaluate each examination assignment to determine if some or all of the assigned examiners should niake a separate preparatory site visit. The purposes of such a visit may include providing examiner orientation, retrieving additional reference material, or reviewing and validating the examinations. When making a decision, the region should carefully weigh the costs and benefits associated with each additional trip to the facility. The region should also consider such factors as the experience of the assigr)ed examiners, the quality of the facility licensee's examinations (if applicable), the number of written examinations and operating tests to be validated, and the status of the simulation facility (e.g., is it new or recently upgraded?). In addition, the region should consider the alternative of reviewing the written examination (s) and operating test (s) with the facility licensee via telephone (if the examination quality is high) or in the regional office, as well as the attemative of validating the operating test (s) on-site at the beginning of the examination week.
' ^
. Y., ,
- g. Upon receiving the preliminary license applications'^;approximately 30 days before the examination date, the regional office shall review the applications in accordance with ES '202. In ad.dition, the regional office shall evaluate any waiver requests in accordance with ES-204 to determine if the applicants meet the eligibility criteria specified in 10 CFR 55.31.
n- , ,, g.
~
After reviewing and approving ths preliminary license applications and resolving all waiver requests, the region will prepare an examination assignment sheet (in the format of Attachment 4) as far in advance as possible, but at least two weeks before the scheduled examination date. The region will review and revise the assignment sheet as necessary after receiving and evaluating the final license
' applications. , '
% :y , . , .
The assignment sheet will identify the chief and other examiners by name and list the applicants by name, docket number, and type of examination (e.g., SRO j upgrade, RO' written only) to be administered. All applicants listed on the ' assignment sheet should be administered complete examinations (written and operating) as indicated under " Examination Type" unless waivers have been granted in accordance with ES-204. A copy of the assignment sheet will be
- distributed to all assigned examiners, the Operator Licensing Branch (OLB), and regional distribution.
- h. The responsible regional supervisor will review the examination outlines and the draft examinations and evaluate any recommended changes and corrections noted during the chief examiner's review. The supervisory review is not intended NUREG-1021 7 of 26 Revision 8
ES-201 to be another detailed review, but rather a check to ensure that all applicable administrative requirements have been implemented. If the outlines, examinations, and recommended changes are acceptable, the supervisor will authorize the chief examiner to resolve any noted deficiencies with the author or facility contact. If any of the facility-developed examination materials (written, walk-through, or simulator) require substantive changes and cannot be made to conform with the examination standards at least five working days before the scheduled examination date, regional management shall consult OLB and make a decision whether to proceed with the facility-developed examinations or develop the ) examinations in-house. If the r6gion does not have the resources to ensure that acceptable examinations are prepared by the scheduled administration date, regional management shall negotiate with the facility licensee to reschedule the examinations as necessary. It is generally easier to postpone the written examination and focus on the operating tests so that they can be administered on s:,nedule and without affecting other examinations'. ';;;m:n;On pic%;m; er; 1..r2d ;t tM ' 2 mlnd;, M ix.c.;On;idd M pedpened; ad:::nW: exem:n;On ;Mn;:: ^;;'" ad M me-f: it;; M ;;;m:n;On ;;;;k The region may delay the written examinations or the operating tests for up to 30 days without OLB approval. Howeve~r, the region"should keep OLB informed and consider notifying the facility licensee in writing if the delay is significant.
,q g-The responsible supervisor will also ensurdthat any significant deficiencies and problems are addressed in the examination report in accordance with ES-501.
- i. After the chief examiner has verified that the necessary changes and corrections
~ have been made,' the responsible supervisor will review and approve the examinations for~ administration. Before signing the applicable quality assurance :
form (i.e., Form ES-301-3 and/or Form ES-401-6), the supervisor must be satisfied that the examination is acceptable for administration. 2 J., - if thire is an indication that an examination may have been compromised, the responsible supervisor will take action as necessary to ensure and restore the A integrity and ' security of the examination process. ' Actions may include not giving the examination, making additional changes to the examination, voiding the results if the examination has already been given, and possibly imposing
~,~ ' enforcement action in accordance with NUREG-1600. The supervisor shall keep regional management and OLB informed of any concems regarding examination NO7
+ v , integrity.
- 3. - Assigned NRC Examiners
- a. When assigned to administer operating tests for the first time at a particular facility, the examiner should inform the chief examiner and the responsible
.NUREG-1021 8 of 26 Revision 8
ES-201 supervisor so that arrangements can be made to conduct an orientation trip to the facility as described in item C.2.f, if deemed appropriate.
- b. NRC examiners must monitor and ensure the integrity of the examination process, if they perceive that a compromise has occurred, they must immediately report it to the responsible regional supervisor so that the necessary actions can be taken to restore the integrity of the examination. Attachment i summarizes several examination security and integrity considerations.
- c. The assigned examiners shall review and inventory the reference materials received from the facility licensee in response to the 120-day corporate
' notification letter. The purpose of this review is to determine if the materials are complete and adequate to enable the regional office to review or develop the examinations, as applicable. If it is not, the reviewer (s) shall inform the chief ;
examiner and the responsible supervisor and request that the facility licensee l send any additional materials that might be required. If necessary, an examiner l may review and select additional reference materials during a site orientation trip ! (refer to item C.2.f).
. d. The chief examiner will work with the assigned examiners and the designated facility contact, as applicable,~ to ensure that the examination outlines and examinations are developed in accordance with the r Micable examination I standards. The chief examiner should adapt the lev. of oversight and coordination based upon the experience of the indivbuals who are preparing the examinations. Facility employees are generally less familiar with the examination' standards and will require more oversight to ensure that a quality examination is ready on time.' -
- e. The chief examiner.will review the examination outlines using Form ES-201-2, j
- Examination Outline _ Quality Assurance (QA) Checklist," as a guide. A thorough and timely review (i.e.,' within 5 working days) will minimize the potential for j
- significant problems with the examinations.
;. w. .
The chief examiner will note any necessary changes and forward the outlines to the responsible supervisor (or a designated attemate other than the chief . examiner) for review and comment before resolving any deficiencies with the author or facility contact. If the outlines are significantly deficient, refer to item m C.2.h for additional guidance.
.s e f.- The chief examiner will review the written examinations and operating tests for quality in accordance with the applicable QA checklists (refer to ES-301 and ES-401) forwarded with the examination. If the chief examiner wrote the operating tests, another NRC examiner shall perform the independent review. The regional office may conduct additional reviews at its discretion if resources permit.
NUREG-1021 9 of 26 Revision 8 l
1 4
~
ES-201 It is especially important that facility-developed examinations and tests be reviewed promptly because of the extra time that may be required if extensive
. changes are necessary. The QA reviews should be completed within two weeks after the examinations and tests are received from the author or facility contact.
The chief examiner will note any necessary changes and forward the examinations and tests to the responsibht superviser (or a designated attemate other than the chief examiner) for review and comment before reviewing the examinations with the author or facility contact There are no minimum or maximum limits on the number or scope of changes the NRC may direct the facility licensee to make to its proposed examinations, provided they are necessary to make the examinations conform with established a,cceptance criteria or to attain an appropriate level of examination difficulty. Chief examiners shall exercise their experience and judgement to ensure that the level of difficulty remains consistent with that expected on NRC-prepared examinations. If the examinations are significantly deficient, refer to item C,2.h for additional guidance. '
- g. Upon supervisory approval, generally about two weeks before the examinations are scheduled to be given, the chief examiner will review the written examinations and operating tests with the facility licensee. lf 2.; ;;;;m ..;;;; .;
;;;. ; devi ;d ? by i.e N".0, 0.; ;v:;;; .. . . - . .. . -. . . .- ..... .
m.: .: c The chief examiner may conduct the examination review via telephone, in the l I regional office, or at the facility, as appropriate to the circumstances, depending on the extent of the changes,*and as approved by the responsible regional supervisor (refer to item C.2.f). -
, y .
- a. .,
if the' examination was prepared by the NRC, the regional office will provide a copy of the writteWe~xaminsbon(s) and operating test (s) to the facility reviewers
. only after'tliey sign the" security agreement (Form ES-201-3). The facility reviewers'sholM make their comments directly on the examinabon, retum the marked-upicopy to the NRC chief examiner, and ensure that he or she understandiitheir comments and recommendations. The facility reviewers may retain a copy of the marked-up examination, subject to the physical security
< guidelines in Attachment 1.
4 if the facility reviewers have significant disagreements with the chief examiner, ME the chief examiner will inform the responsible regional supervisor so that the x p disagreements can be resolved before the examinations are administered. _:u
- h. After the examination corrections have been made, the chief examiner shall verify that the changes are appropriate and route the examinations and the mark-up drafts to the responsible supervisor for final approval.
i NUREG-1021 10 of 26 Revision 8 5
r . ES-201 1. As soon as possible after the responsible supervisor has approved the operating l- tests for administration, the chief examiner shall distribute copies of the scenarios, job performance measures (JPMs), and questions to the other assigned examiners so that they can familiarize themselves with those materials and be better prepared to probe the applicants' deficiencies if required.
- j. The chief examiner should work with the designated facility contact to schedule l
, the operating tests to optimize efficiency and the mix of RO and SRO applicants in the crews assembled for the simulator examinations. The number of applicants on a crew shall not exceed the number of assigned examiners (i.e.,
one-on-one evaluations are mandatory), except as noted below. However, if the facility licensee's technical specifications routinely require more than two ROs to be stationed in the control room, OLB may authorize the use of additional
' s.urrogates. Only one individual (applicant or surrogate) is allowed to fill a shift supervisor or manager position during the simulator operating test.
If a three-person operating crew consists entirely of SRO-upgrade applicants (who do not have to be evaluated.on the control boards), the region may assign only two examiners to observe the crew. Although the applicants in the RO and
~
balance of plant positions may not be' individually evaluated, they will be graded and held accountable for any errors that occur as'a' result of their action (s) or C '~ inaction (s). - Normally, for purposes of test integration and continuity, the same examiner
,, should administer all three operating test categories to an applicant. However, under certain circumstances, the operating test (excluding the dynamic simulator scenarios) may'be divided among different examiners._ Such division is appropriate if a facility licensee's simulator is not located near the plant, because of limitatioris in examiner resources or scheduling, or if a facility licensee requests examinsbons for an unusually large group of applicants. Refer to ES-302 for specrfic instructions regarding administration of the operating tests.
4 ;p
' Operating tests will normally be administered on regular work days. If weekend
, or shift work is required to administer the operating tests, the chief examiner will 5
coordinate the arrangements with the assigned examiners and the facility licensee.} As a general rule,' the operating tests should be scheduled after the written
* ~ examinations; however, other sequences are permissible if agreed to by the m facility licensee. Normally, the written examinations should be administered no I more than one week before the operating tests. However, under extenuating circumstances and with prior approval from OLB, the written examinations may be given as soon as the license applications are accepted, any applicable waiver requests are resolved, and the examinations are approved. Of n;sn;ri, OLS
.T.;y i; ;;'t.;- e P.; .d';n s;-2;Sa i; b; il:y;d gas l ;t:'t.;
NUREG-1021 11 of 26 Revision 8 l I
'I
ES-201 1
;p; ret lr.;; t;;tr. er; eerr,iit;.
If, as an efficiency measure, the facility licensee prepared the written examination in conjunction with another facility, then the two examinations must be administered at the same time. If the examination schedule has to be changed on short notice, the chief examiner will work with his or her supervisor and the designated facility contact to reschedule the examinations to a time when examiners are available and other examinations c.7 tt.; .;t r.;l :n..T.:r.;^ r. ;;t.;ik are not affected.
- k. If the facility licensee will administer the written examinations,' the chief examiner shall review the ES-402 requirements (e.g., proctoring and responding to applicant questions) and confirm the applicant's status on the assignment sheet (i.e., examination type and waivers) with the facility contact before the examinations are given.
D. PERSONNEL RESTRlCTIONS - it is impossible to define criteria that anticipate every possible conflictEf-interest issue. Supervisors must apply sound judgment to the facts of each case.llf any doubt exists regarding a particular case, the supervisor should consult with regional. management and/or OLB to resolve the issue. < , 1
~
'l}
- 1. NRC Examiners .
r
, ic a. ., *
- s. . The regionaioffide shall nobassig'n arIexaminer who falied an applicant on an
' . operating test to administer any part of that applicant's retake operating test.
a pc w y~ .
. b. _ If.an examiner was previously employed by a facility licensee (or one of its contractors) and was significantly involved in training the current license Lapplicants, the regional office will not assign that examiner any direct
, responsit2 ties for developing or administering written examinations or operating f
tests at that facilitj. Regional management will control other in-office examination activities concoming the facility, such as technical consultation and quality assurance reviews of examinations.
- c. If an examiner is assigned to an examination that might appear to present a conflict of interest, the examiner shall inform his or her immediate supervisor of the potential conflict. Such notifications should include the following information:
.~. .
* ' ~ , .
the nature and extent of previous personal and professional relationships with the applicants anything that could affect the administration, performance, evaluation, or - NUREG-1021 12 of 26 Revision 8
ES-201 results of the examination anything that could create the appearance of a conflict of interest
- 2. Facility Personnel
- a. . Facility employees or contractors who had any, direct involvement in training the lb;n;; applicants during their site-specific license training program shall not prepare the outline (s) for the operating test (s) but may prepare the outline (s) for the written examination (s) c 't; Opiath; 's.% using a systematic process, as described in ES-401.
Only en; ps;m ;;h; F;;;ded mue 't; . 40 i ' h;; ; cf ;ceSubd .';i.nc.am
; nd ; m f :':e b ;t ; ;t b n e n d n ; = ; ;; t F;;:f:I 5 pseent ; mc; cf th;
;fcdubd c'::::;;m betiu '!:n = 20 ps.;;; a mem ;f 't; %%' ;f.eduled c'::::eem ad ;;mubig b;t.;;tbn m.y F.tc';i b d;;;bpb; 'h; J.dt;n s;mb;'_ba qu;;tbn;.' I; thumac, p; 'j;l- ; :nt .-;y a;l d;;-bp ;;dtg.
_..__.,__,_.m...._,.._r_,._._m_...2. m_......... .. ., . - l There are no restrictions on test item developers based upon their time spent instructing the license applicants. However, in the case of the written j examination and prescripted walk-thiough ciuistio~ns, personnel who were j directly involved in training the license applicants may develop new questions, I select questions from the test banks, and modify test bank questions for only
~
)
those topic areas in~which they}did not provide instruction. Instructors (and i supervisors) may review questions on all topics, as necessary, in order to verify
~
their technical arid psychometricivalidity. Moreover, instructors may develop and bank questions in'their area ~of expertise provided those questions have no greater chahce'of sppearing on the examination than other bank questions. yw ; Simulater instructors may prepare the implementing documentation for the
- dynamic ' simulator and job performance measure (JPM) portions of the operating
' test withouGegard to their instructional specialty, once the outlines (i.e., the 4'
~
simulator event sequences and walk-through task lists) are defined. As a means of NRC verificaten, the facility licensee may be requested to identify
^
(1) which test items were developed and reviewed (including the effects of such ; reviews) by individual examination developers and (2) which content areas were l taught by the examination developers.
-:my I
~
7 ff the facility licensee considers these restrictions too burdensome, it shall define and discuss with the NRC chief examiner the process it proposes to ensure that exam integrity is not compromised. The NRC chief examiner must approve the facility proposal before it is implemented, and the regional office shall inform OLB of any such situations before commencing examination development. NUREG-1021 13 of 26 Revision 8 l i
ES-201
'b. The facility licensee shall minimize the number of personnel who have detailed
- knowledge of the NRC licensing examination.
- c. All personnel who will receive detailed knowledge of any portion of the NRC licensing examination, including the examination outline, must acknowledge their responsibilities by reading and signing Form ES-201-3, " Examination Security Agreement," before they obtain detailed knowledge and again after the t examinations are complete. Examples of prohibited activities include the following: -
f the design and administration of any classroom and simulator instruction (including scheduled sessions, individual coaching, and remedial training) specifically for the license applicants (Simulator booth operation is
; acceptable if the individual does not ' select the training content'or provide direct or indirect feedback. Continued pa'rticipation in requalification training for groups including SRO upgrade applicants is also acceptable,
~ l as long as it is documented on Form ES-201-3 and is limited to areas in which the instructor has no examination knowledge.)
all on-the-job training,' practice, coaching, and sign-offs the preparation review, grading, and evaluation of periodic quizzes, examinations, and simulator exercises, including the audit examination l
. ; ,- i Supervisors and nianagers having knowledge of the examination content may continue their general oversight of the training program for the license applicants, )
including the ' review of examinatiorMuizzes, and remedial training programs. However,~ thossisbpervisors'a'nd managers may not provide feedback regarding thi~ content of those examinations, quizzes, or programs.
- d. The facility. licensee will provide' a copy of Form ES-201-3 (listing the expected
, signatories) to_the NRC chief examiner at the time the examination
.- ' arrangements are confirmed. The facility licensee shall also inform the chief examiner if additional personnel need to be added. - The chief examiner shall review the list'of personnel, consider the nature of each individual's association with the hcense applicants, and evaluate the pros (e.g., increased technical
^ ,
accurgand validity) and cons (e.g., increased security risks) associated with each proposal. The regional office (i.e., the chief examiner in consultation with the responsible supervisor) may deny the facility licensee's proposal to use
, ;JNcertain individuals (e.g., the applicants' supervisors or coworkers ) to develop
, f -and review the examinsbons, but should discuss such decisions with OLB before informing the facility hcensee.
The original security agreement forms must be submitted to the regional office for retention after the examinations are complete. NUREG-1021 14 of 26 Revision 8 d
ES-201 1 E. i ATTACHMENTS / FORMS j l
- Attachment 1. - " Examination Security and Integrity Considerations" l Attachment 2, " Reference Material Guidelines for initial Licensing Examinations"
. Attachment 3, " Sample Corporate Notification Letter" l Attachment 4, " Sample Examination Assignment Sheet" f.".;2.a,;nt 5, ".Ouid;: ne; ~_,,, . .~ ..., . ....e::ve c' Mr0-0;;;bpd lnPb' Lben; n;;
.m......
..__.__.~__..
Form ES-201-1, " Examination Preparation Checklist" Form ES-201-2, " Examination Outline Quality Assurance Checklist" ~ Form ES-201-3, " Examination Security Agreement". ; i W
~
l NUREG-1021 15 of 26 Revision 8
ES-201 Examination Security and Attachment 1 Integrity Considerations NRC and facility licensee personnel must be attentive to examination security measures to ensure compliance with 10 CFR 55.49. At the time the examination arrangements are confirmed, an NRC examiner shall review with the facility licensee the following guidelines covering physical security and limitations on the use of examination banks as well as the facility licensee's specific plans for ensuring examination security. Additional restrictions covering the assignment of personnel are addressed in Section D of ES-201 and shall also be reviewed. Physical Secunty Expectations i
- 1. Th; =0 sp- ct; th;; th; f;-:"Ny ::ssa .;;:: :::::i: th; srs phy;:s: usay pre-::$-:s ^;;;th th; :n:::;: s;T.:. :S:s a :t -f::: ';;;th h; r;;sl:'at:s s;..T.:..;;;s;.
The NRC expects that personnel will be made aware of the facility licensee's physical security measures and requirements, sign the NRC's examination security agreement, and understand their security responsibilities, including the limits on their interaction with the license applicants, before they are given knowledge or custody of any examination materials. .
- 2. All examination-specific materials (i.e.,-the exarninstion outlines and final examinations) shall be positively and continuously controlled and protected as sensitive information (i.e., under lock-and-key or in the. custody of someone who has signed the security agreement). Drafts, copies, and waste materials must also be controlled and disposed of properly.
The NRC expects that the examinations will NOT tm developed and stored on a computer network to which the license applicants could gain access.
- 3. The examination outlines, written examinations, and operating tests that are mailed to
' the NRC regional office shall be placed in a double envelope. The inner envelope shall be conspicuously marked "FOR OFFICIAL USE ONLY" and "TO BE OPENED BY ADDRESSEE ONLY." Furthermore, the cover letter forwarding the examination matenals shall direct that the materials be withheld from public disclosure until after the examinations are~ complete.-
4J
~
- The examination outlines and examinations shall not be transmitted via non-secure electronic means (e.g.~ the intemet); they may be transmitted via the NRC's " AUTOS" j
local area twtwork in the resident inspectors' office.
- 4. '
The facility licensee shall immediately report to the NRC chief examiner any indications
- or suggestions that examination security may have been compromised. The NRC will
~ evaluate such situations on a case-by-case basis and determine the appropriate course of action.
NUREG-1021 16 of 26 Revision 8 e
l ES-201 2 Attachment 1 !
- 5. The faci'ity licensee and the NRC should determine if examination security problems were noted in the past and ensure that corrective actions have b3en taken to preclude 3
recurrence.
- 6. The facility licensee and the chief examiner will review the simulator security considerations in Appendix D to ensure that the instructor station features, programmers' tools, and extemal interconnections do not compromise examination '
integrity. The primary objective is to ensure that the exam material cannot be read or l recorded at other unsecured consoles, and that examination materials are either . , physically secured or electronically protected when not in use by individuals listed on the l
- security agreement. .
Examination Bank Limitations .1. The facility licensee and chief examiner shall ensure that written examinations and l operating tests conform with the guidelines in ES-301 and ES-401 regarding the use of items taken directly from the bank, modified items, and new items.
- 2. If the facility licensee has an open bank, it will not place any new or modified items to be used on the examination (written questions, job performance measures, or simulator i
scenarios) in its examination bank until after the last examination has been j administered. ' Other Considerations ~
= \
~
- 1. The NRC will consider an examination to be potentially compromised and a possible violation of 10 CFR 55.49 to have occurred if a situation exists or existed for an applicant to gain an u'nauthorized advantage as a result of the facility licensee's failure
~
to exercise ~ continuous posithiscontrol over the integrity of the examination. If the NRC determines ttist's facility lice'nsee has manipulated the scope, content, or level of difficulty 'of an' examination t6 enhance the chances that its applicantr, would pass the examination, the NRC will utilize its enforcement authority including, as warranted, civil i penalties, orderfagainst the individuals involved, and, charging the individuals involved with deliberate misc 6nduct pursuant to 10 CFR 50.5.
- 2. The license applicants should not be able to predict or narrow the possible scope or f content of the licensing examination based on the facility licensee's examination practices (other than those authorized by this NUREG or in writing by the NRC).
- 3. ~ Facility licensees are responsible for the integrity, security, and quality of examinations prepared for them by contractor personnel.
NUREG-1021 17 of 26 Revision 8
. ~
ES-201 Reference Material Guidelines Attachment 2 For Initial Licensing Examinations This attachment discusses the reference materials that facility licensees are expected to provide for each NRC initial licensing examination. The regional office will customize the list of . reference materials as required to support the specific examination assignment; additional materials may be requested at a later time if necessary to ensure the accuracy and validity of the examinations, in determining the need for reference materials, the regional 6ffice will consider the facility - .. licensee's level of participation in the examination development process. If the facility licensee will be preparing the examinations, it may be sufficient to obtain only those references necessary to review and validate the items that appear on the examination, plus a set of key procedures and other documents required to prepare for the operating tests. The regional office will duly consider the administrative burden it places on facility licensees and request only those materials that are actually necessary for the NRC examiners to prepare for the examinations. All reference materials provided for the license examinations should be approved, final issues and should be so marked. If any of the materialis expected to change before the scheduled examination date, the facility licensee should reach agreement with the NRC chief examiner regarding changes before the examinations are administered; _
.- . w The reference materials may be submitted on computer diskettes (in a format compatible with the NRC's word processing software),' as hard copy, or a combination as arranged with the i NRC chief examiner. If the facility licensee prepares the examinations, the hard-copy {
references should normally be limited to those materials required to validate the selected test 1 items. ~ All procedures and reference materials should be bound with appropriate indices or tables of contents so that they can be used efficiently; a master table of contents should be
~
provided for all materials sent.? Failure to provide complete, properly bound, and indexed . reference material may prompt the NRC to retum the material to the person at the highest level
~
of corporate management responsible for plant operations. The retumed reference materials will be accompanied by a cover letter explaining the deficiencies in the material and the basis for postponing or cancelling the examinations. 4- N-1 4 Unless otherwise instructed bylthe NRC regional office, the facility licensee is expected to provide the following reference materials for each NRC initial licensing examination:
- 1) Materials used by the facility licensee to ensure operator competency
.3.
a.,
~ licensing,
].The following as necessary to types supportof materials examination used to train applicants for development:
leaming objectives, student handouts, and lesson plans system descriptions of all operationally relevant flow paths, components, controls, and instrumentation NUREG-1021 18 of 26 Revision 8 i j i a
ES-201 - 2 Attachment 2 material used to clarify and strengthen understanding of normal, abnormal, and emergency operating procedures l complete, operationally useful descriptions of all safety system . ! interactions and, where available, balance-of-plant system interactions 1 L under emergency and abnormal conditions, including consequences of l anticipated operator errors, maintenance errors, and equipment failures,
' as well as plant-specific risk insights based on a probabilistic risk analysis (PRA) and individual plant examination (IPE)
These materials should be complete, comprehensive, and of sufficient detail to' I l support the development of accurate and valid examinations without being redundant.
- b. Questions and answers specific to the facility training program that may be used in the written examinations or operating tests I c. Copies of facility-generated simulator scenarios that expose 'the applicants to abnormal and emergency conditions, including degraded pressure control, degraded heat removal capability, and containment challenges, during all modes of operation, including low-power conditions (A description of the scenarios used for the training class may also be provided.)
- d. All JPMs used to ascertain the competence of the operators in performing tasks within the control room complex and outside the control room (i.e., local l
operations) as identifiexf in the facility'sjob task analysis (JTA) (JPMs should evaluate operator responsibilities during normal, abnormal, and emergency conditioris and events, and during all modes of operation including cold shutdown, low power, and full power.)
'2. Complete index of procedures (including all categories sent) zu - -
- 3. All administrative procedures applicable to reactor operation or safety
- 4. All integrated plant procedures (normal or general operating procedures)
- 5. All emergency procedures (emergency instructions, abnormal or special procedures)
- 6. _ Standing orders (important orders that are safety-related and I.:ay modify the regular procedures)
- 7. " Surveillance procedures that are run frequently (i.e., weekly) or that can be run on the simulator
- 8. Fuel handling and core loading procedures (if SRO applicants will be examined)
NUREG-1021 19 of 26 Revision 8
- ES-201 3 Attachment 2
- 9. All annunciator and alarm procedures
- 10. Radiation protection manual (radiation control manual or procedures)
- 11. Emergency plan implementing procedures-
.12.- Technical Specifications (and interpretations, if available) for all units for which licenses are sought
- 13. System operating procedures
- 14. Technical data book and plant curve information used by operators as well as the facility precautions, limitations, and set points document
- 15. The following information pertaining to the simulation facility: .
- a. list of allinitial conditions '
- b. list of all malfunctions with identification numbers and cause and effect information, including a concise description of the expected result or range of results that will occur upon initiation and an indication of which annunciators will be actuated as a result of the malfunction I
- c. a description of the" simulator's_ failure capabilities for valves, breakers, indicators, I and alarms -
- d. - lthe range of severity of each variable malfunction (e.g., the size of a reactor coolant or steam leak, or the rate of a component failure such as a feed pump, turbine generator, or major valve)
~
- d. ' a list of modeling condhtions (e.g., simplifications, assumptions, and limits) and problems that may affect the examination
- f. a list of any known performance test discrepancies not yet corrected
- g. a list of differences between the simulator and the reference plant's control room w
.h. simulateinstructor's manual
- 16. Any additional p ant-specific material that has been requested by the NRC examiners to i develop examin:rtions that meet the guidelines of these standards and the regulations NUREG-1021 20 of 26 Revision 8
i ES-201 Sample Corporate Attachment 3 { Notification Letter i i Idatfd (Name. Title) (Name of facilitv) (Address) (City. State. 7in cade) Dear (Namet in a telephone conversation on (data) between MrJMs. (Name. Title) and MrJMs. (Name. Title). arrangements were made for the administration of licensing examinations at (facihty name) during the week (s) ofIdain) [As agreed during the telephone conversation, your staffj((The NRC)) will prepare the examinations based on the guidelines in Revision 8 of NUREG-1021, " Operator Licensir.g Examination Standards for Power Reactors."[ The NRC regional office will discuss with your
. staff any changes that might be necessary before the examinations are administered.)(( fn '
- dens with th; ;;.;;d;::as in f.".;2..T.;at 5 ;' E0 201, Your staff will be given the opportunity to review the examinations during the week ofIdata).1] >
N. m; To meet the above schedule, it will be 'necessary for your staff to fumish the [ examination outlines by { data). The written examinations, operating tests, and the supporting] reference materials identified in Attachment 2 of ES-201 [will be due] by Idata) [ Pursuant to 10 CFR 55.40(a)(2), an authorized representative of the facility licensee must approve the outlines, examinations, and tests before they are submitted to the NRC for review and approval.] Any delay in receiving the required [ examination and] reference materials, or the submittal of inadequate or incomplete materials, may cause the examinations to be rescheduled. In order to conduct the requested written examinations and operating tests, it will be necessary for your staff to provide adequate space and accommodations in accordance with ES-402, and to make the simulation facility available on the dates noted above. In accordance with ES-302, your staff should retain the original simulator performance data (e.g., system pressures, temperatures, and levels) generated during the dynamic operating tests until the examination results are final. > Appendix E of NUREG-1021 contains a number of NRC policies and guidelines that will be in effect while the wntten examinations and operating tests are being administered. To permit' timely NRC review and evaluation, your staff should submit preliminary reactor operator and senior reactor operator license applications (Office of Management and Budget
'(OMB) approval number 3150-0090), medical certifications (OMB approval number 3150-0024),
and waiver requests (if any)(OMB approval number 3150-0090) at least 30 days before the first examination date. If the applications are not received at least 30 days before the examination NUREG-1021 21 of 26 Revision 8 4
r; .)
. ES-201 2- Attachment 3 date, a postponement may be necessary. Signed applications certifying that all training has
' been completed should be submitted at least 14 days before the first examination date.
This letter contains information collections that are subject to the Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.). These information collections were approved by the Office of Management and Budget, approval number 3150 0101, which expires on April 30,2000. fdthtb The public reporting burden for this collection of information~is estimated to average fnumbeM [500] ((50)) hours per response, including the time for reviewing instructions, gathering and maintaining the data needed, [ writing the examinations, jand completing and reviewing the collection ofinformation. Send comments on any aspect of this collection ofinformation, including suggestions for reducing the burden, to the information and Records Management Branch (T-6 F33), U.S. Nuclear Regulatory Commission, Washington, DC 205554001, or by Intemet electronic mail at BJS1@NRC. GOV; and to the Desk Officer, Office of Information and Regulatory Affairs, NEOB-10202, (3150-0101), Office of Management end Budget, Washington, DC 20503. - The NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information unless it displays a currently valid OMB control number." Thank you for your cooperation in this matter. (Name) has been advised of the policies and guidelines referenced in this letter. If you have any questions regarding the NRC's examination
- procedures and guidelines, please contact (name of regional contact) at (teleohone number). or (name of renoonsible reoional annarvisori at (telephone number)
+ -.
c , , Sincerely, x ;. ._.
.r.: <
.- ~
(Anorooriate reoional representative. a,, , g 2c: Docket No.f 50-(Number)
, ,+ A Distribution:
Public -e
. NRC Document Control System Regional Distribution
[) ' ' l Include ok for examinations to be prepared by the facility licensee. (()) = include only for examinations to be prepared by the NRC .
- ay
,o, J
NUREG-1021 22 of 26 Revision 8
r: <
.y i
. ES-201 . Sample Examination Attachment 4 4 Assignment Sheet MEMORANDUM TO: (List NRC examiners by name)
FROM: 3- (Reaional Suoervisor's Name. Title)
SUBJECT:
EXAMINATION ASSIGNMENTS APPLICANT- DOCKET NO. EXAMINATION TYPE 1 9;- ,
*i m !
Facility and location - Facility contact . NRC chief examiner. Written examinations to be prepared by (RO) ve, -
. (SRO)
Dates of Examinshons* ' ' G' i 3
;j .n !
;I y r s l
;( NRC Supervisor
$ s. . we - 6 #- ,
cc- Residentinspector ! n.. Project manager. (Standard regional distribution) OLB NUREG-1021 23 of 26 Revision 8
.i -
E* O SM4 #"t . . : J 1 : _ _ _ im. F_:t.A.. f'4 _ _ _ m ..:_. AAA__L _ m_a f" h_hr 5.w a wr ts s%sta s u s sw w sws s sirwris ty a s he s w w sta va f ute%su su s sws st t# mt L IM #% Mm.l__m) t _ *A!m t t I m ___!__ we s v u wr c r w w w w wp rw w s s senrumi s_rwws swa u uM s. f".n. usasusuussvasw
.__?_ma!_ -
4 hm - -- - _ . .! . . . m t L 1 Pi p's J_..t___J!_iA!tf___t__ m__t__A!m_. _. . , L. _ _ _ _f . . A . J mA AL. se e a rg, yi un s g, w rw ww wa y vs sw wg, , wswyge w si au ssus aswg,s seus sg benus a uss senswa se a a suay why wvu sww%,5wbs u n ti rw _Am f_AL- L t Fi f% ___?___f _ Xf* _ _ m. _t ! _ _ . . . . J l _ S _ ma! _ _ r"t 85 _fALf_ _A__J-J TL. wesw wi sus tirw s i n g %sr sw yrwssui vsusv%,, mise wiewkswws,bs su s whawnivi s v.w.g vs ta rve wnumu s% sus %s. E u rii, f*m___?__ m..__!__Atm_ L _ tf L. _m..?_...J .fm__ _ _ . ., m J L , . A L - ____m__!LG- __ _f__t l rywwu swas rg wnums s sus suursrvs se we supuis ws, s e, y sw ww w ts uns sus p y yswwvgs wy tusw snu,w y wssenwsw amagswsaus l m.,_m-. !- m _ L f___ AL_.. _ Em__ _ ll . . -____..!.....J..!AL AL_ f. "f : A . . t! _ m-__m wwpris u v swws ww tw3 W Eu rgey usui te s wa s s etsuusy prs tr u te v rte ww wts ww stu s Er ste Ingerssty suwws swww. M "PL . f _ _
- I! A. , _m..!m...m_ - .Itt k m m!..__ ___-_m Am AL _ mum I_ m AI m _ _mam_I_l_ mfa m.AL_.. _f_ m
- 6. .
s s sg, s uswissty # %, w sw wwtas w ww ass ww ga w ges s tuwwwww tw Es sw genui s sis sutuvu s s a susstw s stusw ump stes tu ng,y ways s u
**_.._:A., ________A st m_ r et cru es wwwtsi sty urgi wws t rws us gs wu s s a t_gue a=w s wf.
e if iL . _m.. _ . . A 1. . _m AL _ ___ J_.. AL _ Lirir% _ L f _ f _ . . - _ ? _ _ _ L _ . .t _t L _! _ f A L _ w, sa as sw a me w sw vv tuurwww r u svs up tu surs a vu rw way , as it, a is gv we erws wnuus aus age s era swwsgs ws age s as age m____*Lt ..._--.m__AL- _A_A... mf AL . _m..
. . . if . --__m af_L____- . J u to WyVs styrwww urbsprwu T swvs vu a tu tw eptumstes vu na sw s na w aw ww g s . hr . , www p wu bas sums agiurur ums s%s
.__m- 1. . J f _ m .im _ \ mA AL _ __J ml _ mmL J-.. M- ?--mf _---_--- A T_ - . .- - mA J A m tes ss wurwu v ww swwtstreirf ust tu sgr w s r%s vs wuswu a weuy . a swgews suiza s s aus suygr u a Taw s et sw waprgewtwts tw
__ _ ti m_ AL _ _m..t_.._______Im_ ____Lt_ t _ t ? _ _ i? - _ _ _ft mJ__.._A_ f _ m ? ci. . _ .. f _ _ _ _ _ _ u u svs artws te sw s w w rw ww pri www weir tvi ywwwswww as abetwe tzwu sw wa su sumwge hghsmu tw s uwsisty a wswu ws swuia
- A _ _* 9 m- ? _ _ . .Xf
- m_A m.. _ IIA. . _ _ _ . - - _ _ - L..AL_ L t F% f%
s y sustw u sums wt as surtsr u gwwws in qwu==s eg userwtss sul Iwne wy la sw u is sw.' J *t"L _ f_ _?t!A.. _m.? m ..__ _ mL _ . .tJ -mt- AL _ ? _ . m----J_A*___ __f m--_m_1_ m_ AL - - - _ . . T. sasw surbrsusty a w w uw wwws air urs swessge s u swriw Es rwst s wwwt s su s swu stuungswa sw uurs sts wwu s su u stas she hrs a 55 rg, wwyy ofAL. ..._* AA _ _ m. _ _!__A?__#mi m_J ___.i m_ t. . . # \ ___. . J _ J A
- AL - _ L. AL- L i t"t #"t m.
wp ta sw ww s ratwa a %enusu i sri suunsvi ugurf ums sw ups sw vv ws sung guyf prs w w agswes tw Es swu s s wy ts una u vs the ge r L'9!_ _Im _J *.A _ _? _ t mL_____ AL _ A J. -A mL__ . AL. t_A__A ml AL m ...__Afm_ __-..!__ __ wrss # uprvin wwstwa sumu wu seus agwup tu surg gsw s vwn ws egass agge tu rw su stws rt we tu rw qwwwtswa e a wqtsus g, ssw
...AT__A*m . L.....__ m..m_. ..L_A _A _L_ -_ /m - J_t* _ m ..-_ Aim- __ I-*-_ m f tswns g ewusstuwa se a sw ww w w ws g me w ws y urwwwtuuss stu r w %rs sumu rgw gge.g., trwsw ass yg aus ytswwtreurs s, a grpruuswiu rg um l
_t ? A _Am_ __.t_*__ ___...__% _.._A km ...-- - A-8 L, _- - _ _ . . _ J f-
- m. m_ ? t ?A. . __f__---_ 1 wuures urways , vi s w w surin rg umi 5 usu swww we f II stuurt ww urwprprvu twts wy asprpts w w was a upws u s ty a w s w s up u twg,
- _A__f_t if AL m _"_?_m6 _E- ____ _ A _ _* _ f _ . _ L _ iAA t Jm__ _mA ...__m a AL _ mLm __ A L _.
I s uns sw a Iuuss e e s 4s Wins ws rgus sums u wIws wu sw%s s a uustwu rups Eirtswu 5 Urttual WW%f up 5svtartsgegevet ts sw hPu suim3 Ugw g ts sw ___ _._m_A _ J A m -__..?J_ AL _ __m_____. J _. ___A_A?__ L_f___ AL. L t P4 f"4 . .f t t uns w wnywwtwns tw pri w w tww as rius s sw _-,. _ _ __ s y , _ -. s swu stammtws s wgue s ws w tu ug, u vs sw ww ses _L___- AL_ ..__?_mA?__ tav susi rg%s as r%> wituus a sus sustrwa a e =rt . __m__A*__ A__i_ #m?_..i_i________*__ m_ _s i m L __ r_ ___m _.m_m..___g __.L_
- w. uasw wpew s umtra vg twerter gures a u _ _- upwws sus rway sus sts fvw paws a wi s s sums sws, e s swuswnsi g,w/ s a rey wge
___..f m..J __a..ml.._A_J !_ AL_ _ AaE* _ _ _ _ , , m t! J _ A _ _s A A L . f_ _ ? tiA , . tg AL _ _m_____*Lt. pri ws w w rw ww wws wu sgs w w umswusstwuss su g gs sg, w e e sww ws v unautsuntwas es t ts tw Iuurbrius by as as sty a trerprws perswrge _m_*-_m9 .._-_!-__ J_A- f_ AL_A . _ IA _ .? _ ?A 1. __m____m_. Rf AL - _ -__A! _ A A. ___ s wyswi rums U4 g 45vaurws surw tw a r r sys tw er tu vent um wu nge w uwst sur a rvtrwerwuss y . si Es sga wprgs u us tsu rg twwntir wa ge _mA .._tIJ_A_ 9 J. ._f _ _ AL m ____m..?_.., m_ 12 AL . .._f___ _ ? _ _ f f* _ _ _ A m tA _ _ _ A ? m_ _ m ___.,[A mf sswt v .. - tww wtsusesg asrge prs ws w w sw vv g ws su na swy ws swwu gw epigu rasswuss at suurtws untuvu a umgir us a werwet ws AL _ ___e?_?_m ..._I?J_A*__ AL _ . . _ L _ . .t J _ _ _ *. . 2" ._. s . . _ t ? J _ A ? mAAL_ L__?__?__ ml AL _ tu sw grs *wuss a sus pusu y w n-_ _. _ _ . rg tiswy urusWWTtsawgrwuvto pIs suus v umgsweis tuwu a 4N6 ts rise erwgis us ses ug hrs ha Ewa
..t. #- # - Am F" f*n 83 M89 % . g ww w w ag gewswa tw t-tur wwe.f.
f_4 A. k RP% #4 *
*tt L. _ _ _ ?f _ L t - AL _ m..L_..A AL m .?_...m_ 9 , _ t? I 1? . -______ Am W. E is s s is igur wrtums y uns swa wy sus wee up y __ _ _ - -
nr ss wwys swtu 6 as r%e a w w swww urs rgs v musugsuutews a priwwwww tw
.a . _ . . _ AL _ ___?__i?__ ___1__r_a_ -_JAL_ _m..!...__t m___ _ _ _ -rt , f _ _ ? n u. . __ ? _ . . . _ __
te s - _-- g J tu rw _- '_ _~- a sus sam asws a u a _ _ . . _ _ _ up s sgs ta sw a te w sur wwwe up wwurnrwsasw. p3sw e uuswaisty a te y sw ww wu eir m L _ . .t .,,3 - A m._ AL _ _ _ _f . J .m ..__?_mA?__ __A_m*_t_ A. AL_ k ir% pt _L*_f _. _ _?___ __3 urs uwtests s w twv 5 s as yta a s seus nww espr wituus u uvu suunsws u 5 e sumetwa ruussw tw 65 str 5 w u 4 w bru urge B tensius a sus Ne uns s%s
- _ _ . - AL_A La --L- .._J-_A-J_ AL m ?_ ______A_ m_J _____,,._m__f A ! _ _ L f. _ _ AL__
we swws De su sunt asw ws urs tw est
- swws urtums rwsup nu swus w%ru a as a ateu untir aus uns a wwwu u si a sws swustsws sur wir u vi %s ha sgeg
__-ms . . J _ ik _ ______._..___A?_- [f AL _ f__?nu.__.?...___ L... _?__w___A mm_____
. wws swiwsste bs r%e pri wu w w swww u y swwtse eg. r s tu rie s uimwsuuty u me w sw ww we up i sus w ge arrgs gru rvuus v4 wwu shr%ws a per
- _ _ _ _A ? _ _ AL - m. _.__I__A?_
tlk . .- _BM - . ta . mf AL , 5 wguus wur sg nu sin gg _ _ _.y ws gsus s uwemyty vs tu ug wrmuuss a usu uurtswi s a u uustwa ammier, at stir as stess # gatyprws siiruwisuty tw _ _A__* Im !A !_ AL !_ _ _____*L?!la..A_ f __-.. .. m.mL - - _ _ _ - - Am h tP% pt _________A gm. ___mi. A? _ wwssvwy eeeI wtrs strespu a sur tw e iu %%# 5 e runs sungws a swa ut swa s weirwswnsvs s. s 87 arL m f _ ? f *A.. m_.?_.__ _ m.. A?_ m _ _ _ _ . ml AL _ _..__._?__L?__ _ mA-_*_tm f. . ._* AA _ _*_.I_A__ 5e e s 5%e sembresity p w w swvv we ur a s susy 5 w numus s um %rwpry vs as ag, tenuss s sus sustswu 5 5 e u uu tw a sus u ns gww 5 entwa sg durus s summuustwa g
- m. ..._ta_AL _..mLg m_ __JT_J J , _* _ _ A L _ .,m.*_ .. _______
v1 wv muun tu gu wargs If wW s s svtuus swas bstsu su rg 45 rw a w w sw wy gr3 vgrwersy. . _.A ;t?______ _L_, _ _ _ _ . . .1A L Am _ m ._ , _ _ t ___.._.A __..,____A_ _,:__..___J :_ gr. .r c..w __, s aim um a s rt, we resus w ,s u sys,. ., s t. , tu ~
,,s s, w.t,wl .,, t, u s s 6,. . s ,wuswsa ~ s,6. ws.Wt... .s n s
, a AA _ _ L _ _ _ i t- en
, ut,um.s r, s ~ a t a, At m ur -,.r ,%n,
.) as.
NUREG-1021 24 of 26 Revision 8 l l l
.i )
i. ES-201 Examination Preparation Checklist Form ES-201-1
, Facility: Date of Examination:
' Examinations Developed by: Facility / NRC (circle one)-
Target Chief Date* Task Description / Reference Examiner's Initials
-180 1. Examination administration date confirmed (C.1.a; C.2.a & b)
-120' 2. NRC examiners and facility contact assigned (C.1.c; C.2.e)
-120 3. ' Facility contact bnefed on security & other requirements (C.2.c) -
4
'120
- 4. Corporate notification letter sent (C.2.d)
[-90R] [5. Reference material due (C.1.d; C.3.c)] . , 40R 6. Integrated examination outline (s) due (C.1.d & e; C.3.'d) _
-55R 7. Examinaten outline (s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e)
~
? "
1
-30 8. Preliminary license applicatens due (C.1.k; C.2.g; ES-202)
L
-30R 9. Draft examinatens,' supporting documentaten,' and reference materials due (C.1.d, e, f, & g; C.3.d) ,
-14 '10. Final license applications due 'and $ssigriment sheet prepared
i (C.1.k; C.2.g; ES-202)
~'
-14 11. Examination approved by NRC supervisor for facility review
. (C.2.h; C.3.f)
-14 12. Examinations reviewed with facility (C.t.l; C.2.f & h; C.3.g)
-7J 13. Written examinations and operating tests approved by !
j NRC supervisor (C.2.i; C.3.h)
' 7- -
- 14. Final appicatons reviewed; assignnent sheet updated; waiver l
l letters sent(C.2.g)
- 15. Proctonng/ written exam administraton guidelines reviewed with I
,-7c ' facility licensee and authonzaten granted to give written exams l (if applicable)(C.3.k)
- 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.1)
Target dates are keyed to the exam date except where noted with an "R" for review date. All dates are for planning purposes and may be adjusted on a case-by-case basis. []: , Applies only to examinations prepared by the NRC. NUREG-1021 25 of 26 - Revision 8
ES-201 Examination Outline Form ES-201-2 Quality Assurance Checklist
]
I Facihty: Date of Examination: Initials item Task Desenption a b c
- 1. a. Verify that the outline (s) fit (s) the appropriate model per ES-401.
- b. Assess whether the outhne was systemmetcaNy prepared and whether all ew knowledge and j g
few ability categones are appropriately sampled.
- c. Assess whether the outhne over emphasizes any systems, evolutions, or generic topics.
E N d. Assess whether the repetiten from previous examination outunes is excesasve
- 2. a. Using Form ES-3015, verify that the proposed scenario sets cover the' required number of normal evolutens. instrument and component failures, and major trenesents.
S I b. Assess whether there are enough scenario sets (and spores) to test the projected number and M mix of applicants in accordance with the expected crew composition and rotation schedule without compromesing exam integrity; ensure each apphcant can t,e tested using at least one new or significantly modined scenario and scenarios will not be repeated over successive days.
- c. To the extent possible, assess whether the outhne(s) conform (s) with the quantative and cuantitative criteris specined on Form ES 301-4 and described in Appendix D.
- 3. a. Verify that the outline (s) contain(s) the required number of control room and in plant tasks and verify that no more than 30% of the test matenalis repeated from the last IWtC exarmneten ]
W
/
- b. Verify that the tasks are distributed among the safety functon groupings as speclRed in ES-301; '
T one task shaR require a low-power or shutdown condition, one or two shall require the applicant to implement an allemate path procedure, and one should require entry to the RCA.
- c. Verify that the required administrative topics are covered, with emphess on pefformance based activides
- d. Detemune if there are enough different outhnes'to test the protected number and mix of applicomts and ensure that no mesw4 hen 4pbef4he items are duphcoted on successive days.
- 4. a. Assoas whether pien64pecfBc priodNes (including PRA and IPE insights) are covered in the appropriate exam section G ._ ..
E b. Assoas whether the 10 CFR 55.41/43 and 55.45 sampling is appropnote N E c. Ensure that K/A importerme rebnes (except for plant-specific pnorthes) are at least 2.5.
- d. Check for duplication and 'esertap among exam sections L
- e. Check the entre exam for belance of coverage
- f. Assess whether1he exam fits the appropriate job level (RO or SRO).
- -E Pnnled Name / Signature Date
- a. Author . .s -
- b. Faculty Revismorf)
- c. Chief Examiner
- d. NRC Supennect
(*) Not applicable for NRC<leveloped examinations. NUREG-1021 - 26 of 26 Revision 8
3- f s o i E 8 f n ole si. J 1 0
~
o . a T n 2- snt e n C A o aekdo b p D i s S e c si" M R. i t np i v E bai bh d ._- oa e od t no . w,,y sa es t h R m r n emc . nmo
)
2 o ef er o e;-, o ( h aamh acst hN vef ic e F t E hnnf wn mxe ,
. R o ao noI e mf k U f T ohmha ee e.e b ao x nM. A
)
(s sf wromt r ml a s ev eoh J N G
~
k ne e opxvaieh on a gt ne i. I S e s r eet aic y i l sp^ . wedg pi hya l nm e o e vmt it m c c'f, hyo t nr sdcy N b . r apnnai o r eaf t r Ceh . E f o oise ct c u Rt d T A d t
,l NN /.
e s eeet rhe s D t l nt sdr e n f" hu u oa enon t t gn dheslat uhueio n em f.
., hcivden rmt a e
- smeeht rt s m me ee l. C .
)
sa nx nterum osf a m 1 a cr ng
.R yN
( oeiuI gx E ha ecu t
.a e o cy a Ge R metsumCna U t
- a. r t h d nih r t
. mhs n aRoh i t ou h c .b
_y T A e Nt cs t e e^ _d
. at i N ; x u yb ean eon ms ,e G nt io ht gbetot ws[ r I S
4y ki h n adn yes r i s nie bmg e n oel deg
- i nt yt o Mtt o v
t eht u et u nnc u ic a adr os n r ai t a 6 u c l mlpet r ehon r rf sd*ly 2 _ n rer f = e Cotche o oe l o Rfner sts un ns st ha ca S 6 n Ni9t e yo n aa o r ndi eeaic f 2 o nt elyrh pI eel h at cl oa h / dt p . t adec e Y rhcaes a tu e g noupi pian cn d T m ol f t t n s I ho amtpea a r L m bumac v epn e sy i t I B . a aia n I x dI s o t udr oe S E yt o sia t aef rcx N ghtahsar - Wl an e nhee n ut p O nt ,t i yme s P wdn: .. pmi
~
w o o a .cx ea m n odn S _ xe a E ari o - kn knt sr ;e c xe o Fvioa t R -
,hefe t
dt et .rpin / eod n e E izn aH u: a oc l l n niof h t t i l g u v ro m
,a x L
T - ee I i cwI .t r nC i d t T _ eI pt r. cn snR ag B . h sa en . inn un i t . o l O _ n f ais dthi: m _a mE e d=h doe vn J _ ee r ex a- rwt r d) ,e c i n ca o i ur (stcl i qgeJ caf o nn t I ek ue r aI e .tsie get s e .h i t dese E . q ve nl m t r o ewit nh M . o ar c/ .iums _aeg e i l wet A - g a huC- t aR; xr a t a n ohod n t ne r N n I nN ;eyn i k gd e D i m t ag f a a m ni t E a yidis E-a x hie t ey g sh 1 omm t nt y oni t E-x f r muI ,in d nm T N 1 dmb ema y =ieit l t odod I R 2 - e ,l c s o t eb a P 0 P r efod pef sr eer tare 1 . e meo P l w . _ 1 oter- rt bt ss G - 0 nar t o gt c aro eiie nnw E 2-S kdoht c a et u l t si nip hii t odd h mmo 0. 1 2 3 4. . R U ha uuhe E . . . . . . . 5 - Taaw 1 .2 . 3 4 5 6 7 8. N 1 l t t r 2 9 1 1 1 1 1 1
ES-202 PREPARING AND REVIEWING OPERATOR LICENSING APPLICATIONS A PURPOSE' This standard provides instructions for facility licensees and applicants to prepare and the NRC , to review initial licensing applications. It also discusses the experience, training, education, and I
. certification requirements and guidelines that an applicant should satisfy before being allowed
' to take an NRC reactor operator (RO), senior reactor operator (SRO), or limited senior reactor operator (LSRO) licensing examination. ;
B.' BACKGROUND The Commission-approved licensed operator training programs at most power reactor facilities'. ,
. are based on a systems approach to training (SAT) and use simulation facilities that have been . ,
either certified by the facility licensee or determined to be acceptable by the Commission under 10 CFR 55.45(b).' in accordance with 10 CFR 55.31(a)(4), these facilities are not required to L include details of the applicant's qualifications, experience, and training on the NRC license application form. In lieu of these details, the Commission will generally accept certification by an authorized represer)tative of the facility licensee that the applicant has successfully
. completed the facility's Commission-approved traming program. a if the facility licensee does not have a SAT-based licerised operator training program that uses !
a simulation facility acceptable to the Commission, the NRC will not accept the license I application unless it includes the details of the. applicant's qualifications and training. Detailed i license eligibility requirements and guidelines are derived from 10 CFR Part 55, Subpart D, l
., " Applications," and Regulatory Guide (RG) 1.8, Revision 2,'" Qualification and Training of ;
Personnel for Nuclear Power Plants," respectively. .With respect to license applicants, RG 1.8 i
- s. . endorses, with exceptions,' the guidance in American National Standards Institute / American !
Nuclear Society (ANSI /ANS) 3.1-1981, " Selection, Qualification and Training of Personnel for i Nuclear Power Plants." NRC examiners should refer to those documents as necessary when . ; 4-evaluating the' eligibility.of applicants at facilities that do not use an NRC-approved or facility- I I certified simulator as part of a SATy licensed operator training program.
~
C. ~ RESPONSIBILITIES: n . , - ,
.- e u .
The regulatory requirements associated with the license application process.are detailed in Subpart D, " Applications," of 10 CFR Part 55.' The medical requirements for license applicants and licensed operators" appear in Subpart C, " Medical Requirements," of 10 CFR Part 55. These requirements should be referred to as necessary when preparing and reviewing license _ applications p. L1. . . Applicaht/ Facility Licensee a; To apply for an RO or SRO license, an applicant must submit'an NRC Form 398,
' " Personal Qualifications Statement - Licensee," and an NRC Form 396,
" Certification of Medical Examination by Facility Licensee." (Computer-NUREG 1021- 1 of 12 Revision 8
E ES-202
. generated duplicates are acceptable.) The application is not complete until both forms are filled out, signed by the appropriate personnel, and received by the NRC. Detailed instructions for completing NRC Form 398 are provided with the form. Additionalinstructions regarding waivers of training, experience, and examination requirements are provided in ES-204.
If the applicant is reapplying after a license denial,10 CFR 55.35 applies, and
, the applicant must complete and submit a new Forms 398 end-896. The applicant may file the second application two mor;ths after the date of the first final denial, a third application six months after the date of the second final denial, and successive applications two years after the date of each subsequent denial. Each new Form 398 shall describe the extent of the applicant's additional training since the denial and shall include a certification by the facility licensee
~
' that the applicant is ready for reexamination.
if the applicant previously _ passed either the written examination or the operating test, he or she may request a waiver of that portion of the licensing examination. Such waivers are limited to the first reapplication and must be requested within one year of the date of the failed examination. Refer to ES-204 for a more detailed discussion of this and other waiver criteria. , The medical data in suppor of NRC orm 96 ' re a normally good for six months from the date of the medical examination. If, because of a delay in administering , the examinations, mone than 6 months have passed since the date of an RO or ' SRO instant appli6 ant's medical examination, the facility licensee shall certify in writing that the applicant has~ii6t develciied f any physical or mental condition that i would be reportable under 10 CFR 55.25. If it is likely that the time since any applicant's last'm'edical exa'miristion will exceed 24 months before the licensing action is coni'pleted,'the applicant shall be reexamined by a physician and the facility licensee ~shall[risep the applicant's medical fitness on NRC Form 396. M an applicaht is reapplying after withdrawing a previous application or accepting's final license denial, he or she may request a waiver of.a medical reexamination by checking item 4.f.4 on NRC Form 398. The time since the last medical exarnination can not exceed 24 months and the applicant must certify
;;ter prefi.; ;r. ;4:;..;;% , in item 17, " Comments," of the form that he or she has not developed any physical or mental condition that would be reportable under 10'CFR 55.25 2 . 5 6 : ::;nt L;Zr ;;T. Os ;pp:L O ..
. b. Each applicant (except those applying for an LSRO license) must satisfactorily i s c : complete the NRC's generic fundamentals examination (GFE) section of the !
written operator licensing examination for the applicable vendor. Refer to ES-205 for more information on the GFE program. LApplicants do not need to take the GFE if they were pmviously issued an RO or NUREG-1021 2 of 12 Revision 8 i
ES-202
' SRO license or an instructor certificate based on a site-specific written examination (on the same type of facility) that was administered between February 1982 and November 1989 and included the material covered by the GFE. Enter the date of the examination in item 4.g on NRC Form 398 and an I explanation in item 17; a waiver is not required.
- c. As noted in ES-201, the facility licensee should submit preliminary, uncertified l license applications and medical certifications for review by the NRC regional office at least 30 days before the examination date. This will permit the NRC to make preliminary eligibility determinations, process the medical certifications, evaluate any waivers that might be appropriate, and obtain additional information, if necessary, while allowing the facility licensee to firyish training the applicants before the certified applications are due.
d.' The facility licensee's senior management representative on site must certify when an applicant has completed all of the facility licensee's requirements for the desired license level (i.e., experience, control manipulations, training, and medical). Such certification involves piscing a check in item 19.b of NRC Form 398, signing the form, and submitting it to the NRC regional office at least 14 days before the examination date. 7 , The facility must also sr5it a wntten request that the written examination and operating test be admin; tad to the applicant ~ m
, a
- e. When the NRC regional othee denies a license application, the applicant may not accept the proposed denial.iln such instances, the applicant may request that the Director, Division of Reactor Controls and Human Factors (DRCH), Office of
. Nuclear Reactor Regulation (NRR), review the application denial or request a J hearing in accordance with 10 CFR 2.103(b)(2). Further action will be taken in accordance with ES-502. .
-.: +n'
; 2. NRC Regidnal Office ,
V5 , . .
-, a. ~
The NRC regional office shall review the preliminary applications as soon as possible after.they are received, in that way, the regional office can process the medical certifications, evaluate and resolve any waiver requests in accordance
~
- with ES-204, and obtain from the facility licensee any additional information that
. - might be necessary in order to support the final eligibility determinations.
J .
' ~ With regard to the medical certifications, the regional office shall forward the applicant's NRC Form 396 and the supporting medical evidence to the NRC physician at the Headquarters Health Unit or the regional contract physician for evaluation any time the examining physician has recommended that the applicant be issued a restricted license' or that an existing restriction be changed (by checking block A.4 or A.5 on Form 396).
~ NUREG-1021 3 of 12 Revision 8 7
.I Y
.p 3
ES-202 The NRC will not process a retake application if the applicant's request for reconsideration or a hearing on the previous license denial is still outstanding (refer to ES-502). Before entering the applicants' data in the operator licensing tracking system (OLTS), the region shall verify that none of the applicants' names appear on the
" Restricted Individuals List" found on the NRC's intamal web site at http //www.nrc. gov /OE/ rpr/ia.ii.urir th; .";;; .g '."fi ;pp'%;;br.; ;f th; NRCe4eeel-eree netweek. The region shall check with the appropriate contact in the Office of Enforcement by telephone or electronic mail'to verify that the information on the subject individual is current before using the information on the list to deny a licensing action. ,
- b. The regional office will verify that the applicant has successfully passed the GFE -
if required, and review the data on NRC Form 398 to ensure that it is complete. Affirmative responses to items 12.a and 12.b of NRC Form 398, indicate that the applicant has successfully completed a Commission-approved, SAT-based training program that uses a simulation facility acceptable to the Commission under 10 CFR 55.45(b). If the facility licensee checks "yes"in response to these items, the licensee need not complete items'13,L" Training," 14, " Experience," and 15, " Experience Details," of NRC Form 398. .The region may accept the application without further review unless there is reason to request further information concoming the applicant's' qualifications. j
~
.. Occasionally,'a lity licensee completes items 13,14, and 15 even though
- they are not required as explained above. In such instances, the region may
^
rroview the informahon provided against the eligibility guidelines in Section D for c the requested license level and resolve any deviations with the facility licensee. New apphcations'm'ust still' include the number of significant control manipulabons in item:13.3; at least five are required on the facility for which the license is sought. This requirement can only be waived or deferred under the
- conditions specl5ed in 10 CFR 55.31(a)(5); situations other than those specified
. would require a6 exemption in accordance with 10 CFR 55.11. For ROs applying for an SRO license, certification that the operator has successfully
" operated the controls of the facility as a licensed operator shall be accepted as evidencii of having completed the required manipulations. '
"If an applicant checks "no" in response to items 12.a and 12.b on Form 398, the j [rkgion shall review the application against the specific RO, SRO, or LSRO eligibility guidelines described in Section D.
If the applicant is reapplying after a previous examination failure and license denial, the regicn shall evaluate the applicant's additional training to determine if NUREG-1021 4 of 12 Revision 8
ES-202 the facility licensee made a reasonable effort to remediato the deficiencies that caused the applicant to fail the previous examination.
- c. .
The region may determine that the preliminary application is incomplete, that more information is necessary to make a waiver determination, or that the applicant does not meet the requirements in 10 CFR 55.31. In such instances, ; the region will note the deficiencies and request that the facility licensee supply 1 additional information when it submits the final, certified license application (or soonerif possible). - 1 Conversely, the region may determine that the preliminary application is complete, and the applicant meets the eligibility requirements orjs expected to ! meet the requirements pending the receipt of additional information. in such
~
instances, the region shall enter the applicant's name, docket number.,and examination requirements on the examination assignment sheet in accordance with ES-201.
- d. Upon receiving the final, certified license application, the reviewer shall evaluate any new information to ensure that the eligibility criteria are satisfed. If so, the reviewer shall check the " meets requiremants" block at the bottom of Form 398, l sign and date the form. If necessary,'the reviewer shall add the applicant's l
name and other data to the examination assignment sheet in accordance with l ES-201. The reviewer shall also ensure that the assignment sheet accurately - reflects any examination waivers that may have been granted in accordance with ES-204.
- nq T If the region determines that the applicant still does not meet the eligibility ;
. requirements, the regional licensing authority will discuss its decision with the i Operator Licensing Branch (OLB) and notify the applicant in writing that the y application is being denied and identify the deficiencies on which the denial is based (Attachment 1).EThe responsible regional supervisor, or designoe, shall
. check the "does not meet requirements" block at the bottom of Form 398, and ;
~shall sign and date the form. The applicant's name shall be stricken from the l
, examination assignment sheet; the applicant shall not be permitted to take the m "
' licensing examination until the region determines that he or she meets the eligibility criteria.
2 With prik,7 approval from OLB, as necessary, in accordance with ES-204, the region may administer a license examination to an applicant who has not ,
... ,, satisfied the applicable training or experience requirements at the time of the !
3 examination, but is expected to complete them shortly thereafter. Assuming that
%. # . 37 the applicant passes the examination, the region shall not issue the applicant's license until the facility licensee certifies that all of the requirements have been completed.' . (Refer to ES-501.)
. D. LICENSE ELIGIBILITY GUIDELINES NUREG-1021 5 of 12 Revision 8
ES-202 Threugh lt; One! sf;ty en;lyd; rep;;t (IO ^R), ishn ;d ;p;;l0;;^ enr. 70), er qu;:ny ,
- urenz (O ^) pregrem, ;;;h f;;:lMy llx=x he; semmhted i; ; ped 5; gulddla;; for nudar p;;;;r pl;nt w; 'mr qudl0stles (e.g., ,^.N0l 10.1-1071 er ,^ N0ll^JJO 0.1-1001). '.h;n 1
rs ;;;lng th; eppl sat;' ;l glbll y, NRO =;m:ncr;. mu;t eneum that th; pap;r fed::ty llcc=;; j ammnment l; c::d. Th; NCO agl;nd ;"l2 ;heu'd af; ;ll qust es ag;rdlng ll=n;; j
;llglbll y t; th; Oh ef, OLO, f;r indut en. '
Revision 2 of Regulatory Guide (RG) 1.8, " Qualification and Training of Personnel for Nuclear Power Plants," describes a method acceptable to the NRC staff for complying with the Commission's regulations with regard to the training and qualifications of nuclear power plant personnel. For the positions of shift supervisor, senior operator, and licensed operator, this RG generally endorses the guidelines contained in ANSI /ANS-3.1-1981; specific clarifications, additions, and exceptions are noted in Section C, " Regulatory Position," of RG 1.8. The license l eligibility guidelines in RG 1.8, Revision 2, and ANSl/ANS-3.1-1981 are summarized below.
"s; gddd:nx m;y n;; b; eppll=t'; ;; d: fed:My llxn:::: end ;hd net b; ;;n;;r.md ;; ;;s.la;;;.-:;.
The NRC regional office should refer all questions regarding license eligibility to the Chief, OLB, for resolution.
- 1. Reactor Operator '
'.t .
I
- a. Experience m _
3 .. ; (1) The applicant should have a minimum of three years of power plant ' experience, at least one of which is spent at the nuclear power plant for which the license is sought (preferably in the performance of nonlicensed
, operator duties). -
(2) f ,. The applicant should spend at least six months performing plant !
. operational duties as a nonlicensed operator at the nuclear power plant '
. a u forwhich the license is sought.
. , < - .c , .
b.: Training - t-(1) The' applicant should complete at least 13 weeks as an extra person on shift in training for the RO position. This training should include all phases of day-to-day operations and be conducted under the supervision A oflicensed personnel.
=.,.J v.(2) The applicant should be trained in nuclear power plant fundamentals and
~:a# plant systems, use of those systems to control or mitigate an accident f
during which the core is severely damaged, and operating practice. (3) The applicant should complete at least 500 hours of lectures on the principles of reactor operation, design features and general operating j 1 NUREG-1021 6 of 12 Revision 8 l I
ES-202 characteristics of the nuclear power plant involved, instrumentation and control (IC) systems, safety and emergency systems, standard and emergency operating procedures, and radiation control and safety procedures. (4) The applicant should satisfactorily complete an NRC-approved training program involving at least one week at a nuclear power plant simulator. The simulator training center should certify the applicant's ability during a reactor startup to manipulate the controls, keep the reactor under control, predict instrument responses, use instrumentation, follow procedures, and explain annunciator alarms that occur during operation. (5)- The applicant must manipulate the controls of the reactor"(not simulator) during five significant changes in reactivity or power level (refer to 10 CFR 55.31(a)(5)). Every effort should be made to diversify the reactivity. and power changes for each applicant.' Startups, shutdowns, large load changes, and changes in rod programming are some examples; these changes could be accomplistied manually using such systems as rod control, chemical shim control, and recirculation flow control.
- c. Education The applicant should have a high' school diplo'ma or equivalent.
?..
,, 2. Senior Reactor Ooerator ~
~
. a.- ' Experience l~
,w-(1), The applicant should have a minimum of four years of responsible power i [jplant exponence,' as defined in RG 1.8. At least two of those four years j
' , cshould be nuclear power plant experience. i "N "
' ,P 7(2) "7 ?.Q' applicant should have actively performed licensed RO duties The
. . 'g. least one year at the facility for.which the SRO license is sought. The
~-
NRC'may accept any one or more of the following education or
, experience qualifications to satisfy this requirement provided that the applicant supplies sufficient details in the license application for the staff ' h. jto make a judgement regarding equivalence:
{ jQ ln ;['/ : . A four-year degree in engineering or the equivalent (e.g., a degree in engineering technology or the physical sciences that
' "'a.7 includes course work in physics, mathematics, or engineering; a professional engineer's (PE) license obtained by passing the PE examination).
LNUREG-1021 7 of 12 Revision 8 l l
ES-202 At least one year as an active licensed RO at a comparable facility (same vendor, similar vintage) or 18 months as an RO at a L noncomparable commercial power reactor. At least two years in a position equivalent to a licensed RO at a militarv = actor. Expenenc,, obtained in licensed positions (or their equivalent) on other large-scale reactors will be evaluated on'a case-by-case . basis. Applicants must also submit a waiver request in accordance with ES-204 if they want this experience to apply. toward the requirement. . (3) ' At least six months of the applicant's nuclear power plant experience should be at the site for which the license is sought.
.(4) During the two years of nuclear power plant experience, the applicant should participate in reactor operator activities at power levels greater than 20 percent for at least six weeks.
~ '
- b. Training (1) The applicant should complete at least 13 weeks as an extra person on shift in training for the SRO position. This training should include all phases of day-to-day operations and be conducted under the supervision of licensed personnel. Any portion of the 13 weeks that is spent at or above 20 percent power may also be used to satisfy the experience guideline in Section D.2.a(4).
,y
. (2)' ' .lf the applicant has not held an RO license at the facility and one of the
> _ . qualifications specified in Section D.2.a(2) is substituted for that exponence, the training guidelines of Sections D.1.b(4) and D.1.b(5) should be met. The applicant should satisfactorily complete a training
- program that is comprehensive in its coverage of both RO and SRO k&;M ;+, skills, and abilities and must take an SRO-instant license
,; examination.
U *
g .(3) applicant should be trained in nuclear power plant fundamentals and
~
' plant systems, use of those systems to control or mitigate an accident J. J during which the core is severely damaged,' and operating practice.
w
- .V(,
(4) The applicant should also complete the additional instruction specified in Section 5.2.1.6 of ANSI /ANS-3.1-1981 in subjects related to the duties of an SRO. NUREG-1021 8 of 12 Revision 8
ES-202
- c. Education .
' The applicant should have a high school diploma or equivalent.
- 3. Limited Senior Reactor Ooerator
. a. ' Experience The applicant should have three years of nuclear power experkence that includes active participation in at least one refueling outage at the site for which the j license is sought or at a similar facility. Six months of the nuclear. power plant
{ experience must be at the site for which the LSRO license is soyght or at a ; similar facility owned by the same facility licensee. i
- b. ' Training l1 The applicant is expected to have satisfactorily completed a training program that ensures that he or she is qualifie'd to supervise fuel handling operations.
The program should be based.on a systems approach to training and is
~
expected to include instruction in at least the following areas: c g (1) nuclear power plant and health physics fundamentals and the principles ; of reactor theory and thermNyr;smics ~ (2) design features of the' nuclear power plant pertaining to fuel handling activities, including plant systems and equipment associated with fuel l
~
handling operations,~ pertinent IC systems, and features of the emergency core cooling systems (ECCSs) associated with the refueling mode of operation l (3): the use of installed plant systems to control or mitigate an accident in
, , : which the core is damaged during refueling operations 4
(4) operating practices and procedures that pertain to refueling, including < administrative, operational, surveillance, emergency, radiation control, and safety procedures; the technical specifications applicable to o refueling; and the requirements concoming communications and
' interfaces with the main control room l
_ The applicant should also complete a minimum of 80 hours of on-the-job
'~
j training (OJT) in refueling activities, including manipulation of the refueling bridge or similar refueling equipment.
- c. Education NUREG-1021 9 of 12 Revision 8 !
I 1
l i ES-202 The applicant should have a high school diploma or equivalent.
- 4. Cold License Eliaibility -
Cold examinations are those administered before the unit completes preoperational testing and the initial startup test program as described in the FSAR.
- a. Each applicant must satisfactorily complete the-training programs described in Section 13.2 of the FSAR and approved by the NRC. The NRC's review and
- approval are based on information contained in Section _13.2.1 of the Standard Review Plan (SRP)(NUREG-0800).
I Note: These NRC-approved training programs typically require ten startups on a l research reactor. This requirement may be waived if the applicant has i completed a plant-referenced simulator training program accredited by the I institute of Nuclear Power Operations (INPO).
- b. In lieu of the control manipulations on'the facility for which the license is sought (per 10 CFR 55.31(a)(5)), the Commission may accept evidence of satisfactory performance of simulated control manipulations as part of a Commission-approved training program on a simulation facility acceptable to the Commission under 10 CFR 55A5(b). ,
E. ATTACHMENTS / FORMS. _-
..__- a
^
Attachment 1, " Sample initial Application Denial from Region" l I
)
[ )
% (.yj--
NUREG-1021 10 of 12 Revision 8 U
1 ES-202 Sample initial Application Attachment 1 ! Denial from Region ' NRC Letterhead , l (date) 1 (Anolicant's name) l (Street address) (Citv. State. 7in enda)-
Dear (Name):
This is to inform you that your application ofIdata) for a (reactor coerator. senior reactor operator) license submitted in connection with the (facility name) is hereby denied. (Reaion to discuss deficionelae and which oart of 10 CFR 55 31. ES-202. NRC- annroved I facility trainino cresicm. or Raoulatorv Guida 1.8 was involved.) When you have met the requirements of 10 CFR 55.31, you may submit another application. If you do not accept this denial, you may, within 20 days of the date of this letter, take one of
- the following actions
i ,
~
! ' You may request that the NRC reconsider the denial of your application by writing to the 3 l' Director, Division of Reactor Controls and Human Factors, Office of Nuclear Reactor l Regulation, U.S. Nuclear Regulatory Commission, Washington, DC 20555. Your request must include specific reasons for your belief that your application was improperly denied. If the NRC determines that the denial of your application remains i appropriate, you still have the right to request a hearing pursuant to 10 CFR 2.103(b)(2), , as described below. You may request a hearing in accordance with 10 CFR 2.103(b)(2). Submit your i request, in writing, to the Secretary of the Commission, U.S. Nuclear Regulatory l Commission, Washington, DC 20555, with a copy to the Assistant General Cc,unsel for Hearings and Enforcement, Office of the General Counsel, at the same address. I If you have any questions, please contact (name) at (telephone number) l 1 Sincerely, ) l 1 l !- x (Regional branch chief or above) l Docket No. 55-(number) l - oc: (Faellity reoresentative who sioned the annlirant's NRC Form 398) l l . CERTIFIED MAIL - RETURN RECElPT REQUESTED L , NUREG-1021 11 of 12 Revision 8
ES-204 PROCESSING WAIVERS REQUESTED BY REACTOR OPERATOR AND SENIOR REACTOR OPERATOR APPLICANTS A. PURPOSE This standard provides guidance concoming the processing of waivers requested bygeactor operator (RO) and senior reactor operator (SRO) license applicants at power reactor facilities. b B. BACKGROUND #
/s$%f Y
k!?1 A I$ In accordance with 10 CFR 55.35, "Reapplications," andg 10 CFR 55.475?Waiverof Examinatiork and Test Requirements," an applicant may request to biscused froin"a written Msmanatiorf5f an operating test. The NRC may waive any or all of theffsdaminatiOIroquiremendi@C7 determines that the applicant has presented sufficient jUstilication in an effort to'ess edits the resolution of applicant requests, the NRC Operator LiceniidiBranch (OLB) has delegated the
' authority to grant routine waivers of certain operator licensing requirements to the NRC regional offices.
ff N :'(:R
%NM C. RESPONSIBILITIES v
yfhg kg/V 9#~ Wah%
- 1. Applicant /Facshtv Licensee a.
h An applicant may request a waiver
> ;,;';$ MyV.Ajt a licer!se requirement by checking the Statement - Licongeef/The app)licant shoakis also explain th requesting the waM in itemg " Comments."
Aix dFdME7 b.[4bThe facility Soonose's seniorgnonspement representative on site must certify t 4 j [NnallicenseMon, therii6 substantiating the basis for the applicant's N s malwar requestr$
\Y % $$ .
c. hacik; dienseesits designed by the same nuclear steam supply
~
dual . M$ists and EMted at approximately the same power level may re
~
)6tJheir operators. Similarly, if the units of a multi-unit facility are nearly i facility licensee may request a waiver of the examination v requiromidII the second and subsequent units.
In'eltheJ4ase, the facility licensee must justify to the NRC that the differences g betweekthe units are not so significant that they could affect the operator's t algio operate each unit safely and competently. Further, the facility licensee l-l
+
gg t submit for NRC review the details of the trrining and certification program. analysis and summary of the differences o: 'hich the applicants must be
- trained will include the following, as applicable:
facility design and systems relevant to control room personnel technical specifications NUREG-1021 1 of 5 Revision 8
ES-204 procedures (primarily abnormal and emergency operating) control room design and instrument location
- operationalcharacteristics administrative procedures related to conduct of operations at a multi-unit site (e.g., shift manning and response to accidents and fires).g
. the expected method of rotating personnel between units arsd'the refamiliarization training to be conducted before aNb erat'diassumes responsibility on a new unit ' A
<Aj W ;; *k @p . -^
NRC Regional Office h -
- 2. t;*
ff AAcM*yNe$l% d g%
- a. The regional office will evaluate waiver requests on se-b chb [gsinst the waiver criteria discussed in Section D of this Et3. NigP b.
V 'k The regional office may grant routine waive'rs iderstified in Section D.1 without first obtaining OLB concurrence. ' ff , ts, comNMtraining, or
~ However, waivers of experience @the;6u completion of examinations (e.g( genedclundamentals examination) not specifically identified in SecticifD.1 r . g Abiver regdesiliind fo$innilts approval office shouldtoevaluate recommendation the Ct the,sief, OLBffdr corid$NndE.' )
The region does Mknotsequire8 wdtlen OLB co[ncurrence to deny a waiver request, bdfshould din:uss its_diEision with OLB before informing the applicant; formal ~ currencNWigAs%diirable in some cases. A
- bkN
. on is M to reach a decision on the waiver request, the
%edditio shalt ger, irallfrequest the necessary information from the fa C:g$engiog N i u
Niriad.cIord.anceMthiES
.- 202.
- d. _.
grakor deny a waiver, the regional office shall promptly notify
, the apptoant. Jwriting conceming the disposition of the requestfincludingan
,. explanati6DdENEldenial. If time is too short to' notify the applicant in writing df
~
before t'hF$isiiifnhoridate, the regional office shall notify the facility training
. AM represenhive#by telephone concoming the disposition of the waiver request and
$!(- providegollow-up written response to the applicant. The regional office shall includeihe OLB Branch Chief on distribution for all waiver disposition letters.
region shall document the disposition of every waiver request, whether
!W _M ranted or denied, by completing the block designated "For NRC Use" on the applicant's NRC Form 3g8 and by entering the data in the operator licensing tracking system (OLTS).
. f. NRC examiners assigned to a particular examination will be notified of approved
- NUREG-1021 2 of 5 Revision 8
.c
ES-204 waivers by the appropriate regional supervisor and by an entry on the examination assignment sheet (ES-202, Attachment 4).
- g. If the applicant is determined to be ineligible to take the licensing examination, the regional office shall issue a denial letter in accordance with ES-202.
py D. WAIVER CRITERIA ffNk dY gn& Routine Waivers m.p "t k
- 1. % A If an applicant fails only the written examination A or ore c$kkM!$h
- a. ategogrof theoperating t
f test, the region may waive those examirdstion areas 7(categories passed. This is only applicable for the fifM"a" Examination and g(l place within one year of the examination Qthe ..pplicant failed.
-i I
b.~ The region may waive training requirerps speallied, the final safety analysis report (FSAR) when the FSAR authorizes waiver'of;thoseTapecific requirements and the applicant otherwise meets;%requiremerikhnaiver of some training requirements for applicants y,;ND licensed at a comparable facility).
- c. The medical dataA in suppgbf NN kANRCform396g. i./[
s ormallyjgood for six months from the date of the medicial examination f$r;aperson applying for an RO or an SRO instant license./greapplicistions follt$I5ng a license denial or withdrawal of an application, wedvers extenillhg the sidihonth period may be granted if the date of the originailiHEdical exeiminatiogdiwithin ; ; y;;; 24.. months;of the ocheduledfeemediMetion an65fkated licerising date innd item:177.fComments&~of ! dl 'M[thaN~appNhasMdevolihedinp/phisicalfor 46*/ ieNRCform?98$$ihental co^nMHGat;wouldbTe rop 6ttable iSROMpgM@Ein6"'[jkmedical Bi$$ good fo examination documented on NRC Form from the date of the medical examination.
- d. NNUM$h,'
M ha8e=8Eaam all ~ Regulatory Guide 1.8, Revision 2, are not considered to be walviisiindyberefore, do not require approval. For example, substitution of related i$dr$dalMaining for up to two years of experience for an SRO or up to one year @RO is not a waiver. However, training for the examination applied fg.may not be counted as related technical training.
" e. If the sty licensee certifies that the applicant has successfully completed a k
~-. trainiki program accredited by the institute of Nuclear Power Operations using
_. , acceptable simulation facility, the region may waive the requirement for ten i startups on a research reactor typically required by NRC-approved cold license training programs.
- f. For those facilities unable to meet the requirement for six weeks on shift at greater than 20 percent power (because of extended plant shutdowns or other NUREG-1021 3 of 5 Revision 8
r ES-204 extraordinary circumstances), this requirement may be waived upon application if the following criteria are satisfied: (1) Facility training objectives for the desired licensed position have been developed using a properly validated job and task analysis (JTA). gV The facility licensee's training program 4 is/*dk :INised en a sy (2). approach to training (SAT) usinithe fivisiements defined in 10 CFR 55.4. 6V MM6i? # hf le VW m.
"G9Wh w (3)' The facility licensee can accomplish, training- .,,(7 required for plant operation lstegreater than 20 per_ cent power
- using a plant-teforenced or NRC approved simulation"faiciiity, v.r/ w,x g, if an operator was previously licensed a fAcGltyhreapplies for a license at the same facility and license level, the twgion mayppursuant to 10 CFR 55.47, -
waive the requirement for the applicant:to pass a viritten'ifiiimination and an operating test ifit finds that the (1) previously rged dsor
.. ]kff Jesponsibilities competently and safely andscapabi f co' $j-n
[do so (2). to partiMon in ' facility licensee's requalification projiiIsiless thedtwo yeasifbefore the date of the license a on v
, ( fessfully . " Additional Training," pursuant to 10 CFR pc "
),and lity-prepared written examination and
"" shi, st which ensure that the applicant is up-to-date in the
(" liconestlipperator requalification training program
" " . - kbhY
, . isuccessfully completed at least 40 hours of shift functions J
(J@$)gr ppriate,the anddirection in the positionof an to whichoperator or senior the appPant will be operator, as A ns. signed (see 10 CFR 55.53(f));beforui;beingyssigned tilicertsed g duties.'
* ) complies with the requirements of 10 CFR 55.31
$q*
3 .jan.jspplicantistlfacBit/pathas..' c ompleted preoperational. testing.isTunable.10 4% PerformMMaignificantpontrolJnanipulations1requiredDMcE55;31.(a)(5) twause4.ar[ extended shutdownEMregion;may;processithejapplication;and administer.Mexarriination/tiutishall not;issuojthe.l license Until.thel facility jicenseejs@plieshMredgadence; that,the applicant. pas; successfuBy completed Mcontrol;manipulationsfrefer to;ES-501)sSituations ;other;than NUREG-1021 4 of 5 Revision 8
ES-204 n
.these..specifiednihe regu!stjen require. an exemption and must be. processed through;O.LBl
- 2. Examination Waivers for Previousiv Licensed Ooerators at Comoarable Facilities Depending on the justification provided by the applicant and the facility licodee, OLB will consider examination waivers for operators who were previoklicerisid at a comparable facility. Pursuant to 10 CFR 55.47, the Commission may waive any or all .
requirements for a written examination and operatirgGst. EM <
- 3. Multi-Unit Examination Waivers B
- a. Generally, personnel will not be examineN$n or wed to hold lich "different units" simultaneously. "DifferekSN$sf, ewned or managed 6y"A single facility licensee are defined for purposes o MEidard as follows:
N dh. units having the same vendor b ignifi <d$srent age and/or power level (e.g., Dresden Units 1 and
- i units having the same or rdesi ut different locations )
(e.g., Sequoyah andfhitts B ~ ~ *
*~
od) V i units having d
- n
' t venders (PWR ut located on the same site j (e.g., Arkan T ~ its 1 and 2, Mills e Units 2 and 3) i AP' OLB may authorigem limited f
r regr operator (LSRO) to be licensed at ; multiple site ed the manufactured by the same vendor and ' '
^
of si . The * ' Eust pass an examination that addresses
, differe ~ , designs, procedures, technical data, and administrative 4 oft facilities for which the license is being sought.
2
- b. ; g g the .
tion requirements for " identical" second or l --=_ s at tii%,same site, OLB may waive any or all requirements for a ! written and operating test if it finds that the applicant meets the criteria s . 10 CFR 55.47, as noted in item D.2 above. If the situation warrants, mmission may impose other examination requirements, such as _
! NRC-ad istered operating tests and written examinations concoming the plant differe s.
y
, /;
l l NUREG-1021 5 of 5 Revision 8 m
ES-205 PROCEDURE FOR ADMINISTERING THE GENERIC FUNDAMENTALS EXAMINATION PROGRAM A. ~ PURPOSE This standard describes the procedures and policies pertaining to administration of the generic fundamentals examination (GFE) section of the written operator licensing examination at power reactor facilities. It describes how the examinations are scheduled and constructed, how to solicit facility licensees for applicants to take the examinations, and how to promulgate the
- examination results.'
B. BACKGROUND Sections 55.41 and 55.43 of 10 CFR Part 55 require that the wriden operator li$snsing. examinations for reactor operators (ROs) and senior reactor operators (SROs) include questions on various mechanical components, principles of heat transfer, thermodynamics, and fluid mechanics. These regulations also require that the written examination address fundamentals of reactor theory, including the fission process, neutron multiplication, source effects, control rod effects, criticality indications, reactivity coefficients, and poison effects.
.kI'. .
The fundamental knowledge and abilities (K/As) required of an operator do not vary significantly between license levels or among facilities of the same ' vendor type.' As a result, the NRC implemented the GFE program to standardize the fundamental examination coverage for all applicants at pressurized and boiling water reactors (PWRs and BWRs). Having passed a GFE
~
as an RO or an SRO applicant, an operator will not have to take another GFE unless he or she transfers to a facility of the other vendor type.~ The GFE program does not pertain to limited senior reactor operator (LSRO) license appiscants. T ,.
- .. x~
Applicants do not need to take the GFE (nor obtain a waiver) if they were previously issued an RO or SRO hoense or an instructor certificate based on a site-specific written examination (on
- the same type of facility) that was administered between February 1982 and November 1989 and included the material covered by the GFE. Applicants who were issued a license before 1982 will have to take the examination'or apply for a waiver in accordance with ES-204.
The' GFE examinations f' or BWRs and PWRs are typically administered twice a year, on the Wednesday following the first Sunday in April and October. C. RESPONSIBILITIES 1 Facility 1[^ a.$ The facility licensee must cortify that all individuals who' plan to take the GFE are
" ~ '~
enrolled in a facility-sponsored training program that will satisfy the eligibility requirements for an RO or SRO license. The operator trainees need not complete all of the training required for the license before they take the GFE. I NUREG-1021 1 of13 Revision 8
1 ES-205 The facility licensee may use the sample registration letter enclosed with the NRC notification letter (Attachment 1) or any similar format that contains the - required information and certification. If the. facility licensee must add or delete an individual after submitting its registration letter, it should inform the NRC Headquarters' Operator Licensing Branch (OLB) of the change, as specifed in the examination cover letter, before the examinations are administered.
- b. When the examinations are received from the GFE contractor, the facility licensee shall reproduce and safeguard the examinations as described in the examination cover letter. )
- c. On the designated examination day, the facility licensee shall administer and proctor the GFE in accordance with the instructions contained in the examination j package. .
The facility licensee will start and stop the GFE in accordance with the time zone l map contained within the examination package. Late arrivals will be allowed to l' take the examination; however, all examinees must hand in their examinations at the completion time designated in the proctor instructions enclosed with the i examination cover letter (refer to Section C.2.d). - ~ p ,
- d. No later than the day after the GFE is administered, the facility licensee shall send the following items via overnight mail to the name and address designated in the examination. package: .,
[ .
~
the original answer sheets,- the signed exam cover sheets the sig security statements 2.- NRC Onorator Licensina Br'~anch and GFE Contrador R - ,
- s. , ; OLB will' designate si codrdinator to oversee the GFE activities with the regional offices', the'GFE contractor, and the facility licensees.
(,,,W
' b. The NRC will send a notification letter (Attachment 1) to each facility licensee 60 days before the GFE administration date. The letter will notify the facility licensee of the date of the examination and request a registration letter listing the licensed operator trainees to whom the facility licensee plans to administer the examination. A sample registration letter is enclosed with the notification letter.
- c. The GFE contractor will prepare the examinations as described in Section D of
" ~
this ES. The examiner assigned responsibility for developing the GFE shall submit the examinations to the OLB GFE coordinator and any other designated revowers at least 20 calendar days before the scheduled administration date. OLB will provide comments and recommended changes to the examination NUREG-1021 2 of 13 Revision 8
r l
)
i ES-205 l- author as soon as possible. The final examinations should be ready at least 14 l days before the GFE administration date. i .
- d. ' The GFE contractor will assemble the approved examination packages as described below, and mail the packages to the names and addresses designated by the participating facility licensees. The examinations should normally be mailed one week before the examinations are scheduled to be administered.
The examination packet will contain the following information, enclosures, and attachments: l cover letter (Attachment 2 is a sample letter) , i l - proctorinstructions ! security agreement single copies of appropriate exam, forms A and B exam time zone map sample answer sheet f l facility docket number sheet applicant docket number sheet appropriate number of answer sheets applicant answer sheet instructions' c
- e. On the day that the GFE is administered, the OLB GFE coordinator and GFE contractor shall be available to answer questions from facility proctors if the need
~
arises. :
. f. When the examination answer sheets are received from the facility licensees, the GFE contractor shall score, grade, and tabulate the overall item statistics, and generate facility and regional grade reports for each GFE examination. The contractor shall forward the regional and facility grade reports, including individual. scores and copies of individual answer sheets, and corrected answer keys to the applicable regional office for distribution.
The GFE contractor shall develop individual item statistics on all questions used 2 - on the GFE examinations. Questions with acceptable statistical characteristics
,D shall be m'oved into the " validated" GFE question bank.
The corbetor will provide copies of all grade reports to the OLB GFE coordinator, along with the following additional items: D..
~
exam-wide item statistics (PWR and BWR; analysis reports of specific items deleted or answers changed corrected answer keys original answer sheets origir # 3gned exam cover sheets
+' signeo security statements NUREG-1021 3 of 13 Revision 8
ES-205
- g. The OLB licensing assistant will ensure that copies of the final master BWR and PWR examinations are placed in the NRC's Public Document Room.
- 3. NRC Reaional Office -
- a. ' Regional management should assign an individual to coordinate GFE administration in the region.
- b. The regional operator licensing assistant (OLA) shall assign a docket number to each individualidentified in the facility licensee's registration letter. The OLA shall forward the list of names and docket numbers for each facility to the GFE contractor, with a copy to the OLB GFE coordinator, no later than 20 days before the examination administration date.
- c. The regional GFE coordinator should keep the OLB GFE coordinator informed of l any changes in the number of applicants scheduled to take the GFE at any facility. ,
- d. The regional office shall distribute the GFE examinations to their respective facility licensees. Sample cover letters forfacility licensees that did and did not participate in the examination are provided in ~ Attachment 3.
~
- e. The regional OLA shall' update the applicants' status (pass or fail) in the operator
. licensing tracking system (OLTS) and ensure that a hard copy of the GFE results is placed in each applicant's docket file.
D. EXAMINATION SCOPE AND STRUCTUitE Each GFE shall contain 100 questions covering the " Components" and " Theory" (including reactor theory and thermodynamics) sections of NUREG-1122, " Knowledge and Abilities ! Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors," or NUREG-1123,
" Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Boiling Water Reactors."
The passing grade for the GFE is 80 percent.
. .Qy l
The knowledge and ability (K/A) topics applicable to the GFE for PWRs and BWRs have been categorized into various component, reactor theory, and thermodynamics groups as shown in Attachment 4. The attachment also identifies the number of test questions required to evaluate each topic.
- The NRC will uN Institute of Nuclear Power Operations (INPO) Academy Documents 89-003,
" Generic Fundamentals. Test item Catalog - PWR Operator," and 89-004, " Generic Fundamentals Test item Catalog - BWR Operator," as the primary sources of test questions when developing the GFE. The ratio of previously used (i.e., " validated") test questions to new or unvalidated test questions will be adjusted as the size of the validated question bank ,
increases. NUREG-1021 4 of 13 Revision 8 him - I
r ES-205 The questions used on the GFE examination shall conform with the applicable construction and style guidelines in Appendix B. E. ' ATTACHMENTS / FORMS Attachment 1, " Sample Notification Letter" Attachment 2, - " Sample Examination Cover Letter" Attachment 3,. " Sample Results Letter" Attachment 4, "GFE Test item Distribution"
/
aS t' t
,1 NUREG-1021 5 of 13 Revision 8
l ES-205 Sample Notification Letter Attachment 1 NRC Letterhead (Date) (Name: Title) (Facility name) (Street address) l (Citv. State Zio code) {
Dear (Name):
The NRC plans to administer the generic fundamentals examination (GFE) section of the written operator licensing examination on (date) To register personnel to take the GFE, an authorized representative of your facility must submit a letter to the appropriate regional administrator with a copy addressed as follows: Chief, Operator Licensing Branch Mail Stop OWFN 9 D25 ' U.S. Nuclear Regulatory Commission Washington, DC. 20555 Your letter should identify the individuals who will take the examination, and it should certify that they are . enrolled in a facility licensee-sponsored program leading to NRC operator or senior operator licensing and that they will have completed their fundamentals training by the date of the examination. The letter should also identify the personnel who will have access to the examinations before they are administered (e.g., proctors) and the address to which the examinations are to be sent.' To silow the NRC to assign docket numbers, the letter should be received by both the NRC regional administrator and the Chief, Operator Licensing Branch, 30 days before the examination date. A sample registration letter is enclosed. Sincerely, (Anorooriate reaional reoresentativ3} Docket No. 50-(Number)
Enclosure:
As stated NUREG-1021 6 of 13 Revision 8
ES-205 2 Attachment 1 Enclosure (Name) Regional Administrator U.S. Nuclear Regulatory Commission Region (Number) (Street address) (City. Simta 7in en)
.i
Dear (Name),
(Facihty name) requests to have the following (number) individuals take the (BWR or PWR) generic fundamentals examination (GFE) section of the written operator licensing examination to be administered onIdais)- . Name Date of Birth Previous Docket No. (Insert the name. d=*a of birth. and orevise 10 CFR Part 55 Daelrat Number (if r.r##6) for each person.)
' l i
All of the listed personnel are enrolled in the (facihty name) operator licensing training program ' and will have completed the generic fundamentals portion of the program by the examination date. -" '
. s ._
- The following personnel will have access to the examinations before they are administered:
~
Hama Ildt , (Insert the name and title of ==nh norson who will have =ne=== to the avaminatians before thev are administered (e o.. nroctg -
~
Please address the examinations'as follows: Name.Tdle Street address - Cdy. State Zip code pg. . if you have'any questions, please contact (facildv contact name) at (telephone number). a, Sincerely, p
- - Name. tdle ec. ,; o Chief, Operator Licensing Branch
. NUREG-1021 7 of 13 Revision 8 !
1 ES-205 Sample Examination Cover Letter Attachment 2 (Date) (Name. Title of desianated addressee) (Facility name)' (Street address) (City. State 7in enda)
Dear (Name):
Your facility is scheduled to administer the generic fundamentals examination (GFE) section of
' the NRC's written operator licensing examination on Idata). (Name of coidi *6 is authorized under contract to support the NRC in the administration of GFE activities. .
~ Note: For security reasons, please open the sealed envelope now and page-check the examination using the enclosed checklist.. Then contact immediately and no later than idata) one of the persons listed below informing (him or her) that you have received this package and noting any discrepancies: ,
(Namel. (Telephone Number) (Name). (Telephone Number) _ r . This letter and its enclosures provide the instructions and guidelines for administering the GFE and retuming the completed exams.and related materials to (Name of contractor). Please read
. this letter and follow the directions in the accompanying enclosures now.
L . ;, /. Enclosure 1. Security Aer m ..L Please' refer to the enclosed NRC Security Agreement. A copy of this agreement must be completed by each and every exam administrator and/or proctor seeing or having kr-r;if-;= of the GFE contents. For security reasons, the number of-persons seeing or having knowledge of this exam's contents before the exam must be limited to . three persons who have a need to know. The top porbon of tho' security agreement is expected to be completed now, and the bottom portion immediately after the exam has been completed. Fill in the spaces for each individual's name and the name of ths. facility for both portions, and have the individual (s) sign the form (s). A ' Please note: The signed security agreements must be retumed to (Name of contracted along with the completed exam answer sheets before any' scoring will be performed.
, _. I l
i NUREG-1021 8 of 13 Revision 8 4 l _____-___ - - -- A
l 4 ES-205 2 Attachment 2 i Enclosure 2. Ewam Cooles. Two single copies of Forms A and B of the exam are provided. These attemative forms are identical in content; however, for security purposes, the test item sequence on each form is different to reduce the possibility of an applicant copying any answers from a nearby test answer sheet. (See the separate Proctor instructions in Enclosure 3 for further exam administration instructions.) l
~ You are responsible for reproducing the number of exam copies required for the number of individuals taking the exam. Prior to the exam, store the original copias in a locked cabinet or safe and reproduce the necessary number of copies only on the day immediately preceding the exam; in this case, copies should be made on Idatt). Please note: your total number of copies should consist of one half Form A and one half Form B. After making the necessary number of copies, secure the original and all copies from view of unauthorized persons, storing them in a locked cabinet or safe until the exam date.
Each individual taking the exam must sign the security statement on the exam cover page. This page must be removed from the exam copy and mailed to (Name of contractor) along with the answer sheets and administrator / proctor security. agreements. ' After the exam has been given, the exam copies become public knowledge and no longer need security. Exam copies, therefore, may be kept or disposed of as desired. Enclosure 3. Proctor instructions. The proctorinstructions detail the guidelines for : administering the exam. Please note that the specific instructions presented are designed to be adhered to and followed identically by each proctor at all facilities. This process will ensure uniform administration and equity of results nationwide. As noted in the Proctor Instructions, all GFE exams will be administered at the same time in accordance with the local time zone in which the facility is located. Enclosure 4 8vam Ansr Sheets. The appropriate number of answer sheets (extra copies included)is enclosed for the number of applicants you identifed to take the exam. All applicants must use the original enclosed answer sheets for recording answers during the exam. - Summary of items to be Returned to (Name of conir-.ter) The following items are to bI mailed via Ovem/pht Delivery Service to (Name of contractor) and postmarked no later thanIdatt) completed answer sheets
. applicant-signed exam cover sheets
, ad,ministrator/ proctor-signed security statement (s)
NUREG-1021 g of 13 Revision 8 l
s 1 I ES-205 I 3 Attachment 2
- l Mail all of the above exam-related materials addressed as follows
i (Name) (Name of contractor)- (Street address) 1 ' (City. State Zio mda) For further questions regarding the specifics of this exam, please contact (Name) at (telephone number). For questions regarding the GFE in general, please contact (Name). NRC, at (telenhone number). For matters regarding candidate withdrawals or cancellations, contact either (Name) or IName) I at (telephone number) for specific guidance. (Name). Chief Operator Licensing Branch. Division of Reactor Controls and Human Factors
. ' Office of Nuclear Reactor Regulation 1
Enclosures:
As stated - # Distribution: w/o enclosures'.
~
N-Director, DRCH J'
~ ; Chief, OLB .
OLB GFE Coordmator Project Manager
' Public 4 1 ". -
' ~
OLB R/F - "1 ' k;
~ i.
- p I "sig . Wm l
4 NUREG-1021 10 of13 Revision 8
ES-205 Sample Results Letter Attachment 3 NRC Letterhead (Date) (Name. Title)
-- (Facility name) .
(Street address) (Citv. State Zio code)-
Dear (Nama):
(*)' On {daial, the NRC administered the generic fundamentals examination (GFE) section - of the written operator licensing examination to employees of your facility. Enclosed with this - letter are copies of both forms of the examination, including answer keys, the grading results for your facility, and copies of the individual answer sheets for each of your employees. Please forward the results to the individuals along with the copies of their respective antwer sheets. A
. P" in the column labeled RESULTS indicates that the individual achieveci a passing grade of 80 percent or better on the GFE. Those individuals having an "F"in the RESULTS column failed the examination. '*
l (") On (datal, the NRC administered the generic fundamentals examination (GFE) section ) of the written operatorlicensing examination. (") Your facility did not participate in this examination. However, a copy of the master ; (BWR or PWR) examination, with the answer key, is enclosed for your information. If you have any questions concoming this examination, please contact (Name of the OLB GFE coordinator) at (ohone number). .. Sincerely, i (Anorooriate reoional reoresentative) Docket No. 50-(Number) - (*) EnclostNes: _ I
- 1. Examination Form "A" and "B" with answers
- 2. Examination Results Summary for Facdity Name ,
- 3. ' Individual Answer Sheets j
(")
Enclosure:
As stated [ Paragraphs marked (*) apply only to those facility licensees that participated in the
- examination, while paragraphs marked (") apply only to those facility licensees that did not participate in the examination.]
... j
' NUREG-1021 - 11 of 13 Revision 8
l .3-t ES-205 GFE Test item Distribution Attachment 4 Pressurized Water Reactors No. of K/A Topic Items Groun I Comoonents 191001 Valves 4 191002 Sensors and Detectors 10 191003 Controllers and Positioners 5 191004 Pumps 7 191006 Heat Exchangers and Condensers 3 191008 Breakers, Relays, and Disconnects 7 Groun ll Comoonents
- 191005 Motors and Generators 5 191007 Domineralizers and lon Exchangers 3 Grouo i Reactor Theorv 192004 Reactivity Coefficients . . 4 192005 Control Rods 4 192008 Reactor Operational Physics 8 Groun ll Reactor Theorv .
192003 Reactor Kinetics and Neutron Sources 2 192006 Fission Product Poisons 6 Group 111 Reactor Theorv 192001 Neutrons 1 192002 Neutron Life Cycle ' 2 192007, Fuel Depletion and Burnable Poisons 1
. Group i Thermodynamics 193009 Core Thermal Limits 2 193010 Brittle Fracture and Vessel Thermal Stress 5 Grouo ll Thermodynamics 193003 Steam .
2 193007 Heat Transfer 2 193008 Thermal Hydraulics 8 1 i
.v Group lb Thermodynamics 193001 Thermodynamic Units and Properties 1 193004 .. Thermodynamic Processes 2 193005 Thermodynamic Cycles 1 i 193006 Fluid Statics and Dynamics 5 l L
l Total items 100 i l NUREG-1021 12 of 13 Revision 8
e ES-205 2 Attachment 4 Boiling Water Reactors No. of K/A Topic items l Comoonents 291001 Valves 5 291002 Sensors and Detectors 9
- l. 291003 Controllers and Positioners 3
!, 291004 Pumps 8 291005 Motors and Generators 5 l 291006 Heat Exchangers and Condensers 6 291007 Domineralizers and lon Exchangers 3-291008 Breakers, Relays, and Disconnects . 5 Groun i Reactor Theorv 292004 Reactivity Coefficients 2 292005 Control Rods 4 292008 Reactor Operational Physics 8 Grouo ll Reactor Theorv 292001- Neutrons 2 292002 Neutron Life Cycle 2 292003 Reactor Kinetics and Neutron Sources 3 292006 Fission Product Poisons 6 292007- Fuel Depletion and Bumable Poisons -1 Grouo l Thermodr.amics 293007 Heat Transfer and Heat Exchangers 3 293009 Core Thermal Limits 7 Group ll ihermodynamics 293003 Steam. , - 2 293004 Thermodynamic Processes 2 293008 Thermal Hydraulics 7 293010; Brittle Fracture and Vessel Thermal Stress 2 Grouo ill Thermodynamics 293001 Thermodynamic Units and Properties 1 293005 Thermodynamic Cycles 1 293006 Fluid Statics 3 Totalitems 100 l NUREG-1021 13 of 13 Revision 8
p
- ES-301 PREPARING INITIAL OPERATING TESTS -
A. PURPOSE' All applicants for reactor operator (RO) and senior reactor operator (SRO) licenses at power reactor facilities are required to take an operating test, unless it has been waived in accordance with 10 CFR 55.47 (refer to ES-204). The specific content of the operating test dipinds on the type'oflicense for which the applicant has applied.' .- gyg gjP~ e This standard describes the procedure for developing ope test [t te requirements of 10 CFR 55.45, including the use of react $6iant simulajidkfacil$es and the - ' conduct of multi-unit evaluations.. F V NjnTK g[n w .
~ B. ' BACKGROUND. S
[ ,
- To the extent applicable, the operating test will require be. ;to demonstrate an understanding of, and the ability to perform, the actions [necessE MAccomplish a representative sampling from the 13 items identified iny.0 CFR 5535(4)lil013]tems;do,not need to be sarnpied onjeverygoperateg; test). Ir program and from information taned in the final gajib fsportpsystem description manuals and operating procedures, the facility license and I!censeamenden$nts, licensee event reports, and other materials requested from th cility licaiEee by DAhmission.
The structure of the operating test' 9Nstat 'part, byJO CFR 55.45(b)(1). It states that the test will be administered in a plarfiIlalk-th * ' and in either a simulation facility (as defined in 10 CFR 55.41that the Comrniss , has apprewa(idisimulation facility consisting solely of a plant-referenced,simulat _ defined Arps% 55.4) that has been certified to the Commissilosi facil . V The walk-th y of th j jest consists of two categories, each focusing on specific knowl . ties ( uired for licensed operators to safely discharge their
. assigned d ilitie third category of the operating test is administered on an NRC-a%7oved or . ed simulation facility. Unless specdically waived in accordance with ES-2 mented on the Examination Assignment Sheet (Form ES-201pchment 4), all .. "togories must be completed for every license applicant.'
" ; category of the ~ ting test is briefly described below. Section D of this standard ions for developing each category. Procedures for administering and p"N"nddes graijrg a detailed in , test are contained in ES-302 and ES-303, 1 [bre A. "Adidnistrei;se T=!cs" This category of the operating test covers K/As that are generally associated with the administrative control of the plant. It implements items g through 12 of 10 CFR 55.45(a) and is divided into four administrative topics, as described below. The depth of NUREG i1021 1 of 27 Revision 8
1 ES-301 coverage required in each topic is based on the applicant's license level. The
. applicant's competence in each topic is evaluated by administering job performance measures (JPMs) or by asking specific questions.
Topic A.1, " Conduct of Operations," evaluates the applicant's knowledge of the daily operation of the facility. The following subjects are examples of the types dfInformation that should be evaluated under this topic: : ,P'
/:,,, f,,m shift tumover %g ;t shift staffing requirements / ,
temporary modifications of procedures /"N~: ~ ~;g%
},[';f k reactor plant startup requirements mode changes hr g j,pr
/j('Q(N , ,y
.J 1
);,Qgy 7, e -
plant parameter verification (estimated critical position (ECP), nce,heat etc.) bala,;y short-term information (e.g., night and stan'ding'crders), key control security (awareness and familiarity) 4 / ' %V }' M%!
#U% p '4 fuel handling # '6t'/,
x6 ng :!% M pe?ib,' ~ . Topic A.2, " Equipment Control," addresses the administrativetequirements associated with managing and controlling plardiystems'isEdie(dihhunt?The following subjects l exemplify the types of information that sho6fd be e' 'under this topic: surveillance testin
/*E i maintenance - . ~-
tagging and clearances c _emporaryjviiiiillinistion of s" (
.: y Muniliarit/G5liTAidhse of pi and instrument drawings '
$$ hM
- Topic _ ytadiation Captml,Ayevaluates the applicant's knowledge and abilities w
- respect ~to vadallen hazardiifaddWotection (of plant personnel and the public). The followird" Minemplifyytypes of information that should be evaluated under this t .
]
use and . of portable radiation and contamination survey instruments and I personne
?,4 - - - knowledh{iino6itoring of significant radiationequipment hazards -
j - the abdity to perform procedures to reduce excessive levels of radiation and to n guadlYngainst personnel exposure
~
inddtion exposure limits and contamination control, including permissible ievels 3 2 in excess of those authorized ! radiation work permits ' control of radiation releases Topic A.4 " Emergency Plan," evaluates the applicant's knowledge of the emergency NUREG-1021 2 of 27 Revision 8
1 l ES-301 plan for the facility, including, as appropriate, the responsibility of the RO or SRO to decide whether the plan should be executed and the duties assigned under the plan. The following subjects are examples of the types ofinformation that should be evaluated under this topic: lines of authority during an emergency emergency action levels and classifications j Q ,4 [ i emergency facilities g emergency communications
, 3:^7 4p ' y4 emergency protective action recommendatioris y V%; u,f %p >
l e mA : Category A is administered in a one-on-one, wa _ rougL at and that four tops 4y l areas are graded collectively (refer to ES-302 a5d M). kN@d3
\f 4 j i . 2. Cateaorv B. " Control Room Systems and Facility Wa0$n% k"h I 8 %$$
This category of the operating test is used to determine ifM; nt's knowledge in the area of plant system design is adequateliI$I(gdeterms i, jpplicant is able to safely operate those systems. As suchjMcagimplema items 3, 4, 7, 8, and g identified in 10lCFR 55.45(aksit ompassesalso enc types several $the requiremei of systems, including primary coolird, emerg$$$QTelay heat removal, auxiliary, radiation monitoring, ariid instrundritationWoodtrol. y yp-- Category B is divided into beate
~
ses on%gones. Thellrst and larger subcategory (B.1 "mostJnvolved Control Room (i.e., thosdSystems")$ving co8IrlNsand frdications in the mai . second subcategg'(dMFacility WakfThiedgh") ensures that the applicant is familiar winWclesign and operation of syitimii5cated outside the main control room. The A$50cindii1knowledK4Inii'$lulities relative to each system are evaluated by ad * 'JPMsa W " llow-up questions. ! Su and B.2 are dministered in a one-on-one, walk-through format and a " ded' r t'o ES-302 and ES-303). 4 l
' 3.- ' Catsoorv C. "Ir.i - - Plant Ose stions" I g *- !
This category ofthe operating test implements items 1 through 8 and 11 through 13 identified in 10;CFR 55.45(a). This is the most performance-based category of the l ( rating
^
~ 'and is used to evaluate the applicant's ability to safely operate the plant's
{ er dynamic, integrated conditions. j e simulator test is administered in a team format with up to three applicants (or surrogates) filling the RO and SRO license positions (as appropriate) on an operating crew. (Refer to ES-201 for additional guidance on crew composition and ES-302 for test administration instructions.) This format enables the examiner to evaluate each NUREG-1021 3 of 27 Revision 8
ES-301 applicant's ability to function within the control room team as appropriate to the assigned position, in such a way that the facility licensee's procedures are adhered to and that the limitations in its license and amendments are not violated (refer to 10 CFR55.45(a)(13)). Each team or crew of applicants is administered a set of scenarios designed so that the examiners can individually evaluate each applicant on a range ofpompet,encies applicable to the applicant's license level. Appendix D describes'thosecompetencies, and Forms ES-303-3 and ES-303-4, the " Integrated Plant Operdsidid85mpetency Grading of specific ratingWorksheets" for ROs factors to be considered during and SROs, th*shgrading break process dognach9 c6@dra> Each applicant must demonstrate .h?f sf hh tsison:.t every proficiency #or copeten her license level. The only exception is that SRO1 Competency Number SNl Control
~
Boa'rd Operations,"is optional for SRO-upgradeIiippbcants.
,~ n y C. RESPONSIBILITIES
[/ .
- 1. Facility Licensee g[$h (h The facility licensee is responsible 5WC 26 f [the V folloising activities es apI g
upon the examination arrangeme confirmdiiMui35ENRdiegional office in accordance with ES-201 appro " ely foudInonthYE5kUMihe scheduled examination ; date: F f~ f a.' Prepare proposed eramination outlineswaccordance with Section D and submit th f
& em '-
to theionalo]%)stg NRC and approvalin accordan i th'e terials necessary for the NRC regional office to prepare l iew W examination (s) (refer to ES-201, Attachment 2). a l c. Aview , sfinal operating tests in accordance with the previously ) appro examination outline (s) and the instructions in Sections D and E, and !
.- M$the NRC regional office in accordance with ES-201. l submit t t$gW l
. d. Make theWimdlation facility available, as necessary, for NRC examiners to prepare niir the operating tests,
- e. geg h the NRC in the regional office or at the facility, when and as necessary, to review the proposed operating tests and discuss potential changes r to ES-201).
- f. Revise the operating test outlines and the final tests as applicable and as agreed .
upon by the NRC regional office (refer to ES-201). The NRC retains final ) authority to approve the operating tests. i NUREG-1021 4 of 27 Revision 8 ____----_-}
p \ ES-301
- 2. NRC Reaional Office The NRC regional office is responsible for the following activities:
- a. Ensure that the operating tests are developed in accordance with Section D. l w i x -
- b. Ensure that the operating tests are reviewed for quality in[accordance with l
Section E. A. @nE%f j Meet with the facility licensee, when and j*% as$800
?W l
- c. appropriate.ito prereview the
' /k+ ; operating tests in accordance with ES-20W F V* . M g' I n %g;a,.;gpl? ' D. INSTRUCTIONS sjb 9gg37 N S$A Prepare each category of the operating test in accordanceMiNitidfollowing general guidelines and specific instructions: g71 $$$$ l 1. General Guidelines Can%tg a. i k in an effort to reduce examina n preparabon effort, the same operating test may be used to examine m6It'iple appil55WI6diir60fator crews. Depending on the number and license .WiIihel of the[shplic$dbeirijiexamined, it might be possible to use the samB# set of JeMs and siii~rsirios to examins all of the applicants if the ope'rii@ig testJi[idministeNId in multiple segments (e.g., single scenarios or two-f60DPMs) afibh of whicWcan be given to all of the applicants in a single day.gJh5Ificility lic$EIsse andMIe NRC chief examiner shall discuss the options and to greemeksdM$$ocess before developing the operating
; tests. .
- MF s
' . ize p aand maintain test integrity, varied subjects, systems,
'sNE5pieristions sSIlj6@ usted with applicants that are not being examined at
-n m x we
- same11% unlessynessures are taken to preclude interaction among the appiid$ntiM9trl;p er i;" ;;;w b;t;;na nen;f;; ;per;;;ng int; ;h;ll b; -
c.;rlrrl.dhNme 3 s'and; simulator scenarios shall not be repeated on successidis%UM@d}.;r; th;n thr;; (;n; ln th; ;;n ;f ;pgred; 000;) c JIM;::I$ dlf;;tly frera the b;nk may b; reputed d;y t;-d;y (;.;., mw JIM; m.;y m M 7:;::::: a ;; a n di; d;y;).
)g
'4fing tests written by the facility licensee may not repeat test items
.8 ulator scens ios 'or JPMs) from the applicants' audit test (or; tests if the icant;is retaking the' examination);given at or near the end of the license training class. Simulator events that are similar to events that were tested on the audit examination are permitted provided the actions required to mitigate the transient are significantly different from those required during the audit examination.
NUREG-1021 5 of 27 Revision 8 t l
7 ES-301 Sufficient operating test materials shall be developed to ensure that all applicants
. can be tested witn Le available personnel according to the schedule agreed upon by the NRC regional office and the facility licensee (refer to ES-201).
- b. ~ To the extent permitted for each category of the operating test, select _and modify testing materials (i.e., JPMs, questions, and simulator scenarios) friirithe facility's examination banks. Every selected test item muIshsatisfy the qualitative and quantitative criteria specified for the applicable sectI66"Of$in operating test i or be modified accordingly. $ %'J ~h g
f&y @, s,. j$,
- c. Consider the K/As associated with normal . bnormal, sand emergencyitasks and" 1 evolutions as a source of topics for use iNivaluatiniiiipplicanfcompetehepW {
each category of the operating test.
%dr y:A
+gj l'/ )
The knowledge and abilities associated with the tasks,and questions planned for the operating test should have importarycefa' c tors of atleast 2.5. Tasks with importance factors of less than 2.5 may be used if,there is a, substantive reason for including them (e.g., a recent licensee event or a. t system modification). 6@ 'i ;
"s The K/As should be appro e to th[ y(.m. equirements for the applicant's license level. for to .. . cil '~' J'd an task analysis (if available) leaming objectives, and 'her refefisihce ma H5 confirm that the operating test is correctly orient'edlo the fitSlity and " applicant's license level.
hf
- The facility licensee's site-s . tasiclist may be used to supplement or ordceEby-cas hn@isi6ted individualitems in the NRC's K/A override,d$deEto
. alogs maintMidizaMdh,ation consistency, the site-spe not bessed ace ofIbYentire K/A catalog.
~
~ d.
ing materials (JPMs, scenarios, and questions) for ill,e;, Meting test, ensure' t the materials contribute to the test's overall ntinifhetween those applicants who are competent to safely
/op~ erat .,_
' would reliin license s nd those who are not. Any test items that, when missed, ons regarding adequate justification for denying the applicant's o Eot be included on the operating test.
- e. SRO a nts, whether upgrade or instant, will be examined for the highest on-shift position for which the SRO's license is applicable (e.g., shift supervisor),
? , less of the position to be assigned when licensed. SRO applicants should i
% nstrate their supervisory abilities and an attitude of responsibility for safe j
( operation, and are expected to assume a management role during plant transients and upset conditions while taking Category C of the operating test. The operating test briefing, discussed in Appendix E, ensures that the applicants are advised of this policy. NUREG-1021 6 of 27 Revision 8 i
ES-301 Differences in administrative controls and facility design will affect the SRO's i responsibilities, but, in general, the following guidelines should be used to
' differentiate the SRO operating test from that of an RO.
In directing licensed activities, the SRO must evaluate plant performance and make operationaljudgments accordingly. SRO applicant 5'should, therefore, be more knowledgeable in areas such 4(opegsUdg
~
characteristics, reactor behavior, and instrument iriterpretation. In directing licensed activities, the Nt ust h and more[ thorough knowledge of facility administrative co6trols and idethods, / including limitations imposed by thiregulationiand the^facihty'stech cat specifications and their bases. pjgv 9/ u Ngf The SRO may be assigned responsibilities,for auxiliary systems that are outside the control room (e.g., wasFd@sistNi((Because are not normally operated by licNir.ed op have these additional responsibilities, the SRollicense applicant should demonstrate N knowledgeg$Edhi[$ddnss of sud)iMims maximum permissiblexo ffluent isl$ase rates, and other as they rel radiological consider $il6ns. f. Incorporate facility-spedlic and i -operating experience into the operating test wheness5ossibilf. try-geDocume$tation such as licensee even reports, significantEriEt repoM[and servide information letters are readily available sourcesWoperationally orient $1 plant anomalies. Evaluate ant acci nces (DAss) for the facility to determine if jlicy are sIMabli36ftesting, AifliIsampling basis, during the dynamic simulator Mihik-throtI%i$isiMDAss are those sequences that contribute significantly to NEMncy ofMbiEhsge as determined by the facility licensee's
'M risk as*iiOstnint (PRA) or individual plant examination (IPE).
$$$ji[dA
$ Y The P $ltould also be used to identify risk-important operator actions.
Chapter onal Perspectives," of NUREG-1560, " Individual Plant Exami ram: Perspectives on Reactor Safety and Plant Performance," r identifies @ number of important human actions that may be appropriate for evaluati0d on the operating test. In determining what actions to evaluate, do not overlodliactions that are relied upon or result in specific events being driven to i
" low'dsk contribution. This will help identify those human actions, assumed to be
% reliable, that might otherwise not show up in a list of risk-dominant actions.
- g. If the applicants at a facility qualify for dual or multi-unit licenses, the operating tests should evaluate their knowledge of the design, procedural, and operational differences between the units.
NUREG-1021 7 of 27 Revision 8
ES-301 Divide the operating test coverage among the units and do not become predictable by conducting the walk-through tests on only one unit. Different applicants may be examined on different units, or each applicant may be asked to explain or demonstrate his or her understanding of variations in control board ! layouts, systems, instrumentation, and procedural actions between the units at
' the facility. /
Most dual- or multi-unit stations have a simulator that after only oneis;J bmod of the units. Therefore, ensure that the appligants are%g[ tested on the different systems, control board layouts, aridany other ddlerances;between theg idE.1:5fushiple, after units during the walk-through Test on Brcportion of thfi5peratingr$ control room systems portion of Categori of>the!6perating test"chuidM$" administering administered on Unit 2 or Unit 3 orCategory C ofM8#the operatir i both. \$ 5 h. The systemsoperating and components, and test should operations and e examine nts a brya[h've;kh@ jhree categories of material that is the test covered should on the not be redundant, written examinatioi iM%icularlynortshould ldp$ri i they dM"dnt that C and C be developed and revioIMd"@$5YM to probidde the sam events from appearing on bdth parts $ttielest%8 I.
// [ N&f Every facet of the operating test,Jncluding ttie~qdestions and answers, JPMs, and simulator scengshould N planned 3esearched, validated, and documented to the. maximum extent possible before the test is administered.
J. e s k Examiners vdibwill be ad operating tests but were not involved in
'their devel$hid$niTare expeE$$$ search and study the topics and systems to aminlisih que f rating test so that they are prepared to ask whatever j~~ be necessary to determine if the applicant is in thdC amination team members are strongly encouraged lNi$NMagroup%Ahe chief examiner to review the examination materials
%fd$s been approved for administration by the responsible supervisor.
The dis uld focus on those test items that might require extensive cueing b ,m inor and those that are unique to the facility and require a respon , fMnt from what the examiner might expect based on past exponerlap. Q JP, ould include the elements identified in Appendix C (e.g., initiating and i sting cues, critical steps, and pelformance criteria). The guidelines and (or equivalents) in that appendix should be used when developing new JPMs. Facility procedures may be adapted for use as JPMs by identifying critical steps and entering comments on how to execute particular steps.
- 1. The prescripted questions for Categories A and B may include a combination of open- and closed-reference items. Open-reference items that require applicants NUREG-1021 8 of 27 Revision 8
p.. : o , i i i ES-301 1 to apply their knowledge of the plant to postulated normal, abnormal, and emergency situations are preferrad. Closed-reference items may be used to evaluate the immediate actions of emergency and other procedures, certain automatic actions, operating characteristics, interlocks, and-set points ltand routineradministrativelectivities, as appropriate to the facilitythoweVeysi.ich - ! questionsjhpuidj.not exceedj30 percent of thejtotalin:eitherjest C,ilibory. , Minimize the use of closed-reference questions that relyply,on memory, and j cwM h dopot use 3 of-open-reference questions whose answer can simply be
. looked up if trie applicant can find the correctTdScumerit.@jA l Refer to Section B.6 of Appendix W C for,regarding more guidancethe X, gp f ? 'k
/%$
development and use of open reference [ questions,f,or'the walk-through. tiiDV 1 If it becomes necessary toif deviate from 5%@d%gg NgjgF ! mi outhne that has been approved by j the NRC chief examiner in accordance wit @ES;201hiscuss the proposed l deviation with the chief examiner and obtisn dENSEre6be.before proceeding with i I hr$$osal could not be the changes. Be prepared to ment explain is wlilt the oriM*60nsidO5"d a implemented and why substitute. 4~the g proposed;%'g4
,;/ }af % 'k,k
- 2. Specific Instructions for Cateoorv N*Administrat'iOis TN s Although the administrative t ma tely, it is preferable, whenever possible, to linkIntegrate [siss0biate,[y
~
d$sm with taskstd andamin events conducted during Categories B and $lowev Ils import 56t to keep in mind that the applicant's proficiency in the adpiiniidiitive to "$hould6Edeliberately evaluated and not inferred solelffrom observaii5$shde du M$15Elator portion of the operating test. (b
~hMh k$NA .
a!W > For each of thei$bliiistrative topics listed below, select the required number of Nisut$b6ts to beWaI650idduring the operating test. Section B.1 provides
$6bjects that should be evaluated under each topic; the
,( <;p pq , ?, 4 ;
IDEiG ( m:4r Number of Subiects
-oyy j A.1, "Co
\' 1 et of Oper?.tions" 2 !
A.2, "Eq0ihment Cont ol" 1 A.3, "[tiidiation Control" 1
- A.4 mergency Plan" 1 1
dR for each administrative subject, determine the best method for evaluating the applicant's knowledge or ability in that area. Although a performance-based evaluation, using a single administrative JPM is generally preferred, two prescripted questions may be used to conduct the evaluation in each specific , subject area selected for evaluation. The questions may be associated with 1 NUREG-1021 g of 27 Revision 8
ES-301 Category B JPMs (as additional questions) or they may be administered separately,
- c. In general, SROs have more administrative responsibilities than ROs, so SRO applicants should be evaluated in greater depth on the administrativejopics. RO applicants need only understand the mechanics and intent of the related subjects, as they pertain to tasks at the facility. /? p,V
/W
- d. The following specific guidelines should be,ahrd wh'en selehting or developin questions or JPMs to confirm the applicant's competenediwitti*rigird to each topic: ,;[
s p
? \yj%u, ,.s
,f Toolc A.1. " Conduct of Operations" 7[\**g;~
$j" A Njg/
Many of these subjects can be covered within the fra, mework of a shift tumover
~
or by integrating them into other discuss!oris,'a~s'they a'pply, throughout the examination.
%fT% a The subject of fuel handling p canho%?lk coveredin kih the conNol room, but attempt to cover this subject in the fuelh&fidirid$resi$fthe planit'whenever possible. The RO applicant should be awid of his4fNsfSutioihriftne control room during fuel handling. These duties insiude morsioring'IdiiU$iIlitation and responding to alarms from the fuel hadiiling ared,Fcommunhiating with the fuel handling and I ting afiitems fronIdhe control room in support of storage facility, and%' For thisRO appnt, evaluate topic (re) fueling operatidi$i.
alterations, new aIMi spent f0iiiAtorageland movement, the design of the fuel handling a 6$$of the fuhdisiiillipitools, and fuel handling casualties.
. %i%h"
. japplica awareness should be evaluated by observing his or her duringYi g test. However, passive observa*lons, in and of theirIANes, ars *iii to justify an evaluation in that subject area. It is esiii6iiiNEli30 questl. n SRO applicant on applicable aspects of the facility's g~ securits;s e operating ,
crew's interactions with the security shift supervis Tonic A. ulomant Control" Q Thesebbjects can be evaluated within the framework of a normal maintenance evof6n.- For example, ask the applicant to demonstrate how he or she would a failed system or component out of service, initiate maintenance on the system, and test the system before placing it back in service. Tonic A.3. " Radiation Control" This topic is best covered in conjunction with the JPMs and questions prepared NUREG-1021 10 of 27 Revision 8
r l l ES-301 for Category B.2 of the walk-through (i.e., local systems and operations). It is most appropriate to evaluate these subjects during an entry into the radio-logically controlled area (RCA). The levels of knowledge expected of RO and SRO applicants in some radiation control subjects are significantly different. The RO's duties generall/7equire knowledge of radiation worker responsibilities and operatidn.ofpl56t systems associated with liquid and gaseous waste releases. Th5hei5ehhe depth to l which RO applicants are eva!uated should flinWed t$$teiponsibilities and l 9 I
. however, the monitoring requirements before, during;;end after the: release 4The S may be involved in reviewing ariid approving %Ieas'i"$$Nnds'andj! l should be cognizant of the requirementsIA5sociated with those~rklAsses[as Well !
as their potential effect on the health anl(Eistygthe public. Th'i'E ) applicants may be asked to discuss or sirnulatete,, lanned with a JPM) a release (e.g., liquid, gaseous, or containmd6fhdrg&} when examining these topics. [ %' ! Tonic A.4. "Emeroenev Plan" g#$ \fY' j There are significant differenc/EM % es between the knowledge required of RO and l SRO applicants in this are8RO ap%$Giididniiifamiliar with the ! emergency plan and witgeir plappecifih7ejjonsibilities under the emergency plan implementing procedures (plPs). SRO applicants, however, must demonstrate aikNlidnal kniiIviedge based upon their responsibility to direct i and manage the iMmentatiGiiof the EpPs during the initial phases of an j emergency. Because of thiir$RO app $sants should have a more detailed ! understandiMMeprossieve fgmgcedure EPIPs actio@n fecommendations, and com mentsp. s.
, , is best by integrating it into a discussion of a Category C qui %lementation of the emergency plan, or by conducting a
. //j4 %,PM.
J of the emergency plan. Such a JPM can be conducted
.f immedi . _.
a simulator scenario or during the walk-through (Category
# A or B) examinadon.
, y
- e. The plarmed administrative subjects should normally take no more than 1 hour and 14 hours to administer to RO and SRO applicants, respectively. I 7,
T Vorm ES-301-1, " Administrative Topics Outline," briefly describe the i
^
ministrative subjects selected for evaluation and the method (s) by which each subject will be evaluated. The method of evaluation should include the title of any planned JPMs and a brief summary of the proposed questions.
- g. Forward the completed outline to the NRC chief examiner so that it is received NUREG-1021 11 of 27 Revision 8
I 1 ES-301 by the date agreed upon with the NRC regional office at the time the examination
. arrangements were confirmed; the outline is normally due at least 60 days before the scheduled examination date. Refer to ES-201 for additional instructions regarding the review and submittal of the examination outline.
The NRC chief examiner and responsible supervisor shall. review the test outline c,everage as soon as possible in accordance with ES-2015And fo:G5rd any - comments to the originator for resolution. A s
- h. After the .NRC chief examiner approves t ._ operating 0 test 6 ut,jepare the fk final Category accordance with the A testoperating general materials (i.e., theSectioD3Nh te%guidelinii'in JPMs, questions, an'd reference question guidelines in Append {B$andihe JPM guidelinssj7 Appendix C. @A9 %
i. When through testthe usingmaterials Form ES-301-3,are" Operacomplete, revie[ ting Test Quality Checklist." This review shall be perfoW5sd,in conjunctiod$lliithe associated Category B walk-through and thit@jdhid$imulator o'paigriing test as noted in Sections D.3 and D.4. W ff
-:h Submit the entire operat test p aget .
nated facility reviewer or the NRC chief examirmAas app,rdpriate, for$_E_iew and approval in accordance with Section E. The "must be^ received: the NRC chief examiner at least 30 days before the s,chiduled adininistration%Inte, unless other arrangements have been made.g, [#/ f$%g(p#
- 3. hatructi orv B oom Svstems and Facility Walk-Throuch" lofthe h evaluates the applicant's systems-related K/As by havingihe'a taskgNA"gh'li$nnt ma perSr(n]%ed tasks and probing his or her kn Camgory B M@isend 4pestions are in addition to and should be d
/ents and evoiG0ihisMi6 ducted during Category IdY Refer to Section 1.g of the K/A catalog applicable to the type of reactor for which 4M, s. the applicant is seeking a license (i.e., NUREG-1122 for PWRs and NUREG-h%
1123 fcIt'BWRs). From the nine safety function groupings identified in the ! N& catal6g', select the appropriate number of systems (see the table below) to be ! 42 i [ evaluated for each subcategory'of the test based on the applicant's license level. sgghPrhe emergency and abnormal plant evolutions (E/ APES) listed in Section 1.10 of the appropriate NUREG may also be used to evaluate the applicable safety function (as specified for each E/ APE in the first tier of the written examination outlines attached to ES-401). NUREG-1021 12 of 27 Revision 8
1 ES-301 License Level ' Subcateaorv B.1 Subcateoorv B.2 191Al , RO 7 3 10 L SRO-instant'(i) 7 3 10 !. SRO-upgrade (U) 2 or 3 3 or 2 J5
$ kV
. The 10 systems and evolutions selected for RO and SRO-l; applicants should evaluate at least 7 different safety functions. All of thelyNms,ind evolutions in
- each subcategory of the test should be selected from 'diffe ordafety function 4 lists, and the same system or evolution shdbid not be us$$ $vaiuste more
. than one safety function in each subcate .
gf-j $%)g g% ff +- 3 The 5 systems and evolutions selected fdir0SR6U applicant s evaluate ' at least 5 dif% rent safety functions. One'5f$$iiibntrol room systemi9 safety feature, and the evolutions (nubcategory same system or evolution should not be O B.1) must be an @bd 'tE50aldi'le function. s y +"'," s% \%Y,e: m' hn:p' Keep in mind that the systems and evolu6cns selected for evaluation in Subcategories B.1 and B.2 nufatN5tiented$mtard ccInlrol room operations and iocal operations, respectivd[.' y .
*'
- Eg4M/
Q b. A For each system sel for av tion, seiset from the applicable K/A catalog or the facility licensee #$iite-sp liNic task lidone task for which a JPM exists or can be developed $9tiview tid %ssociated' simulator outline if it has already been prepared (refgthetion Yapdj aWIsid those tasks that have already been lected foreiisilustion on simulator test. l
- k. p rto k
- igtegrity and security of the examination process, no more
..~S . galpplicant's walk-through test (i.e,8 out of 10 or 4 out of 5 JPMs),0j,oroen, kapplicabl@Nnisj6"e taken directly from the facility's testing materials
. glidlhotAj$lhhkicant m'6dification. A significant modification means that at least one condiginitas, been substantively changed in a manner that alters the course of actionT test may @ated lie from
. p.the Additionally, no more than 30 percent last NRC licensing examination at the facility.
of the w
~
M^ At leastrane of the tasks shall be related to a shutdown or low-power condition,
, and onfor two of the tasks shall require the applicant to execute attemate paths j withl6ihe facility's operating procedures. facility)censees;may, increase;the JMygGMaltomate;pathp[and;NBC#mirers[may;changq 3fNsyrequim.;the;useggemate;pathsinocessarygimprove.their discnminatoryyalidityl* In addition, at least one of the tasks conducted in the l
' plant (i.e., Subcategory B.2) shall evaluate the applicant's ability to implement actions required during an emergency or abnormal condition, and another should require the applicant to enter the RCA. This provides an excellent opportunity for i
NUREG-1021 13 of 27 Revision 8
ES-301 the applicant to discuss or demonstrate the radiation control subjects described in Administrative Topic A.3. If it is not possible to develop or locate a suitable task /JPM for each of the selected systems, retum to Step (a), above, and select a different system or , evolution. After identifying a JPM for each system, list it and its assd5iated ! safety function number on Form ES-301-2, " Individual Walk-Through Test 1 Outline." Also indicate the type of JPM by entering the$$bbie code (s) { identified at the bottom of the form. 4) (({ MA //g j
- c. For each system and evolution selected ' evaluation fe applicable $
section in the K/A catalog and select two$ stem-spEific evaluated with prescripted questions. Tkpyted questions MI or geISi@ diversified among the different K/A categonesLassociated with ea evolution. List the selected K/A numbers [P NtM$ihbistance factors, i description of the topic on Form ES-30g2 ll. !
- d. Forward the completed walk-throughMoutline to'the NRbhief examiner so that it is received by the date aM$$with the NI%Nbional office at the time due at the least examination 60 days beforetifiearranger $iritiwbisN$nned;5iweview date R{
sched6I5dIissiNidi . ES-201 for additional inst $Ictions ra$rdinddsiGidw and submittal of examination outlines,44[ g/ W The NRC chief e rand In accordance,wi&I$S-201 '
- ponsibisifs porvisor shall review the GIny comments to the originator for 4 resolutio I 3k
"' LA$gjha N
, , miner approves the operating test outline, prepare the
_ Category is (i.e., the JPMs, questions, and answers) in guidance in Sechon D.1 and the JPM guidelines in When are complete, review the completed walk-through test for
. quality u . *in ES-301-3," Operating Test Quality Assurance Checklist," and
, make anhhanges that might be necessary. To minimize duplication, this review g shall beperformed in conjunction with the associated administrative topics and j
the sintulator operating test (refer to Sections D.2 and D.4). Je the entire operating test package to the designated facility reviewer or NRC chief examiner, as appropriate, for review and approval in accordance with Section E. The test must be received by the NRC chief examiner at least 30 days before the scheduled administrateen review;date, unless other arrangements have been made. NUREG-1021 14 of 27 Revision 8 i
c. ES-301
- 4. Specific Instructions for Cateaorv C. "intearated Plant Ooerations"
- a. Based on the anticipated crew compositions, determine the number of scenarios and scenario sets necessary to rotate each RO and SRO-l applicant into the lead reactor operator position so that he or she can perform a direct reactivity manipulation. For example, a crew consisting of two ROsgand one SRO-l will ,
normally require three scenarios to evaluate each applicaht'g pgormance on the reactor controls; however, a surrogate SRO wil[have to; fill tlie supervisory role while the SRO-l applicant is in the lead operator positionnSimilarly, the crews and scenarios will have to be planned the supervisory role for at least one see . f] 'so jgftiis6very[ d Jr SRO-U applicants are given credit for their previous RO license eva I6M 77 and experience and are normally not required to ihanipulate the controls.*' m mm
&>R l [45%
It may be possible to significantly reduce
- the number of simulator scenario sets required to examine a large group of applicants bygdministering the 'same set of scenarios on the same day to two (o(more) differentpl6f applicants.
However, provisions must be made 50 ensure that the' crews remain out of f scenailis (refer to ES-302). j contact until all f crews have YN50 cornpliiEdg@h%f Additional or replacement scenarios ad $)hould:$$e tM prepared and a administering the operating tests faccordaheii%ith ES-302 in case one of the , planned scenarios dnis$ot worEis intend $ii. !
- b. The simulator,o i tests sc and mod ng~ scenarios from%d7 @enanorsets) uilr ficility licensee will be constructed NRC scenario banks by selei and b ' Rnew sce'd!P
% yd
%[%Mhrderto ynifica integrity, every applicant shall be tested on at least one NEnelw@haarseor JErm practI5is!!Tsignificant n amodification east one mea s tha donditioriidreirent haII$een substantively changed to alter the course of action in
, the sceM$Mirthermore, any other scenarios that are extracted from the facility lic$rnee$liank must be altered to the degree necessary to prevent the t applicant [irisilmmediately recognizing the scenarios based on the initial conditiorpor other cues.
g c, .The jnbal conditions, normal operations, malfunctions, and major transients 7" be varied among the scenarios and should include startup, low power,
" ~
full-power situations. Review the associated walk-through outline if it has
.q Iready been prepared (refer to Section D.3), and take care not to duplicate operations that will be tested during the walk-through portion of the operating test.
- d. In order to maximize the quality and consistency of the operating tests, develop NUREG-1021 15 of 27 Revision 8
ES-301 new scenarios in accordance with the instructions in Appendix D. Modify existing scenarios, as necessary, to make them conform with the qualitative and quantitative attributes described in that appendix and enumerated on Form ES-301-4, " Simulator Scenario Quality Assurance Checklist." .The quantitative attribute target ranges that are specified on the form are not absolute limitations; some scenarios may be an excellent evaluation tool but may not fit Mthin the ranges. A scenario that does not fit into these ranges shall be evaluated to ensure that the level of difficulty is appropriatet Whenever possi$le, the critical tasks should be distributed so that each applii$nt is rebairedio respond. W Each scenario set must, at a minimum, emereeeerequire enten:to hkO' each appi c% respond to the types of evolutions, failursj.and trar6ii~ ents in the quantities . Idkntified for the applicant's license level]osorrn%S-301-5, "Trahili5i$si Event Checklist." An applicant should only require the applicant to perform verifiable ctionsMat. a~Mpn provide insight to the credit for thos applicant's competence. jinstrumentjandfx$5Ehfialluresjhat arolinitiated l
.after the major; transient phould;be;carsMiy reviEe83sF.ause;they;sometimes nfsi r.e..qu. .i.r..e.-'..l.itt.l..e..is. c.._hange hon. ~a.n..d...p.ro...v. i.d...e..'..l.it..t,le" i once per applicant; for exampi .c can ounted as a normal- 1 evolution OR as a reactivity lidii@
immediately results in a .. transiefil6@0inslailone^or the other, but notlj
/
Furthermore, each sceneno set rasst also&kWone$1ssilow the exa the applicant's perfosi$iEco ondiich compdfoney an6 rating factor germane to the applicant's lice 6iifflevel. Form Edi-%1-6, " Competencies Checklist," to verify that theg.optencie and event imhan that a .." de@io assess each competencyily evalu l
'facil. Tnormall/Mmtes)lth)hd Krequired bilts; technical
. .T6aE~morabah twoLROs:in'the1 control room;the! chief examiner g . _a b.d[uMMkhkogatesjtogou{MEMM[In%di~ ~ ,
ases,1filieicare)MWscenariospensure;that;the: additional _ operators '
./[fghMM*thelexaIniners'labilityjo;
. evaluate .eachjapplicant;on;thejrequired pumberroMiiH{and on:evey competencrand:rshng;facion O AppendikN pNvides detailed instructions for completing Form ES D-1, the "Scenariibutline," and Form ES-D-2, the expected " Operator Actions," that .
8 examiniifs will use to administer the simulator operating tests. In order to l mirindia the amount of rework that might be required as a result of changes in [p .
,. anned scenario events, Form ES-D-2 should be completed after the NRC 2 ief examiner has had the opportunity to review and comment on the proposed simulator operating test outhnes (i.e., Form ES-D 1) in accordance with ES-201.
- e. When the proposed simulator operating test outlines are complete, forward them to the NRC chief examiner so they are received by the date agreed upon with ,
NUREG-1021 16of27 Revision 8
..a-nsas.a -
l ES-301 l the NRC regional office at the time the examination arrangements were ! confirmed; the outlines are normally due at least 60 days before the scheduled
;;;rr.; .;Sn review date. Refer to ES-201 for additional instructions regarding l
. the review and submittal of the examination outlines. j A
The NRC chief examiner shall review the operating test outlines in accordance with ES-201, and forward any comments to the originatopor res6i$ lion. !
- f. .
A 3k! After final simulatorthe NRCby revising test materials chiefFormexaminer approves they ES-D-pas requested by the ff y J NRC chief examiner and completing a deGlIed operatorictiinf5dlES-D 2) for each event. All substantive operator actI5ts d(e.g.j hhning, cidsida5[g throttling valves; starting and stopping e ? raising and lo flow, and pressure; making d gl decisions an%yiEkdirections; cknowledging n alarms or verifying automatic actions) shallWdScdmented, and critical tasks
~
shall be identified. Events that do not reciUIreYr([$$$5Iof to take one or j substantive actions will not count toward the minimum number of events required ! for each operator per Form ES-30,153h f \;[Ql/ !
- g. Review the completed simulator
/M ?S/& %V a oper;^ ting test for quak using Form ES-301
" Simulator Scenario QualigAssurancie Checklist l' $id make any changes that i might be necessary. This.teview ~ sh'all be p'efformed in conjunction with the associated walk-thro est (refdto Sect "D.2 and D.3) to minimize duplication.' 8 ah 8 p Submit the enti ratingg%-peckege to the designated facility reviewer or ;
the NRC chi $f ner, as;aggspgpate, for review and approval in accordance l Sect i _
- befo%$g$]scl$duled ed. i..:.;. ;;;n rev%v w re160 test muWINs' received by the NRC chief examiner at least 30 ggg. -
N8$$h@ l E. QU E REVIEWS 1.
.. 3M lity Manaoementa$h a
;~
$fY If the operating INrt was prepared by the facility licensee, it4he; preliminary; outline and thuyroposedjedshall be independently reviewed and approved by;a knowledgeable
. superyigor'fninagerj:n ;t.'.;;;;;d f;;":j rgr;;;;;;;;;; (r;.fer t; CO 201) before it is
~
!are"sutidlitted to the NRC regional office for review and approvalin accordance E@$6T. The reviewer should evaluate f"te owemmotion outlinejandtest]u' s irdthe on Forms ES-20NAES-301-3, and E 1-301-4 and include the signed forms (for different operating test) in the examination package submitted to the NRC in accordance with ES-201.
- 2. NRC Examiner Review NUREG 1021 17 of 27 Revision 8
ES-301
- a. The NRC chief examiner shallindependently review each operating test for content, wording, operational validity, and level of difficulty. As a minimum, the
. chief examiner shall check the items listed on Forms ES-301-3 and ES-301-4, as applicable. The examiner should keep in mind that counting the number of scenario quantitative attributes is not always indicative of the scenario's level of difficulty. Although there are no definitive minimum or maximum atfN6ute values
. that can be used to identify scenarios that will not discrirdiste bdEIuse they are too easy or difficult, scenarios that fall outside the targeNsdje7specified on Form ES-301-4 should be carefully evaluate fidensur7% SNihappropriate. If the chief examiner wrote the operating te iO5other NRd all perfo Y the independent review. .
- b. The chief examiner should review the o .ths as soon as~hMENter receipt so that supervisory approval can 6Echtained before the fina7e5iew with the facility licensee, which is normally sche'Nuled athut two weeks before the administration date. It is especially imphntMiiii$hief examiner promptly review tests prepared by a facility licensee ^~
becaiTsF&Mextra time that may be required if extensive changes are . The ENIN355Einer shall consolidate the comments fro ^ ji$gional reviewdifs6d submit one set of comments to the author. NWs@#As / if Written 4xaminationi!am als6 bein myiewyejperatingMfirs[%g" se k%,Imapibe';advantageousL Examinations;caibe! mom easiymscheduledM , ;otherexammationsKsignificant,reworKis requiredj f
- c. df the facil deve rating test, P. ;m..rd p.irc.;.7 theri'the 4' $cilitt, ... . 7~' ;rilyireiiiIEi'sible for technical accuracy and compliarice' '
$iiillNye re . . . coming the use of examination banks. However, the nor to use his or her best judgment and take reasonable
,M1)ncludI5h a review of reference materials and past tests, to
- d. The ch _,
will note any changes that need to be made and forward the
. tests to t jfeponsible supervisor (or a designated altemate other than the chief examine f,or toview and oomment in accordance with Section E.3 before
. reviewi She examinations with the author or facility contact. There are no
- minimem or maximum limits on the number or scope of changes the chief p
exadhfier may direct the author or facility contact to make to the proposed tests,
, that they are necessary to make the tests conform with established a.. cceptance criteria.- Refer to ES-201 for additional guidance regarding NRC response to facility-developed examinations that are significantly deficient.
- e. Upon supervisory approval, and generally at least 14 days before the operating tests are scheduled to be given, the chief examiner will review the tests with the NUREG-1021 18 of 27 Revision 8
ES-301 facility licensee in accordance with ES-201. 1 Tests that were developed by the NRC shall be clean, properly formatted, and {
" ready-to-give" before they are reviewed with the facility licensee. The region shall not rely on the facility licensee to ensure that the tests are of acceptable quality to administer. 7
- f. After reviewing the tests with the facility licensee, the chpef examiner will ensure that any comments and recommendations are resolved $ndifitests are revised as necessary. If the facility licensee deve%d the testsfMhifisnerally be E expected to make whatever changes areffecommendeiS by the NRC.A
@ %QMyd"@es g.
After with the necessary the examination, he or she will changes ad[the chief e'xamineriiisiti sign F$h have been M{kES-301-3 and package to the responsible supervisor for fi6al approval. - 3. NRC Supervisorv Review fly nEER
$8s '
- a. Per ES 201, the responsible s ca. designated altomate other than the !'
NRC author or chief examinerf fidii" Mit approde#the operating tests before the facility prerevieufThe supIeNIAN j another detailed review,)$stsseissiment ratherggenerka%lk of test quality, not intend including a review ofAe changes % commended by the chief examiner, and a check to ensure thalMf the,ahlicable adninistrative requirements have been implemented. g g!!i# .#f
- b. hThe resporM$pervisorMI$kfEure that any significant deficiencies in the
""jSiginal oM)sts subn'ij$idify a facility licensee are evaluated in
.accordance%E%201 to determine the appropriate course of action. At.a
, the superwheshould ensure that they are addressed in the final UNaminason repoitlINiiidiniance with ES-501.
Nyy c, . Foll . review, the responsible supervisor should again review the tests to e, n,surel3Mit the concems expressed by the facility licensee and the chief examinerMNen appropriately addressed. The supervisor shall not sign y Form (s) ES-301-3 until he or she is satisfied that the examination is acceptable g,5 ff to be addlinistered. FC?.MS FogESqF1, . " Administrative Topics Outline" FomiES401-2, " Individual Walk-Through Test Outline" 1 Form ES-301-3, " Operating Test Quality Assurance Checklist" q Form ES-301-4, " Simulator Scenario Quality Assurance Checklist" ; Form ES-301-5, ' Transient and Event Checklist" - 1 NUREG-1021 ig of 27 Revision 8 '
y i .. r ES-301 Form ES-301-6, " Competencies Checklist" A
, ,' /
f >>
/ 3
), ' s/ s's' ,e A
AR e' r ,e' !'
< s
,,/y' r>A , s ,< , , , w , , , .
n +
- > ;e
'J ~r ' ' ,' , r "y l'^S,,' ,
? m' v s i
/i /$' ,
- 9,<.,,, : n,, >" ,, '
.e d <>
y t;, > . '
# ..Is
,. u
,s f !f g, \
. r~et q,,e" > ,/
y : '~ i < V s, v% s-"A >
>s
'+
s,
,,v@ %ps,, ,9 p> <,s . ,
$,",$' , , s,
,,f ', 4 ' Is :
.+am,
,s s
- x s,.sa ,v in>,
g,s ^ ,sk # s s
/s >, f 4N(,
, ps*a n n, , ,,'.$ ,%< < b,p,.c%,, d' +
s
,,[# ,
$"j' f5 5 5"'
p,g)j'J 2,,,sy ,e 3 fr +P ^ g u j'r ,4, f> g# 4 4 ++, < b e /
,n p,
., ,#g @, ..
h<,s ys
>q'
! 98/p o g s,% $ ,e> > ' ' '
f,4 ,;., %, ,4 t , , ' r ~,:, i' . ,;m s A > p; ). (Nn,>
,; - ts- > %, g ,s, e,,< ep.
; m,
> >y, ~o3 ;sv., -
,E Ph,><
>'s, %) t N 'd ': ,')
~
s e v, . v;~sg gs a,,,
/pe% ,c:,j, e ,>,,
ir..p, ' -
' ' s10l.;",
< .5 ",
/,a
$t .>, ,i, y r,p ,
@/gy f
?
4 4' >,
.nr su<+s:g se,g
;ge
<a y
Mk. y ht 4. N+;bkh~ Aw er U
;p;UN'g[fhIk wg; 1lge NUREG-1021 20 of 27 Revision 8
ES-301 Administrative Topics Outhne Form ES-301-1 Facility: Date of Examination: Examination Level (circle one): RO/SRO Operating Test Number: Administrative p. Describe method of evaluation: - Ap'
- Topic / Subject 1. ONE Administrative JPM, OR ltA Y
Description 2. TWO Administrative Questions
#m/h% 29 A / '
A.1 vy ' , ,, AD A /' f, +'V ^ ~ ': 1,
+ ,t 3r fy" }?
hh f';! h; edt 3, <
' ~> -3'<?d'
(,,< v s ggy
%is ( , ,
, p.?% ,~ .
..x
% ' N g()%h p w,hs f ;"N i 's~' 5 ,
c w we:.ggg g.g.. yo
, ;e % mgm,s:% v
$sy w gx p , w g)y ,
c.r > - eu t . <
- (
4.f , :t d ...,:y f x $':. p < A.2 >a&y+ ~,fia y; n 9[. ' '
%wkp#!I.r,e m- %
W.> %. c h.g
- .2 405 sev%a,e ,
4.![jja s,. ,, ;g A.3 j:h j , ,, g % 41:;y: prf c y 4 ' % ' ::' %s NgP
>s ,>
.b ' l, !
$$l.4
- X fsR $
/.: id:$, _ . 'f $'
Bhi. M; i i :::d49 6J# s 5
>c ps .# ' AA -
s ,, . y~,ng 4 /
< < ;. gY NUREG-1021 21 of 27 Revision 8
ES-301 Individual Walk-Through Test Outhne Form ES-301-2 Facility: Date of Examination: Exam Level (circle one): RO / SRO(l) / SRO(U) Operating Test No.:
,+.
System / JPM Title / Type Codes
- Safety Planned Follo,w-up Questions:
Function K/A/G - Importance - Description
- 1. A &~..'
- a. jg g ggg
- b. kh )k ?hNs ph
- 2. a. pg',
s w j,/ ur , m , c.,
~
"(.;: <
" m":mpr
"^
- b. yp~mm
*Q &;7
- 3. a.A[T j6!A kA %C %,w2%e
- 4. ja ,s- . Q f pnye .- ,
!K 9pd %,I S.
~
s g' a. gy w
,a j' e l{:
7 9 fu <' bib $ b abo.+-
- 6. xg y hw.]' : ,l 5r kgW;pm egt
.c s ,o . w. a is.
+
% ,;;.' r; A s
y , a q ,, b.
% A- -
- 7. \~ ^.'ssA X3' ,
,,' a. i
\l:di% *% h :}^
,qt'::.:::!;< '~ L n,, '% c' b.
e vYr"p#n,e..Alyng a. 8./)r
, b. !4 93 hi?St a.
j Mer . i ig!!wa%$g , , , b. j ho};j:ll'*y./ wam* a. b. l
- Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)ltemate path, (C)ontrol room, (S)imulator, (L)ow-Power, (P)lant, (R)CA l i
1 NUREG-1021 22 of 27 Revision 8 1 1
)
1
ES-301 Operatina Test Quality Assurance Checklist Form ES-301-3 [ Footiny: Date of Examinsten: Operating Test Number-Initals
- 1. GENERAL CRITERIA e b c'
- e. The operahng test conforms with the previously approved outime; changes are consisterd with esmphng requirements (e g.,10 CFR 55 46, operstonalimportance. safety functon dehhuten). 45
/b
- b. Repetsen from operetag tests used during previous licenomg exemmeliene is within hmits (30% lor the wek-through) and should not compromise test intogey.4. *WW nA
.nmy a- n. g c.
. Them is'no stoy40<x=
disegedeeneminosearp 1. , ley ug = = -- repetiten between 9Q@ v n .n-gthisgand other pr [a operat
- d. Overtep weh the written enemmaton and between operating teeIdelegenes[shhin scooptable kh hmits. t#bmemt. dk Siw [
e.!
- w. eve
. It appears that the operahng test wiH dWorenhete between competerdaad _ .=- % _ m , apphoents et the "- _ z heense level. 47%MWW
- 2. WALK THROUGH (CATEGORY A & B) CbTERIA N % - -- -
- e. Each JPM includes the fotowmg, es appicable. $k inthal conditions . .,p&*M[kf!IkW;q .
a inibeting cues g@ [w references and tools, includmg esseelsted procedures % vehdeted time limits (everage lege'ellowed for sempleton)Q designeten if .' " deemed to be tone creical by thefeclaty boonses h epoolho performones cettermget bolude: ,# h
- detsted expected accons wlptseest f.teorie sahnomenoletumf ,
- system response end otheraisNGnar cue $f j;g )
- eastements deserttung impostent observemme to to modeliy the opphoent '
- cettene for successful completon of the W% .## !
, identNiceton of cribeel etsps and their necedileepeelsynence atendards
~
1 Aestnchons on the eesessee of steps. rausseelse w emwanm *p
- b. <p;w wiAt least 70 percent of the fossegend (AdministribmN gyand JPM fotow-up) questens are .
j
*psesteethieseyopen referspetandetest the criterte in Appendk C. There are no direct look-up
=
1 y level Tmndtaermt the use of references ww;om.h .
%>mMg I ed. At least 203elesteeithe JPMs entgeodbes on each test are new or signMicently modmed
#NkYekbLA70hTE00RYC) CRITERIA - - -
a. ws@wsw . eseociated simaisterspesallag tests (econerlo sets) have been reviewed in ecoordance with -
' Form ES 3014 end's essyAssteched.
Y Pnnled Nome / Signatum Date ' Revower(*) k3% Aw ; 1 018eI5eulgleer(*) l t jep i m e-(*) The incluty signature is not =We for NRC developed tests: two independent NRC reviews are required ES-301 Simulator Scenario Quality Assurance Checklist Form ES-301-4 i i NUREG-1021 23 of 27 Revision B 3 , 1
Facility: Date of Exam- Scenario Nxterst / / Operating Test No. QUALITATIVE ATTRIBUTES Initials a b e
- ::::.:m:. 2;;; :':::' n:w' K-: ~. '
. a. : . a :- - .^ ~
- 1. The initial conditions are realistic in that see equipment and/or instroentation &
may be out of service, but it does not cae the operators into expected events. Y
- 2. The scenarios consist mostly of related eventt $k< Ak
- 3. Each event description consists of the point in the scenario when it is to be initiated #',
/h: 4gl;QgpA. k g the malfunction (s) that are entered to initiateJthe event OO AF %g? T [6 the symptes/ cues that will be visible to the cr w i" the expected operator actions (by shift position) the event termination point (if applicable) H M #!!
d~ %h
%@ g &
4 W ~ n No more than one non-mechanistic failure (e.g., pipe breek)3,s.srincorporated into the SM scenario without a credible preceding incident such as a seismic < event. z< s s. :in
- 5. The events are valid with regard to physics and thu muu,. ..tes N e kih 6.
w v.:., . x m.s. Sequencing and timing of events is reasonable, and all ws the examination; team to obtain complete evaluation results comensurate with>the scenario objectiVesT ~ A m 44s:.:. v:sg.sv
- 7. If time compression techniques are used, thecscenario suunary clearly $0pindicates.
Operators have sufficient time to carry outlempected activities withoutwndue time constraints. Cues are given. AF WW ' A #
- 8. The simulator modeling is not alteredk AI NNN Y
#g p s.
- 9. The scenarios have been validated #w 4# s 10.
~ . . > m Every operator will be evaluefid5ufing at Jesst one newe,ssrc s1.gnificantly modified scenario. All other scenariostave been. modified in accordance with Section 0.4 of ES 301. W ANM AW n m.ye t.
- 11. A11[A)ndividual operator 1 cog etencies can; M . w. Mbeievalusted. as verified using Forn ES-301-f Asutait 5the form along'with*the simulator!:: scenarios),
w.. agem.,, wa py
- 12. Each? applicant will bi*sig,,nificantly involved in the minima neber of transients and D
eventsEspecified on Form E$i301rst(submit the form with the simulator scenarios).
' h ;.44%3Mhk
- 13. The levepofcd'4fficulty is a$m.CWn!E:@pprylate to support licensing decisions for each crew positiersF #"WWA NN a.a - u, .w. .,.. , wa, TARGETTUANTITATIVE 'ATTR1mnE5"{PER SCENARI0rSEE 3ECTION D.'4;d) Actual Attributes -- -- --
gw W,n por
- 1. N Total malfunctionsM5-8)P / /
2r$35 V* V
&- Malfunctions afterTOP entry (1-2)
/ /
n 13Kg 4:g: Abnormal events 42-4) / /
$h b Major transt N (1-2) / /
w.g n & p<g..W
?$?PtM(Wentered/ requiring substantive actions (12) / /
wa oW 6Mi MdEOP tontingencies reauiring substantive actions (0-2) / /
- 7. Critical tasks (2 3) / /
ES-301 Transient and Event Checklist Form ES-301-5 OPERATING TEST NO.: NUREG-1021 24 of 27 Revision 8
7 -- 1 l' Applicant Evolution Minimum Scenario Number Type Type Number l 1 2 3 4 L e Reactivity 1 Normal 1 A RO - Instrument 2 . A c' in : Component 2 Mh &e M6 ' Major 1 /N A bAhr k& f j,f Reactivity 1 hb 8. / MkM ' y . 34 wm Normal 0 4 MW5% As 0 Instrument 1 I Nhb b$E6h.
' Component 1 Nhh bb I Major Abbb!f- A !ff b SRO-I M INNkN!INed Reactivitf ek MIh No kh N'1 '
As SRO ewr wa InstFument /! ?Mi 1 I. Fs A
##,> Component byMakg4*
V7 gg smani 4$ ;qu% %,
~w~
< ,,, w pgtM4 *E' flajor 1 w.mgwmm sm s a,
, % ~ it. p,gjgg> 0
., . g we
() ~yh sees, Normal 1 t* 'n .or SRO-U j a% instrument 1
' 9 Component 1 i
j ~ .' . Major i d l'5 '^ tons: (1) Enter the operating test number and Form ES-D-1 event umbers for each vo ution t w+s krWe.d *. (2).
=: keactivity manipu$at on:; mus(pe.be significant as defined in Appendix D.
[kxaminer: NUREG-1021 25 of 27 Revision 8
I l i l j ES-301 Competencies Checkhst Form ES-301-6 Applicant #1 Applicant #2 Applicant #3 R0/SRO-!/SRO-U R0/SRO-I/SRO-U R0/SRO-1/SRO-U Competencies SCENARIO SCENARIO ICENARIO
#y+k a
1 2 3 4 1 2 3 4. .de 2 3 4 Understand and Interpret f ~he f se , Annunciators and Alarms di N - m 4 m p w~ ~, Diagnose Events W ..
- W V and Conditions
~*
d#
, 't u. ,,
.3 Understand Plant Nk !$$
- w~ +g ,
and System Response # "h v ae l Comply With and " W %r Use Procedures (1) s 8 h N5 m Operate Control Boards (2) I ffP i (#. % W
/f @
( g i Comunicate and ,[ / Y @
}M .t; n
Interact With the Crew x4E /c n g m m Demonstrate Supervisory / P ,, f* Ability (3). mu Ak [ :$~ d#j
, ., n <
Comply 41the a k % ,;us - s wr Use Tecfr65pe 4,(3) csee s
% J' 9h l Notes: f gg \$fhh (1)
Aff TechnicNISpec Inclubes$Ibhkhk;h ification compliance for an R0. (2) ,0$$onal for anM$j((? (3)4 nly applicable *tI$it0s. og eg. I uctions: [nwg!ik JEMIli gc}egjpp}pnt'slicensetypeandenterthe-onetor; pore [.eventnumbersthat ti@dddipu.V f& cGh SG.~.wie m thc set. MWaljowsthejexaminerssto (Vakuate;.everf,ljtompetencf0forfeierjMppl{canti Author: Chief Examiner: NUREG-1021 26 of 27 Revision 8 J
b h ES 302 M ADMINISTERING OPERATING TESTS TO INITIAL LICENSE APPLICANTS A. PURPOSE This standard describes how to administer operating tests to initial license applicants in
' accordance with the requirements of 10 CFR 55.45. It includes policies and guidelines for administering both the walk-through and the integrated plant operations categories.of the operating test. it is assumed that the operating test was prepared in accordanciMth ES-301.
(f
' As noted in ES-201, facility licensees will generally prepare proposed operatiEig Iedtiin gp accordance with ES-301 and submit them to the resporsible NRCfiiIgional officOENN5nd approval. Regardless of whether it was prepared by thk% $iensee or the N'RCMAry
. operating test will be independently administered and graded try;jen. NRC licensin accordance with the instructions contained herein and in C. RESPONSIBILITIES ~
ff
- 1. Facihty Licensee -
The facility licensee is responsible rthe , .
- a. Make the plant and on f avail necessary, for validating and administering Cateh5 (cont l6iroom andln-plant systems) and Category C
, (integrated plant operations)pfthe opepdang tests,
- b. Safeguard Mkrity and'h3d the operating tests in accordance with lES-201. D#
- c. ' ' ::admini ., jogistics support (e.g., personnel to operate the
- s. lity, e operators, etc.) to facilitate the administration of the in a ance with Section D.
.- Inform t ,. 'ional office in writing if an applicant withdraws from the i examinat ss before % is c:::r.piete.
AI. N 2/83 NRC Regional Gifice { . NRC nel office is responsible for the following activities: Work with the facility contact to coordinate the operating test administration schedule in a manner that maximizes efficiency and maintains security (refer to ES-201). IfhiroessaipK1he region [mapyela9 the;6pairating:testsfor[up to:30 days without OLB'approvaWHowever[the' region ishguld, keeppl.B Mformed..and consider notifyhgithepcility,;licenseegwriting~lf:the: delay,is based.on:th.e NUREG-1021 1 of g Revision 8 l.
ES-302 quayof the proposed _ examination.
- b. Administer the operating tests in accordance with Section D.
D. TEST ADMINISTRATION INSTRUCTIONS AND POLICIES 1.' General jfa pA/m
#$kd A
a.- Before beginning the operating test, an examme.r shall tA$ef the applicant (s) using Parts A, C, D, and E of Appendix EiT8Aave time (KE% commended thatf the examiner (s) brief the applicants as a up. F N f7
- b. If an applicant requests to withdraw duriqg anypert of the examina'tionprocess, the examiner shall inform the applicant that this i siilli result in automatI5 license denial and that he or she may reapply in adsonlin$itwith 10 CFR 55.35. The chief examiner will request the facility liosi5eeT$I6Enent the applicant's withdrawal in a letter to the NRC regional administratorM%
- c. Each applicant listed onAthe examir$a$k tion' 3E!)Y assignment sheet (see ES-201, Attachment 4) shall be adminisi&Nd $hioperating testdindicated under
" Examination Type."
]
- d. For purposes of test, integration contin , the chief examiner sheuld generally schedules Mme ner to ~ nister all three operating test categories to an ' 'cant. .H5Miever, certain circumstances, such as when a licensee' isimulation%IcBqyhiinilocated near the plant or if a licensee I g
[jk[hrequests exsiminMions for alihnEssMlly large group o N!Gimperating teiit '
~
and subcategories among different examiners Nilii6Idstor;oper7$il . s, l among examiners igHe[$^ f examiner.g willc[ be multiple responsible scenarios:shall for ensuring that not be: d
^^ gets acomplete operating test and that the tests are thoroughly
'and a Mocumented.
- . xla b8'Nh"ikin ['.. .N.. consists, entirtlyg,Senioryclor operatK c pgf'. WyplicantsJggnotpave;to;be;evaluatedjon;;theicontrol; boards)Lthe 2 chief examinerwassignMMexaminersNobserve;the:crowgAlthouCh p M ja M'Mthejreactor, operator [and.bsiance;of. plant positions.;may;inot;.be indi%yguatedlMW[be; graded and. held.:eccountablegangerfors Woocutasyresultief itwitaction(syorgnaction(sp
- The exarniner is expected to administer the planned operating test in accordance with the prepared and approved walk-through test outlines (Forms ES-301-1,
" Administrative Topics outline," and ES-301-2, " Individual Walk-Through Test Outline") and simulator scenarios (Forms ES-D-1, " Scenario Outline," and ES-D-NUREG-1021 2 of 9 Revision 8
y ES-302 q l 2," Operator Actions"). Examiners shall document every significant aspect of I each applicant's performance for later evaluation, but they shall not use the applicant's unplanned actions and statements to displace any part of the planned operating test. Normally, examiners should substitute or replace planned,1 operating Est materials only if it is determined that an item is invalid orknpossi6le to perform or simulate because of unanticipated access restrictions orIeq$lpi6ent failures. j i gg7-b WM p
- f. Examiners may administer the same oper g test (walk-ihrou%%.gN and simulator to consecutive applicants and crews on this, ame day [6ut they.must ensure that the security of the operating test is mainNed. Thdiame simdlAt6dcorsarin shall not be repeated during successive ba %jf if previously agreednsee; examinersby 5 dupon the facility lice may also administer the same operating test (walk-through andlinidisidrj$ dividing the test into segments that can be administered to alfof the alskhcards^on the same day.
This will minimize the amount of effortTsijuired to dEseloihifferent operating tests but will complicate the s proiess. N
//
m# m m <A
- g. The examiner should normally admir)iirker,Catog B and C of the operating test first and attempt to o66 currently'evalua'tias ininy of the planned administrative subjects in CateggA as pojsib'~le. The remaining administrative subjects should theribEievaluated in accordince with the approved outline.
- h. The examinermu e su t notis facilitate the thorough documentation of any and fillphscant defihlsnciei$accordance with ES-303. The examiner
/[Op$korq2As%$g 3h ihust be a'tSilo5cfdss-refere"rNE55h comment to a specific JPM, simulator
- l. %%%s, ThsMi king of vid%W%
Iduring the administration of operating tests is not
- V w &
47 The nu ~ rsons present during an operating test should be limited to ensure thAf of the test and to minimize distractions to the applicants.
/'
w 3 Except for the simulation facility operators, no other member of the 1* cility's staff shall be allowed to observe an operating test without the 0- chief examiner's permission. Facility management and other personnel N!7 deemed necessary by the facility licensee should generally be allowed
% ,sm. y access to the examination (under security agreements, as appropriate),
provided the simulation facility can accommodate them and there is no impact on the applicants. Although the simulation facility operator will normally assume the role of NUREG-1021 3 of 9 Revision 8
ES-302 the other personnel that the applicants direct or notify regarding plant operations, the chief examiner may permit other members of the facility training or operations staff,(e.g.',t a shift technical l advisor (STA)) to augment the operating shift team if necessary. The chief examiner shall fully brief those individuals regarding their responsibilities, reporting requirements, duties, and level of participation befgre the operating test begins. The examiners must not restrict the surrogate operators' activities to such an extent that the appli. cants bding evaluated are required to assume responsibilities , The surrogate operators will be exp%did ected to assurnehis fulla s withren"STA shall be" conducted iE5ccordanFiwith thelidi$ilii:iE^n~sslifs norma [ control toomEactice@dlSTAsM%beMtati5njs~dtiE ~ simulator.Kthey are on-call,in the - . 2k
. if the facility licensee normally operates' hh Sid is required by its technical specifications to have r6 ore thaEtMIMhtor operators (ROs) in Mutt $ die the use of the control additional room, surrogates OI:&theInchiifisaminerJniaI[hilnses, togEFiIrews.~ s e take care that the presence ofaddihe$al operators does not dilute the )
examiners' ability g$ialuateydfM'd0 ring the: required. number ofeMalindgevery cogipetencydhding factor. Examiners shall not hesitate to1run additional scenarios, as necessary, to ensure that lica$ill giverdie opportu$ity to demonstrate his or her every competen app $5#0nly;onEhiividust}ipplicant orssurrogate) is; allowed, a shift su .or msnager po.sition dunng_the simulator operating test; U c t Nd!Y mstancesMI another applicant be allowed to witness an 3.g
, ope'rEnk$id)pperating tests are not to be used as training vehicles for
~
E ' ure applicantshN@g 3
.examinets'may observe an operating test 'as part of their training or
% lthe performance of the examiner administering the operating test.
, ' examiner may permit other NRC employees, such as resident i ctors, regional personnel, researchers, or NRC supervisors, to
/ 8 [ rye an operating test. Personnel who are not NRC employees (e.g.,
presentatives from the Institute of Nuclear Power Operations (INPO)) may observe the operating tests with prior approval from the Chief, OLB. ! The chief examiner will control the observer's activities in accordance ; with guidance provided by OLB. The examiner should also give the j applicant the opportunity to object to the presence of observers. ;
- k. The chief examiner should confirm with the facility licensee that the simulator instructor's station, programmers' tools, and extemal interconnections do not 1
i NUREG-1021 4 of g Revision 8 l
, 7 i
C__
- w. u ES-302
' compromise operating test security while conducting examinations. The primary objective is to ensure that the exam material cannot be read or recorded at other unsecured consoles and that examination material is either physically secured or electronically protected when not in use by individuals listed on the security agreement.
.g 1.
w kY The chief examiner should arrange for any NRC examiners'who. ore not familiar with the facility to obtain a tour before they administer ar$YMiing tests. The of the*igshlicisirits. In addition,
' the tours shallconcentrate tours should not beonconducted areas of thei or observed by$ plant that w examination process, such as the controli6om,Fthe sir 6Uiati5dissilt jand gf planned walk-through locations.
- m. The chief examiner will conduct an exit briping with the facility licensee after the operating tests are complete. The briefing should$ddress any generic weaknesses noted during the operating tsit'slindMIother significant issues
- (e.g., problems with the reference material, the simulasidnifacility, or the plant) that might be addressed in the exam l6Etibn report. 'Tidindividual operating test results are predecisional until appiIr$Nd" ~ C manA6sEAnt in accordance with ES-501 and shall not be shassid GiiEitiEA licensafduring the exit briefing.
y g4
- 2. WelbThrouah (Cateoories A ands)
- a. The examiner shouldMdate arpy JPMs t 'were not previously validated by the facility licensee o $fihe NRCisluring agaratory site visit. This is particularly xJPMs lthose)ttiat require the applicant to implement an important omstiv for,co ' , directed procedures.
wgggags 4: tlw extenihxthe examiner should have the applicant perform the ioSnlistroom JPMsMe simulator, rather than asking the applicant to describe hiiMsisMshe woIlidsi5diiMplish the task. The . Y r is expected to coordinate the administration of the JPMs to maximize %iiasisf the simulator. To increase efficiency, different JPMs may be adminfilSied5imultaneously to multiple applicants, but the examiners must ensure thiif m*utualinterference is minimized and test integrity is not comprodlied.- When Ms or discussions are conducted in the control room, the examiners i make every effort to accommodate and not interfere with normal shift E operations. ' The chief examiner should request that the facin! training manager notify the shift supervisor when the NRC will be conducting examination activities i in the control room. if the number of persons or the noise level in the control room is excessive, the examiner should, if possible, move to a quieter location, modify the sequence of the JPMs and retum when the level of activity in the i
' NUREG-1021 5 of 9 Revision 8 '
1
ES-302 ~ control room has abated, or ask the facility training manager to address the issue. 1. l i
- c. ' The examiner should encourage the applicant to sketch diagrams, flow paths, or i other illustrations to aid in answering the examiner's questiens. In all. cases, the
. examiner shall collect the supporting material because it provides a62fitional
^ documentation to support a pass or fail decision (refer to ES-303)?To facilitate copying, the applicant's drawings should be restricted to(hirikslSe of separate sheets of 8.5-inch by 11-inch paper; the bagForm E .
orits attachments shall not be used for this purpose. ' 4Y The examiner should encourage the applNant to use 'such maternal as
. .! b T 4
- d. l forms, schedules, and procedures if they nt to the quesiiQaisk$d.
- e. The examiner should be careful not to infedM. system and administrative I knowledge from observations made durghiQQgThe examiner should keep in mind that the applicant's proficiency in every;MQtrative topic and ,
' accordance each control with theroom and operating .. in-plant sy@steiifsiiipuld be de$inisilfeva 3 301. pyV '
- f. The examiner should ashunplanni llow-ggtions only as necessary to clarify or confirm th cant's understan of a preplanned task or prescripted questi :. Y y
- 3. Simulator Test (Categg .C) b iBefore a s n- $theb5 ^'
te examiners will validate each scenario on the tor to'Mkit will run as intended. Scenarios that were adapted kious N$iinamigations at the facility or from the facility licensee's bank ngsgisre alidation. At a minimum, the examiners will " dry run"
} .aving
/tiidaii@ quire re%*ij@ia)ble inpu
.s remas will run ,
cenario with the facility's simulator instructor to ensure that it in some pes, the scenarios can be validated while the applicants are taking the
'i ' written examination. However, it may be beneficial to validate the scenarios
~
duringkhreparatory site visit as determined by NRC regional management (refer t $201). The examiners will take precautions to prevent the scenarios from being revealed to the applicants before the tests begin. If significant portions of the scenarios are dry sun or otherwise reviewed with the simulator instructor (s), the chief examiner shall ask the instructor (s) to sign a security agree;nent (Form ES-201-3) to protect the integrity of the simulator test. I NUREG-1021 6 of 9 Revision 8
U i l l l ES-302
)
' c. - The examiners should revise all copies of Forms ES-D-1 and ES-D 2 to reflect .
? any changes made to the scenario events or the expected operator actions as a !
result of the scenario validation runs and reviews. These revisions should be neatly written in ink so that the forms can be used in the final write-up of the j simulator test, as discussed in ES-303. ' d .' The examiners should review the scenarios together andbiscuss;the required ff procedures, technical specifications, special circumstareis?aldso forth, related e. to thuanans. pv Immediately before beginning the simulator tests, the examiners y tor andj5 vide hiriiM6eOWEE the scenario' events with the simulator oh[Mi the operato copy of Form EG-D 1. This review should sequence of events to ensure that they willfroceed as planned. Thifis the test is not the same particularly important if the simulator op[erator, individual who assisted in validating the ndr16s
.n y^
f.
-tse monitored during The examiners each simulator scenario. should identify The c$miiuhMer important should $slij plant paramep,iis sim i s
to record selected parameters 7# poindi[Idihe facilijyls safety parameter display system (s). Params57 readir intervals, depending onr, ttEparam$giE6615$ti6Ilected the' at me 2 of the simulation facility $$The chielexamine@rAhould retain backup documentah$ augndit the notditaken by the examiners during the simulator test. 7
- g. The exami ' . . arge of rio should arrange a suitable
[50mmun16iE5Ei%) stem with'ttidsf5blator operator so that he or she can be sn$npted toMMineatfunctions without cuing the' applicants.- Malfunctions
% p neif
- termined time or power level so that the examiners f aware of the event that is occurring or pending.
Ifno , ' 4xaminers may use time compression to speed up the respons meters so that the scenario can proceed to the next event
~
within a le time. Time compression is acceptable as long as it is used judicioustMm*dthe operators are given sufficient time to perform the tasks that they would typically perform in real time. If the examiners intend to use time com test sefing
$pr$,(refer Elon, they should inform the applicants of that fact to Section D.1.a). The examiners should also mitigate the during the Q
1 ial for negative training by debriefing the applicants after any scenario in ich time compression was used.
- h. Before beginning each scenario set, the examiners should have the simulator operator advance any control room strip chart recorders thairnay prove 'useful in recreating the sequence of events. The charts should be clearly marked with the NUREG-1021 7 of g Revision 8 i
y ES-302 date, time, and examiners initials so that they can be accurately matched with the correct operating crew.
- l. - The chief examiner should ensure that the simulator operator (or examiner) playing the role of other plant personnelis aware of the time scale for responding to the applicants' requests for information. For example, fast-time doCid be specified for auxiliary operator checks or lineups to preventjong d6 lays in simulated operations, while maintenance and chemistry [ sample;lnformation can be provided with normal time delays to prese$t.the applicants with the same analysis problems that they will face as o $rs, j gf[f ((%;w$$N J. Before the simulator test begins, the ex shallIaution th .simula$M$ =
operator to provide only information that%IpecdiEiilly requested rikssijf applicants;and.doesinotjcompromise;thelgekof the;examinat. ion. 'When the simulator operator is briefing the applicants:orcommunicating with them on the - telephone, the examiners should monitorffe Eir$55aition to ensure that the information provided is appropriate and does not'60iUM3pplicants.
. k. Each examiner should use th a lbM ns and behaviors listed on Form ES-D2 as a guide while ad terEEEEE$iEIr$1stortesti. If an applicant performs as expected, the 5 miner %
the form the timeswhen th[ expected actio%%In nsiccourred. However,if the left-hand an applicant co does not perform as ^ '~cted, tiditxamine%uld note the applicant's actions (or lack of actions io, or fter%wthe e thesimulator'Ncted action appropriate ques . .a scenario is completed (refer and follow up with to Section ]
. D.3.1).
ex .dete best way to document the applicant's
, Ab60ns. So "' M" rs record a minute-by-minute account of all key plant
~
(%land the ctions as they occur; other examiners only record the
.signifi s. Each individual examiner should develop his or gdiininatioli mentation technique; however, the documentation
-~ a .
examine , ust provide sufficient information to allow the examiner to confident ' the applicant's performance on the competencies described in Append .
- l. Examiners shall limit discussions with the applictnts during the scenarios both to masMih realism and to avoid distracting the applicants from operating the plant.
examiners' questions during the scenarios should be limited to those that tre necessary to assess the applicants' understanding of plant conditions and the required operator actions. Whenever possible, the examiner shall defer questioning the applicant until a time when the applicant is not operating or closely monitoring the plant (preferably after the simulator has been placed in
" freeze"). The examiners follow-up questions or concems can generally be NUREG-1021 8 of 9 Revision 8
I ES-302 addressed during a brief question and answer period after each scenario or during the control room systems and facility walk-through portion of the operating test (i.e., Category B) if it is performed after the simulator test.
- m. The examiners who administer the simulator test shall confer immediately after I completing the scenario set to compare notes and to verify that eacF5xaminer -{
observed his or her applicant perform the required numberJof trarisients and
. j events in a manner sufficient to justify an evaluation of All thelsequired competencies. ' If necessary, the examinerssl511 run a#rdddilishal scenario to ensure that the required evolutions and co%tencies will be planned and documented in accordance with Si;ari'bovere ction'D.of ES;301. a hk Ah hhbb'f
- The chief examiner shall ensure that the; examiners' observations are consI tent and that their findings are mutually supp5tiveMfy performance defi5IEncy is
" shared" by more than one applicant, it shdbid;behoted by both evaluating examiners. Ideally, this cross-check sho615 b'5 Soociihhshed as soon as possible after running the scenarios whije still atM%gpe cross-check must be accomplished before finalizin6~ine examindtidiiresults in accordance with ES-303. Tie 1MAb kh
%F
- n. If the applicants did not hA sd#
rm as expectedp examiner shall ask the simulator operator to proAde coplei6f the ighirts, and other materials that may be required afteg ing thgcility to pyaluate and document the - l applicants' performan%ce! The examiner ofiscord shall retain al {; ' related to any operatirIg test f$110re until[ proposed denial becomes final or a l , license is issued , g'ggg/ chief
'f~
- m. should as e simulator operator to retain copies of the i . dmate'r$ ale %isil applicants 3 are licensed or all appeals are settled, as d in itissenqpid?
- q. corporate notification letter shown in ES-201,
- 3. 4MCP
'~ o, , ifthe inci hould become inoperable and cause excessive delay of
'~
the ope %$7 the chief examiner should discuss the situation with the
; facility licensee *nd the responsible regionsi stipervisor so that management can
.;1 - make a de$isi*on regarding the conduct of the operating tests. It may be i
[ necess to reschedule the simulator examinations for a later date.
'm
- l
\
- NUREG-1021 - 9 of g Revision 8
f' ES-303 DOCUMENTING AND GRADING INITIAL OPERATING TESTS
. ' A. ' PURPOSE
. This stand' ard describes the procedures for documenting all categories of the operating test, collating the data to arrive at a pass or fail recommendation, and reviewing the documentation to ensure quality,
(( B. BACKGROUND
$9 9:
This standard assumes that the operating test was prep _ and ad accordance[ with ES-301 and ES-302, respectively. In addition, the procedures..c,ntainel o herein require the examiner to evaluate each applicant's performance on tiiioperatdtest and rnake'a% o V
. Judgement as to whether the applicant's level of knowle . landTsi1derstanding meophe#
minimum requirements to safely operate the facility for ' .Miicense is sought. The examiner should evaluate each noted deficiency in light o ability demonstrated by the applicant in that subject are@[theitotalbreac:h of knowledg When used to evaluate an applicant's performanghpr part of the; operating test as discussed herein, the terms " satisfactory" and K - - ~ F " are defirjinil as follows:
- S -Satisfactory Working Knowle and U The applicant may have som$ ht or r diffi describing system interactions.
Competence in the operati with the system is very good,- although there may , besita be~ while s_ kINfoquipgnt discuss asso@ing or performing some applicant appears tohe . liarwitti nt and procedures. A factory 2or Working edge and Understanding Tj%
.The ,,, s diffi Mf[ igg ; g questions in depth and describing the intera of systemsf _ sions , .
ior while operating equipment show lack of familiarity with ,. p . res. ; The applicant is unable to answer questions or es inco , i- i lete answers. The applicant is unfamiliar with the subject or j
- stem, as evidQ. .$ sitant answers, inability to locate information, inability to fd' locate control bogigdications or controls, and lack of knowledge of system operating
. procedures. ,
'C2 - RESPONSIBILITIES
.. y nsee r
The licensee's responsibilities are limited to providing the NRC examiners with whatever additional reference materials and information the examiner might require to evaluate the
- applicants' performance on the operating tests. Such materials might include simulator strip 1 chart recordings that document plant status during the simulator scenarios, and procedures that NUREG-1021 1 of 2g Revision B i
ES-303 document the expected operator actions.
- 2. NRC Examiner of Record 1 As'soon as possible after administering the test, the examiner of record shall review, evaluate, ,,
and finalize each applicant's operating test documentation in accordance with thermstructions in Section D. ' p.
- 3. - NRC Chief Examiner f ,,
- a. The chief examiner shall arrange a meet 8hh f the NFtdexamm? dbate n lea m j !"
; members after the simulator scenarios am[completifSuch moitinigi4IOiWifdI the examiners to compare notes to ensure this the same operating crewis consistent and lthe;d56umentation for'@s upportive,
~
- b. The chief examiner shall work with the odsir $Namiri$$"en the team to resolve any technical questions that might arise*Suring tnM'p*rocess, and i communicate any additional referencifiiistorial req ~
~ ~ io the facility contact. F 3
- c. The chief examiner or a mapageme pisilmisid noe will review the grading of each operating test toh th exa' omments appropriately j support his or her recogunendaticif,and nsure that the to e%$..test meets operating the requirements of$$401.1% chief edkniner or designee does not agree with any of the ex50iMets readmmendati6iis, he or she shall confer with the
' examiner before 84Enttuming#Mndation. Such disagreements are not common aniNNidEly
"" arise MTNEunsatisfactory grade is not adequately ified. re, ver/MyNirtant for examiners to be complete and
. accurate in and documentation.
. EeMng . .
e
- d. ' ' h~ i f f [ . _ ... , .
noe shall make an independent pass-or- fail
* ,risi " Final Recommendation" block on Form ES-303-1, "O '
- Examination Report," and forward the package to the respons? . sor for review in accordance with ES-501. The supervisor must ny recommendation to overtum the examiners results, and the "3 specific , sons for this action must be explained on Form ES-303-2, " Operating Test ts."
DING A9fD DOCUMENTATION INSTRUCTIONS 7 4 1.' t- __ 'and C '=ize Rouch Nca.: and Docu.T.entation
- a. Review the job performance measures (JPMs) and simulator scenarios that were -
performed and the questions that were asked. Evaluate all rough notes and
' documentation generated while administering the operating test to determine the NUREG-1021 2 of 29 Revision 8 i
ES-303
' areas in which the applicant was deficient. If the applicant generated or used any material (such as figures, drawings, flowcharts, or forms) during the operating test, the material may be used to aid in documenting the applicant's performance, if it contributes to an unsatisfactory performance evaluation, the material shall be appropriately marked and cross-referenced to the applicable deficiency and attached to the examination package for retention.dV b.
h[ Verify the validity and technical accuracy of any, questions at were asked during the walk-through test but had ~ notas been asLwinnt any prescripte unexpected events or actions that occurred $0 ring the ashblai6Ihnerating test. if necessary, work through the chief exardi5hr to obtainny'edd e material that might be required to resolvi5ny techrdd51 questionq. J @U c. N$7(gdY hh/ On the rough notes and documentation, label c(highlight every action, response, note, or comment that may constitute a persikrieIncs, deficiency. d.
$ bf$$$h?%
Label each deficiency related to the aglicant's adthihisths.,hve and plant system knowledge and abilities with the alphnnimeric codelfUiiiENEninistrative topic (e.g., A.1) or the control room % ' " (e.g., By$r B.2.1) to which it applies. L e, perfo eficiency. Using as a guide Review the eachand competency simulator f aling operating factor escripti , in @_. Appendix D and on Form ES-303-3 (RO) or FortpES403-4j8RO), co , ach deficiency with the number and letter of the ratingtsetor(s)itminst accu ' ty reflects (e.g., C.4.a). Whenever ! possible, attemptfidentify lAiirco0cGiIe of the applicant's deficiencies and Mhcode each deficiency with n[5NisiEi$ sin two different rating
$ne signifidiiritVis8ciency miyWe#oded with additional rating factors if the error bef5Mitent with the criteria in Section D.3.b, to be relevant to each Mbe shon%
Ef$$ifcited ratl 302kN essential that the simulator operating test ! doci geons'istent and mutually supportive for all applicants in an operatingsroorgDperating errors that involved more than one applicant should be noted 6 $iiflih h applicant's evaluating examiner. If the examination team membersM[o i riot have the opportunity to discuss and compj before lening the site, the chief examiner shall schedule a conference call after i the ex(rniners retum to their respective offices. l Applicants' Performance or categorizing and coding the rough notes, review, evaluate, and grade each ; applicant's performance in operating test Category A, " Administrative Topics," Category B, " Control Room Systems / Facility Walk-Through," and Category C, " Integrated Plant Operations," as follows: NUREG-1021 - 3 of 2g Revision 8
m ES-303
- a. Form ES-303-1, Category A Review the identified deficiencies and decide whether the applicant's knowledge and understanding of each administrative topic was satisfactory or unsatisfactory (refer to the discussion in Section B). Document the grade by placing an "S" or ,
Every unsatisfactory "U"in the be grade must appropriate supported withblock detailed ondocumentation page 2 of as Form ES-303-1., discuss $d in D.3. f. M/ em%
/s gh N ' p ,,x;(
After grading all of the topics in Categoryg" assess the app'licant's topic grades @ and deficiencies and assign a single "S" or U" grade f6Fthe Euiss6ryhlr th A7 applicant has a "U"in only one administristive to applicant in Category A depending on th(%piceth'e exa However, if the applicant has a "U"in two$moregf the administrative topics, the examiner must assign a grade of "U" fohCategory.A. Place the assigned grade in the " Administrative Topics" bloc 6E5f 5%M3postipg Test Summary" on page 1 of Form ES-303-1. Enter"N/E" ' ot examimkl Xthis category was waived in accordance with ES-2 P
- b. Form ES-303-1, Category B .
/[ .
On page 2 of the applicant's Formes-303 _ he names of the systems and JPMs examined dunne opega,llhg test ,_ categories B.1, " Control Room Systems," and B.2gpity Wg-ThrougigTAlso enter the safety function number from the pppropriateJCnowledge and Abilities Catalog (NUREG-1022 for
. PWRs and NUREIF-1023
.. -V. .
T,o dele __ PMs list Form ES-303-1 were property performed, cualuate ea coded in the rough notes for Category B. If the jeb5I$ng.critenanidifijii,yassign a satisfactory grade by placing an "S"in the "JPMEWisluation"*c' o @Mr that system; otherwise enter a "U":
# was' time-critical, it must be completed within the allotted time.
ical steps identified for a JPM must be completed correctly. l If the a,pgilicant initially missed a critical step, but later performed it correctly and C@'? accorrgllished the task standard without degrading the condition of the system or 9' t, the applicant's performance on that JPM should be graded as Clory. Further evaluate the Category B documentation to determine a grade for the prescripted questions associated with each system listed on Form ES-303-1. It is permissible to allot partial credit, when appropriate, to determine whether or not the applicant's understanding of a particular K/A topic is satisfactory. j l j l j
)
NUREG-1021 4 of 29 Revision 8 ;
., , , , , , , . . ..a
r l l ES-303 l If the applicant's understanding of both K/A topics is satisfactory, enter an "S"in
- l. the " Question Grade" column for that system on page 2 of Form ES-303-1. If l
the applicant's understanding of only one of the K/A topics is satisfactory, enter an "S" unless the knowledge or ability that was missed is of such safety significance that an unsatisfactory system grade is justified; in the latter case enter a "U." The K/A importance value will not be the only consideritibn in arriving at the unsatisfactory evaluation, and the safety sidnificanbe must be l explained in the documentation. If the applicant's understanding of both K/A ' topics is unsatisfactory, enter a "U" in the "Q6estion G* rads *i:68umn for the system. Every grade of "U" on a JPM or thsTy' stem questEiimustbe supported with detailed documentation as%scussed iriSecfionD.Mg 8
"F For each Category B system hk listed MI on N page nter-a2 ofform ES-30
" System Grade" of "S" if the applicant's performance on both the JPlfand the prescripted questions was satisfactory, if thsiiippli6 ant was assigned a "U" for either the JPM or the prescripted question, tisiih eit$Es "U" for the " System 1 After grading the applicant's
% h
.wlth respectWall the Category B systems, determine an overal fEa"$eMCatoiory B by7tilculating the percentage of satisfactory,$N(tem f gr$$$$$f theEgihlicant has (
80 percent of the systemioxamined,#the EM) asses Category B and l receives an "S" overall if the applicant has*n "S" on fewer than 80 percent of the systems, the
- t failsCategory B$nd receives a "U" overall.
Document they nt's g by an "S" or a "U"in block B, " Control
,hRoom SysteIms; _, FacilityM@ ugh,"in the " Operating Test Summary" on
$ $ age 1 of(orm;ES
$g@ialil0ES-204.\MB@s$03-1.
%Ni$nej A %5 @hgg Ent
- c. Form ES 303-1, CategorpC
. Using k .
V 03-3 or ES-303-4, depending on the applicant's license level, y evaluate 5rif$ncies coded for Category C. Circle the integral rating value
,' (1 throug$ corresponding to the behavioral anchor that most accurately reflects the applicant's performance. A rating of *1" would be justified if the applicart]j'nissed a critical task (i.e., by omission or incorrect performance) or comtrutsd multiple errors of lesser significance that have a bearing on the rating !
. factoffMissina one or more critical ta=ks does not nece==arily mean that the i;PM2.,; will fail the sin alster test. nor does nunc+ss on ava;v critir al task 35 .;at the examiner from rscsinn-ndina a failure if the moo!! cant had other
- l deficiencies that. In the aaorenate. lustify the failure ba=ad on the competenev evaluations. As discussed in ES-301, Competency 5 is optional for SRO l upgrade applicants.
' Multiply each integral rating value by its associated weighting factor to obtain a NUREG-1021 - 5 of 29 Revision 8 l
\
ES-303 numerical measure of the applicant's performance on each rating factor. Then, circle the corresponding numbers on page 3 of the RO or SRO applicant's Form ES 303-1. For each rating factor, place check marks in the columns on page 3 of Form ES-303-1 corresponding to the numbers of the scenarios in which the takid behavior was observed. );V A fQwm For each competency on page 3 of Form ES 803-1, sum;the;agcucted rating factor grades and enter the resulting competency 0reIi:le in the% Total": column. (The , grades should range between 1 and 3.) ' Using the following evaluation criteria, d a ' ofbe applicant's Pg7 %% g $7 %SPM9M overallr;r performance in Category C is satisfactorM0nsatisfactory and docI6m5rit the grade by placing an "S" or a "U" in block C
" Operating Test Summary" onrrnpage ._
1 offo$ntopaned
, Enter"N/E"if this Plant Operat category was waived in accordance witfiES-204 M if the " total" grade for a '
s is gre . n 1.8, the applicant's performance is gene , siti (( , p , if the applicant (an) egeF(s) L sabty consequences for the plant or the p661ic, themiaminerhiNYiicommend a failure even if the failure can naisisEjustifiedhn the bails of the competency grades. In such circurdiissbes.
~ "
- ion g mtfsiobtain written concurrence from the Chief O fore eting;thiiiicensing action.
ag . If ency 6, " Communications and Crew [ .,. ^ grade In individualy!s less
; grades ttGI5"or for all equal to other competencies are 1.8 2.0 orbut greater greater, the than 1 plicantis ce is satisfactory.
V 3I _Mtal" g~ : e for Competency 6 is 1.0, orthe " total" grade for any competency is 1.8 or less, the applicant's performance is u ry. N Note: Competency 5, " Control Board Operations," is optional for SRO upgrade applicay&. However, if it is evaluated, it shall be factored into the applicant's
?j g final e.
ify in detail, as' discussed in Section D.3, each rating factor that is assigned an
, integral rating value of 1, regardless of the " total" grade determined for the associated competency. If the applicant's overall performance in Category C is unsatisfactory, justify in detail every rating factor assigned an integral rating value of 1 or2 for each competency that has an unsatisfactory score.
NUREG-1021 = 6 of 29 Revision 8
L
\
ES-303
- 3. Finalize the Documentation
- a. Review and finalize the simulator scenarios that were run for Category C of the operating test.
l A l ' Complete Form ES-D-1, " Scenario Outline," by entering the applicants' names,
' the positions they occupied during the scenario, and the f5cilityYhame on the top of the form. Also enter on Form ES-D-1 an scena$isiivisiEhs made during
- the test, so that each form accurately shows lof the'eYents that.actually occurred during each scenario. Change thiiEent numbers nialhinction numbers, malfunction types, and descriptM6s as necehiry[dre conditions. These changes may be madEhsing pe5And-ink oYbyiSthM$h scenario, provided that the final form is clear and ligible. E/g;?
Update each Form ES-D-2, " Operator A $nect the "as run" conditions. ffhatMiNd5(run and fill out new Discard forms for anyor mark events thatas "not were used" run but n any ever$'$t originAll@s%, . Neatl , l comments, and additional actions ind$ paces bothMelisifiexpected operator actions. 7 gh - The final Forms ES-D-1 and ES-D-2pu a ; kgible, and sequential record of the actual everdEand actipAs thaIM during the simulator operating test. The fondis sont Ithe applicant shall not contain any rough notes l orirrelevant co M ,. - Any events orma$ unctions nction as expected or were not useful jn evaluatirdMiskplicants' ' Q5$fveillance test that required a long time to gorm)Q,tpipinoted o theYnaster copy of the scenarios to aid in future
. 'sgenerio pre :
b. N ants' the applic$ i ES-303-1 and the rough documentation. Justify in fistafiedfism ES-3Mi2,' " Operating Test Comments," every knowledge or
/ abilitf%) hat c"ontributed to the assignm administre8ve topic in Category A or any system in Category B. Deficiencies that contrWutodio Category C integral rating factor grades of 1 and 2, as
!? discussed 1n Section D.2.c, must also be justified in detailif they resulted in an d unsatisfAftory score in the associated competency.
J"x{ /
'(3h De8ein/
cies that do not contribute to an unsatisfactory grade anywhere on the (;4M^8 yihespiifrating test shall also be documented; however, a brief statement describing error is generally sufficient. Examiners should keep in mind that their licensing recommendation and the associated documentation will be subject to review by the chief examiner and regional management. Therefore, the documertation should contain sufficient detail so that the independent reviewer, responsible supervisor, and licensing official can make a logical decision in NUREG-1021 7 of 2g Revision 8 L
)
ES-303 support of the examiners recommendation to deny or issue the license it is expected that the documentation for an applicant who marginally passed the operating test would be very similar in scope and depth to that for an applicant who marginally failed. A Provide the following specific information, as applicable, for each deficiency that contributed to an unsatisfactory evaluation:
- A the question asked or task administehi.e.,
simulator scenario and event, as pas the apy operr*.ing crew)
} j7 N .hJPM on on the ggf:4 or the [
y g2lh p %AW the applicant's incorrect answer or actionInnd an indication ofiwhether
~
the action was a JPM critical step % the lack'of knowledge or ability $the' . demonstrated the consequences of the a _ snN s incorrectanmWororaction the correct answer or NIA on, vdeden ap.propriate facility reference (e.g., lesson plan, system'descriptiairi,'M' ':'^iidine and number) f47 g 13pb"
'ng (as given in NUREG-1122 or the K/A numberland its irgportance NUREG-1123fddd thefiEility's le " ng objective the item V W
)Yt the applicant did not understand or m 10 CF{
was to perform' MSA A' , I statemenII;4tuch as "did not know decay best removal system") e. [hs sible, stantiate comments with printouts or strip chart reco%$uisierated iluring the simulator operating test and drawings and illustrat ed by the applicant. Retain h ocumentation until the chief examiner and management have revi Rhe examinefs recommendations and concurred in the results (refer to ES-50s a w
~, fy
.s-reference each comment on Form ES-303-2 with the specific task, bject, or competency rating factor to which it applies on the applicant's Form ES-303-1. Do this by entering the applicable alphanumeric subject reference from Form ES-303-1 (e.g., A.2, B.1.3, C.4.B) in the left-hand column of Form ES-303-2, and entering the page number on which the comment is found in the appropriate block on Form ES-303-1.
. NUREG-1021 8 of 2g Revision 8 I
l
ES-303
- 4. Make a Final Recommendation
- a. After grading and documenting the operating test, make an overall recommendation by checking the " Pass" or " Fail" (or ' Waive" if the entire operating test was waived in accordance with ES-204) block, and signing and dating the " Examiner Recommendations" section on the applicant's:Fdrm ES-303-1. Make a " pass" recommendation only if allsummary;blockkbf the operating test contain satisfactory (S) grades orIthe lett is "N/E," indicating that the applicant was not examined in that area @p - %N n% ,
b. hY sh% bNL If the written examination was not waived and the written examination' data have kI not yet been entered on Form ES-303-1[ Mute the dka'minatiorEpasi$iige 65 written examination grader (or NRC reviesfif ths~ examination w'asiradid by the facility licensee) for processing in accordancs'with ES-403. - If theGitten examination resuhr, have already been entAssd'efonhard the examination I package to the chief examiner for reviewY T, W . Nk*b i E. ATTACHMENTS / FORMS # Ab:II%wA 'kb$$h - l hWh ?NbN Form ES-303-1, " Operator Licensing Ex,aminatignil ort" Rep $% Form ES-303-2, " Operating Test Corsnents"fjrNygQ , Form ES-303-3, $rkshest'fcNintegrated Plant Operations' Form ES-303-4, "RO Competency etaxiv Gradidq l Grading W[Worksh55Elor integrated Pla "SRO Comp /mW
- ft
$? .
>A rahalb[h ms;, q n
- d. nges hsp gnnes wm mP q
h iiEh Ihg
$9 43 l!!3y% )MndiS mA sgdp#
h}$221 & . % y '" t.w.s
$R$p AG, f?%k .,
9%
- kild9h!A 3:%, y
% .fur pw%gV t gyh i
NUREG-1021 9 of 29 Revision 8
ES-303 Operator Licensing Examination Report Form ES-303-1 U.S. Nuclear Regulatory Commission Operator Licensing Examination Report Applicant's Name Docket Number 55- A I R Examination Type (Initial or Retake) FacilityNamej j# Reactor Operator
- Hot
.[{
Senior Reactor Operator (SRO) Instant 40 #C old [TAc111th SRO Upgrade < BWRsddiscriN3$n3 q 3. ~ z s.- [j SRO Limited to Fuel Handling is ?"'#1^
, PWRi?
& w?gy xq{g?
Written Examination Summarygg, m ms. NRC Author / Reviewer Tota 8SxaminationjPoints eANth V44-' NRC Grader / Reviewer , nts* gotalfApplicant hij?fis.e.ft fp ~*p . p Date Administered jfr ~ App)(ypt.jacaritd:Gradejf % Oper;ating Test:Summarygn y Administered by [if gh DateAdhbistered A. Administrative Topics. pk ./fIk d A aIma y w B. Contro]jAcom Systemstandjfacility@myE-jsirough al N@ j @ W %. NIA,. >w.wx.. $ib C. Integ(ate & Plant OperationsjtSimulator Operating Test) N!Pe A Ny1 Examiner Recommendations Che N N b [! [ QassY Fail Waive Signature Date i WrittNExaminationkjy !ffh! nw y pperating Test gf r ru.2 .s$!y' [&1nal Reconnenda.tjon s;0" t ~ /i y License Recommendation k~ #1s sue License Signature - Supervisor Date I s Deny License ' NUREG-1021 10 of 29 Revision 8
ES-303 2 Form ES-303-1 Applicant Docket Number: 55- Page of A. Ackninistrative Topics Evaluation Conment Page (S or U) j Number
- 1. Conduct of Operations 8 g
,e ., sas
- 2. Equipment Control f 3 Qff
& .,x .. e
- 3. Radiation Control 7
. , /PN. W., , ,od:7
- 4. Emer9ency Plan bh, eM a < es Vsen MM~
'ca . %
?. .fQ 8.1 Control Room Systems Safety 6 [5;41*%$n ^sJPM Grade Function 4 \sps $?f5 or U)
# f *"~
g, tw .sw 4> s< ,
%:::.h 3
%yA Question Grade Q'
//**~ ,
je ,S pg,pe g f . (S or U) j+' %;K % lLjjii 4+( /,l!l .,/ %a
# System Comment
~
Grade (S Page System /JPM Title // //e 4 or U) Number
- 1. , A, f[I hNl(fff}gggfh
- 2. A. h. r _ 4. % . . . .
W5P" f'-:- - hhg Q . '+N.- h 3* AN W;WWf4W k 4:-W*\*v t 4 ef34 *v 4.
% sr
,ggypp>e99g s
wygrgr
- 5. jff DEhj{h w .+ '
6 A.v.x_+F p
.:.f,Vn 3 t em a,p 11 .
r -
,pw 8:2JFac111tyj#alk Through 1%in,ds, -
2. 3. NUREG-1021 11 of 29 Revision 8
1 i I i ES-303 3.a Form ES-303-1 Applicant Docket Number: 55- Page of C. Reactor Operator Integrated Plant Operations j (Simulator Operating Test) Grading Summary { Competencies / p.5cenariosP m ! Comment ! Rating Factors Weight 3.0 2.0 1.0 Total $0bserdd' Page No.
/: 4., ~
.,..y
- 1. Alarms / Annunciators g p 4 42$ 3 l
- a. Notice / Acknowledge 0.30 0.90 0.60 040.i 5 m. 6 ,. _
- b. Interpret / Verify 0.40 1.20 0.80 f
[g:,./0 -j Y Y -- x bI 5 .d
" ' : :<w $ 7
- c. Prioritize 0.30 0.90 0.60 [D%30 8:sc ,,_ . , _
H3.1 q mm '
~s w"
- 2. Diagnosis W % 'W
u-1 2 3
\
1
- a. Recognize 0.40 1.20 0.80 OjdOR A's. - , a
- b. Use Reference Material 0.20 0. 6') 0.40 $0.20 NNl> :n "as .
- c. Diagnose 0.40 1.20 0,.80; 30.,40 '
d d, _ _. s a.e Mg
- 3. System Response
,gp...g *"Ig p:
^"
- v.: g;f4
[*(1p
;lpr 2 3
{
~gd,.
- a. Locate / Interpret 0.33 tiiC0 *5. . .,j. v:33 a .,.wl :a4 '_
0.67J l
. u ._. ,
- b. System Operation Knowledge 0.33g 1.00 %$7 fy7 0.3g
- c. Effect of Actions ,0!3df 1.00 j h.67 0.3N
- 4. Procedures / Tech Specs
) -
/ jf. 1 2 3
- a. Reference A [0.20 SE601 1041e#0.20 p ,# nd. _, _
- b. Eop, int /y/ImmediateActio%l ns p.0. 40 tw :.
T.20! iciB0 0.40 j x , J,, .g wa
- _ _ _ _
- c. Procedure Compliance ; 0 0.60 0.40 0.20 y c. te m ,% $m,20
. s. _ . . _ ,._ _
Tech $pecEntr[( % [0j20)
- d. 0.60 0.40 0.20 _ _ _
y ,,s Control Board Operat'i'ons
- 5. 1 2 3
,gg a
- a. . 0.5 0.75 0.50 0.25 b.
Loc,$hulateate'
$ d' h > 0.25 pug:n;j@$@yg 0.75 0.50 0.25 w$[ _ _ _
c,jR
- p. esponse st@ff ki 0.25 0.75 0.50 0.25
[u}y jdki!HanualControl 0.25 0.75 0.50 0.25 ,,, _ _ Ikk,kCommunicationsj#[ noen At 1 2 3 naMfPeov.ide Infdewation 0.33 1.00 0.67 0.33 _ _ __ hh%emygy thbation 0.33 1.00 0.67 0.33 _ _ _ c7* tarry Out Instructions 0.33 1.00 0.67 0.33 __ __ _ _ NUREG 1021 12 of 29 Revision 8
ES-303 3.b Form ES-303-1 Applicant Docket Number: 55- Page of C. Senior Reactor Operator Integrated Plant Operations (Simulator Operating Test) Grading Summary ; Competencies /. Scenariosgy Comment ' Rating Factors Weight 3.0 2.0 1.0 Total AObserved? Page No. i 1, Alarms / Annunciators . g th i 3 l
- a. Prioritize
- b. Interpret 0.30 0.90 0.60 0.301 g O &g _ _ j 0.35
- c. . Verify .
1.05 0.70 pj35* f$ b; p' _ __.g j 0.35 1.05 0.70 f9f35 a k. ~ shi L4 X j
-, s x 2.
a. Diagnosis Recognize 0.25 0.75 0.50 7 'l '+
,0.25" /^
g' 1 2% J';
~
[, ,',J"', k"
- b. Accuracy 0.25 0.75 0.50 ,'0 25 7 ',
- c. Diagnose 0.25 0.75 0.50 0.25 g,['A _, _ _
- d. Crew Response 0.25 0.75 0.50, D.25' /# ' b
- 3. System Response ye am
#'I* ' ' . % ill 2 3
- a. Interpret 0.35 1.05g [0:70- -0.35 ; ,d{ >. ,, ,
- b. Attentive
- c. Plant Effects 0.20 0;60,3 +0.40, 0.20. F 0.90' ;6A5j *> 2 0.45 4 1!35 4.
a. Procedures Reference 0,25h(0.75 < fk f0.50 Di$i i 1 2 3
- b. Correct use
- c. Crew Implementation jM i
do?25 1.50.3 1.00 Og5J 0.50 0 (fs@!25
- 5. Controh'8oard Operationi,g }g 1 2 3
- a. Locatiri'gg
- b. Ma'eipelateh g%j ' 7fi, $0.25 0, f 50 0.25 bQ( $$J5 0.50 0.25
'(
- c. Responsej*;'fn ]$25 0.50 0.25
,,0.75
- d. Manual 'tenteel'J%
E9125ke 0.75 0.50 0.25 ,_,
- 6. Communi.
(f 1 2 3
- a. Clardf . 0.45 1.35 0.90 0.45 .,,,,_ _ __
i
- b. fidir. formed (I>0.35 1.05 0.70 0.35 ,,_,. , _ .__,,
i c.gticeive Informatiorg . .' O.20 0.60 0.40 0.20 ,_ __, .__ irecting Operati 1 2 3
. W Timely Action 0.20 0.60 0.40 0.20 _, ,._ ;
s
$5afe Directi . 0.40 1.20 0.80 0.40 djfDrersightJ 0.20 0.60 0.40 0.20 jpMorsr % ' O.20 0.60 0.40 0.20 8NMNb Specifications 1 2 3 j
- a. Recognize 0.40 1.20 0.80 0.40 .,, _
- b. Locate 0.20 0.60 0.40 0.20 _ _ .,_,
- c. Compliance O.40 1.20 0.80 0.40 _ _ _ _
NUREG-1021 13 of 29 Revision B
ES-303 Doerating Test Comments Form ES-303-2 Applicant Docket Numto Page of Form ES-303-1 Conments Cross Reference . j f fb [: ,, f f. p: .# - . . 4(:
,p . ::<3a .w y.; 9 :;
x.'4 / r e:,,.-:. N :.: e [$;i:p;y;:::!Wx,,.
.[iNpf j@ i i:..!M > ,th x . ys .n %;. ,,
ilEsp Mf! :Ik .a:$?? jig: " ,J;::.#) MQ+. n. . . . 4:.0:op: "'9eME ^ 'u
*T
- 1. ; *.A
- ..u 4.
.. 4 .?V %.s mo:- .<:.: ev.;s.
sqg;g; hihs. !h' !I$:. j jj>i:d g!E :s:h
's, g;f 9
i: sir s
,,s
+'^ 5 J
jf? ^ - y gy. ' 3sw:: 54- 6 j i v
$${tp.,
4;yf .,ip -
-1 yg7 oW Mia:f:<fM:gg!M,J.ij'f
% j:. : .:.- rd: e 49 e.:.!Y 9:: 1,5@5bF
~,n/- y::,7 x.,,
.ki. k ,
pr qq gr f;-57 <@d?i MN 5A,
'dw:y/cx:wl[fbk :e. f h +$i :!' '
.:Ag
- 4i'.:g(ts 4 90$.8Ai: i:n b::W:':.k:hwi:4:
M::W!$9
/W' :R$ ig w:- . y ; .:/ ,,..;.- , .s u .9/ '
' v'
<Es4pg th s. x :s:.
.+. .v.7,:.;;;y-),
py e):qg a:-- . 44.c X:.-@ p , '. f.g(::ygjg.g;:"('p.9ph :
%; 1574! ;*h: ., % q:o:aus): ' ' ;4jh.,
em,:s . s w . 3. .. +. on #,:way. m;.v y.n.y q:9:x,p,. 23 M m: n I yp**+45p
,3-.,.-- weg,y d kl k' (
.d(P
#::d-
%j.g.
,w
$;d@sy.h.yp ;
;:::s.u,4:4a u
M .. :g.gg W w a'
- /
s> ? at w s Ar
" w,:. M n.a
,Mi1
( h
,. . /
'WWh
... . ,4 <:as#'
-- s :m.sg. : ( ,:: <wx.y 3.
'
- gg4 ;
i NUREG-1021 14 of 29 Revision 8 I ,- c l' j I - ES-303 R0 Competency Grading Worksheet Form ES-303 3 for Integrated Plant Operations -1. UNDERSTAND AND INTERPRET ANNUNCIATORS AND ALARM SIGNALS DID THE APPLICANT: i !+ . (a) l NOTICE and ACKNOWLEDGE alarms? [') f,[ # ! ,A, l i 3' 2 j ~,,+
- jRT,
. b :,ll ,, p. i ., + j~:^y ; ?>. /;* Consistent ~and ' Minor difficulties Failedetonoticeand%,,,Al,' timely . orlapsesinawareness!($ x' , acknowledge important, < acknowledgement or response distracted 4y./ k*s%^% alarms: ,: nuisance alarms: etc. "~ ; .5% ' ' 'J , s 0^ ;^^[ '30 X ?"t,, = ] I
- i ' < s4 >
,y,,, %: , ,s 'y, 'y l
- f. ,w~^, ,, <'hy%yl', #'?
(b) -Correctly INTERPRET and VERIFY tha,t ,annunci4 tors and alarm signals were , consistentwithplantandsysteEcondit%'s'tipcludiig'stheuseofalarm response procedures (ARPs), whe'n necessary)? ! w/* < 3, n / fr*',n QO'0 l 3 W*2 , Jt 1 l - Consistent and inaccu es* Significant inaccur-efficient #{'y., I,/N1' i 'injinterpreting$9 'Facies resulted in - interpretation and verifying signals plant degradation: and verificationis 9 poor use of ARPS- 'A, % lVD}:^ h%,g g } % d Q'p/ y;* , , ,x,rs a, pr ~ g,,e ~;,,# s% % ,17<>,., /,s [.40 X = ] 9e vg:'m (c) / 3 v/ severity? ~m. .,' [~#e 3 )((; 6 #, l ,' { , ATTEND to ANNUNCIATORS and ALARM SIGNALS in order of importance and u ' ,,- :A y' 2 1 h1 D9fh Minor inaccuracies Did not prioritize att @ yj M and oversights attention to signals; all cases inattentive ti important alarms [.30 X - ] NUREG-1021. 15 of 29 Revision 8 f i e 7 ES-303 2 Form ES-303-3
- 2. -DIAGNOSE EVENTS AND CONDITIONS BASED ON SIGNALS AND READINGS DID THE APPLICANT:
(a) RECOGNIZE off-normal trends and status? ,c/r, 9 $N 3 2 f1 '/[/ j$::'?k , s Ouick and accurate Some delays in /l} Serious anissions A, recognition recognizing off- l(' delayO'or '(;;;g' yp% , normal conditions 0 inacc cacies int - ' ,~ # ' [d d"? recognizing evenEs(^/ N l% %"TA0% 3 Qg,s F/ s$,i, s ped?% 0 M':'if (b)- doolbihcharts$'5't'0 5 ' aid in diagnosing Correctly USE REFERENCE MATERIAL (d'Nsk'l 3fj and classifying events and cond fons? .m 3 ? l,f , 'NYY, 1 Correctly used Minor, errors inh references. when necess y us"orrelyihg #on referencesz* }Y ' ) [ Did incorrectly references to not useduse or , ,J ..oh,[h diagnose events s-ki . t (<; , *)@'U's8 ~) f i+!' [.20 X - h ti?'$h J .~.skj ,bfb gAnyhN;*'?k I$hb (c) tly DIAWESE phnt conditions based on control room indications? '3 ', I T'd,,'?*p'f[' 2 1 0w "^oses were ? Minor errors or Faulty diagnoses e f difficulties adversely affected in diagnoses plant status w:,,q.f b [,40X - 3 NUREG-1021 16 of 29 Revision 8 b i l ES-303 3
- j. Form ES-303-3 n 3. . UNDERSTAND PLANT AND SYSTEM RESPONSE DID THE APPLICANT:
(a) ' LOCATE and correctly INTERPRET relevant instruments and other indicators of ! plant and system response (s)? j / ' 3 2 in /'V f, a f ',s '<a >Q. : ,a,yf Accurate and Minor errors in j Serious'kssions',A, efficient location locating and inter- [, / or 1,naccuracies 2 and interpretation preting instruments ig;1nterpreting';,7',3. ,p ,.s y g ,' '^ i of instruments and displays ; '*' instruments Y,','x'def, N , 6. ,, w'd[ M 1 3 &+ y , u, ,,' . s , y, s*?ns %' d 'g> f . t ': , % ,: : o , (b) 4 Demonstrate interlocks, and automatic KNOWLEDGE actions?'r' : of .,j[,,,SYSTEM OPERATION,hnc l 3 4 ,f5'e[,,(y$,LJ' my j 7 /[,$ h :l 1 ; Demonstrated Minor /instancedof Inadequate knowledge 'f thorough under- erh causedfty in- resulted in plant Standing of,4 /' adequate knoirledge-f degradation system operations # l @,I;D ( 'E ';/ 0i?~'h,, # ,: %['%g'f %. ~**"*l*~ , ^ ,' ,, \' [.33 X - ] v t' v17 i (c) /[ pow.;$[ Desenstrate der k mo an*h'h>dtanding of how his or her ACTIONS AFFECT (or ina 8MANT and SYSTEft:C0iS!ff0NS? ) R3 Yl' ff 3 [ 2 1 { ll ' ' ' tood the # Minor misunder- Appeared to act : Medd'ef actio4 standing of effect without knowledge Iiplimt'5iidI*' of actions on plant of or regard for s,Megg4'@ and systems effect on plant and systems [.33 X - J NUREG-1021 - 17 of 29 Revision 8 r 'ES-303 4 Form ES-303-3 4, ' COMPLY WITH AND USE PROCEDURES AND TECHNICAL SPECIFICATIONS DID THE APPLICANT: '(a ) - REFER T0 the appropriate procedure in a timely manner? 3 2 I 8 Quickly located- Minor difficulties and n/Iw' /,' Problems'and failures appropriate. ' oversights in referring 0'1 y procedures to appropriate fh'; # ~n referring t to', sprocedde jf procedures [q 1mgortantinstames , g#';' '^ o ~ , Vv ' +':lh,J <0 X %gLV y5.: Q =
- ~
(b) RECOGNIZE E0P ENTRY CONDI(IONS and carry iategimmediate out<4propr,s yactions without the aid of references or other forms of assistence? % s,' ,$,h,y 3 2 /n s ,a > q'.1> . . Consistent, accurate Minor lapses' errors; /a,O'lO M '";$% and timely but action inerall/f k ,r);EiefuteactionsdDid nof accurately recognition- appropriati g/ /f ,s }y,p['s.sf[.40 X - ] (c') ~ a# di / ' COMPLY WITH ' procedures 4cludingsrecautionfland limitations) in an accurate and timely manner? ,7 A4;'( gb ^<,h : 817 ' ' ' ' ' p# 3 Au/^' , s,:c/E,&) 2 1 %;4 s.tA ~ ms:s-s,, Accurate and #,.km ' - w, ,n, Few error,s: corrections Many significant timely compilag,e$.\'?jh made 'tri;istficient time errors: excessive assistance required N /jfgggtoavoid,"Adverseeffect pqot, >= 4g'$,j,J.,[~ ~,y. sa [.20 X = .. ] (d) g#'ECOGNIZE R plantIiNtions that are addressed in technical specifications? O -f ,;/l{ 3 / 2 1 ' =:< Recdadized and Minor assistance Did not recognize I2iluipligdfi(itF required to recognize conditions and/or LCOs;and agpon conditions and/or comply with LCOs and statements comply with LCOs and action-statements action statements [.20 X - ] NUREG-1021 18 of 29 Revision 8 T _ i i- l v .ES-303 5 4 Form ES-303-3 )
- 5. .0PERATE THE CONTROL BOARD DID THE APPLICANT:
a)' LOCATE CONTROLS efficiently and accurately? . /p' 3 2 di P omptly located. Some minor hesitancy f m b'*V A,IJnable'to locate ! appropriate controls 'and difficulty in /7 I in all instances; . locating controls ,i% [c/ controls'without4)} P' %'i ,as! i N','% ,CA 5 X = *d) ~' V ; h ;f, e'( % S~ i (b) MANIPULATE CONTROLS in an accurate and timely *'mannedN ! gyQ ;; ) 3 2 #$' 'A 1 Control manipulations p e'% NN~4' % 4 as .' Minor shortcomings. s,A'J Impr% ~,,~ aoper; manipul-were consistently -accurate and but efficientlyfg;dy[:8g, mitigatedanyj#' g 5 ^ p majot4fsystem k ations[Ces'ult i timely resulting c6nsequences \ < giertidbations ; ff s 4$^g,if' ' /v7 .};f [.25 X - ] (c) ACT appropriately inponse res/to 5',I N tV WSTRUMENUtEADINGS? i 3 6 7G ','r .s ') , 2p k~T 1 / v '!J i% \;WAY ; \ Responses,were'y'kb v;N'Gelnera))yadequate Failed to react appropriateand'i' A fesponse:t.some appropriately to- ' timely Nj's,!)['^g er'rors'asd[Japses instrument readings -6!T,f' 3: Tg "df without assistance ! y [.25X - ] (d) yTakeMANUALC LTef automatic functions when appropriate? ' ,Gl r "cz, 3 ? g 2 1 , nual corttd[' Minor delays and some Depended on automatic when?ppMiste 6 prompting necessary actions: had to be %*.!j)#* before overriding prompted to take ; automatic functions. manual control E.25 X - J. . NUREG-1021 19 of 29. Revision 8 l n , j l i 1 ES-303 6- Form ES-303-3 i .6; COMMUNICATE ANDLINTERACT WITH OTHER CREW MEMBERS ! , DID THE-APPLICANT: s. (a) PROVIDEclearandaccurateINFORMATIONonsystemstatustoothersfoc$the performance of their jobs? g*g / / / '5IM4'? 3 3 2 er ' 4- 9 p <. % *n'% . ,%,%:. gp Provided others Minor instances of with accurate and needing to be [g,] 9' Failuretto'accuratelyprovide'imp ^' t pertinent information prompted for input: Q information to others' JJ;/ some incomplete and syjeopardized plant *QJ' inaccurate information A (\le' Status ft' %f %<'%$333X;?_,,- ] ys .n. /0$5y I k', (b) Effectively RECEIVE INFORMATION from#ctherl$in'luding c requesting, acknowledging, and attending tofffrof1nat1Im , . 3 f I b'j.5 hi h[ ? Responded and Minorjffistancest > Inattentive to reacted appropriately information offi11ure t { toinformatjon ,qahowledge#a(rope #,j/p[ provided by others 39 Jf fromothe%:ph 4 # %p; respond to~" sformatTon f ' %m $$f?- ~ ? [.33 X - ] %,)%[:,12,t{$p Vf, % ~i' ' ff;41A~tf'f6@A (c)- OUT the*4ftST80CT10NS of the supervisor successfully? [3 2 1 Alnylcarried out. ' ? Minor hesitancy and Failed to promptly hj . difficulty in following and accurately follow . g$ pervisory }gs:. djg orders, but ultimately directions: blindly cuss co@ lied successfully complied with wh($]ues)w,enabletm q g runs erroneous orders [.33 X - ] NUREG-1021 ' 20 of 29 Revision 8 T '~ l. 1 ES-303 -SRO Competency Grading Worksheets Form ES-303-4 for Integrated Plant Operations
- 1. . UNDERSTAND AND INTERPRET ANNUNCIATORS AND ALARM SIGNALS 010'THE APPLICANT:
(a) NOTICE and ATTEND to annunciator and alarm signals in ordertof their importance and severity?
- s. y/ c '-O' /
3 2 b'A fu k',1 .- b^g s ~ J7 Responded accurately Hinor difficulties k Failed oatendto'\ 7 and efficiently in in attending to signalst Js or,prioritize import " all instances or prioritizing attent16n4 Ja'nt alarms: responded 7 '( , 'islowly: distracted by , gp G'(,nuisancealarms y- w [k, X $,^'g ] .(b) s*n, :p. Correctly INTERPRET the meaning anddignificance.of alarms;,,and 'y F annunciators (including the use pflalarm se' procedures (ARPs), when necessary)? 17 3 ll// p % 3 V'f[: h' 1 1 j^[s/,j2 f/ f^< ' Understood and Misinterpretations, quickly determined Min'or[inaccutacies4'fh-@f[' delays, or misus2 of what failurNi< alarms g,for(delaysl> interpret 1ng'aisnia'sY in ARPS resulted in f,,,% plant degradation wereindfcatisg'(?%%/*C/,,'-,Ja 2 :1n &0l/C;4) Y:% [.35 X - ) %y v- ~, ; , s a s Ah%;nh, ' ,: # (c) VERIF . s em condit.1dNs?i n #g/yd'y,that' 3 y;Tg/ gV 2 annunciator and, alar 1 4t f Ensured proper Minor lapses in Failed to correctly Ver:lfication when alarm verification, verify signals on pecesAry but no inappropriate important occasions: [E ky;, #(", C67/g/pda result of inadequate actions were taken as did not notice incon-sistencies between verification alarms and plant conditions 1 [.35 X - ] NUREG-1021 21 of 29 Revision 8 1 o
- ES 303 2 Form ES-303-4
- 2. DIAGNOSE EVENTS AND CONDITIONS BASED ON SIGNALS AND READINGS
.DID THE APPLICANT: (a)' RECOGNIZE off-normal trends and status? ,,M 3 2 t il s.J , ,e; ' ,,s e ~ -Quick and accurate'. Some delays in [ Serious" omissions, recognition recognizing off- g ,f delays . 'p'r ' ~4 g fy normal. conditions gf inaccuracies :in t; ,,p, f,e jtv recognizing trends: Af'.<" , 4, 9 q n, n,' t, > N b25 X - Kf, '* Ny, ,A (b) . Ensure the collection of CORRECT ACCURATE. andyCOMPLETE 1nformation and reference material on which to base diagnose 5? *% >,^,,,;+,, . * %:2 #,< ,. 3 2 \ 'n N:':g'i'> /e'%' :; ,,'"79, Ensured that all Minor instances'cof'over < 'ASerious* instances of relevant indications looking,ov,edrelyinggni ,f,~mijusjngorfailing-and references were checked or misinterpreting ations and/or refefences f ,10dic(-s,f,,Uinformation * 'to;use .important /W5 l* ' ff [.25 X ] (( p[r !j.,7)#, f MA (c) Correctly DIAGNOSE plant' conditions based onl control room indications? . ,x@a 1:a 9 y$$^ Sff;'%p/'% 9 qf'g,5 < n@'1. 2 1-DiagnosesofMa% ntM'?,$. Mif M b rs or Faulty diagnoses difficulties in s adversely affected .accurate conditions weref'p;,'j. -),M M ,: , g diagnosing , plantconditions status [.25 X - ] -(d) ,g* CREW? Ensure -4, i that CORRECTsand TIMELY DIAGNOSTIC ACTIVITIES were carried ,s s-L'O')d 3 2 1 . effeedIie Minor errors or Faulty diagnostic diagn05 tic'4ctivities difficulties in activities by crew anddf agnoses by crew diagnosing by crew adversely affected plant status [.25 X - ] . NUREG-1021 - 22 of 29 Revision 8 k ES-303 3 Form ES-303-4
- 3. UNDERSTAND PLANT AND SYSTEM RESPONSE DID THr. APPLICANT:
(a) _ INTERPRET.: control room. indicators correctly and efficiently to ascertain and verify the status and operation of plant systems? ' ,[ ~ 3 2 ,f,, / _,1-) h 4, %o 1 s '~ ~ Accurate and efficient Minor errors in [;t; ~ f Ser,iods!diiiss' ions, ,d/ interpretation of interpreting delays, or {naccur , u @ {A instruments and instruments ane' ,acles in interpreting 2 displays }v ,, displays iP']instrumentsand?.;,,a*^' V /',;) displays , ps :,9f s +s, <k L35'{5 - ] -(b)' Remain ATTENTIVE to control room indicatfN?s ^ [,$'(:k ,[,Mts(: K 'k gf' ~ Regularly scanned ji Sporadically scanne,d { ; M,Rarely ?> scanned indi-indications: antici . indicattbris: mincfr p/ cations: failed to pated changes'in plant laps $$$Manticf[ating anticipate predict-conditions due to pr litableci$iNges able changes in plant i events in, gress status i .,S. ' [.20 X - ] (c) Demon'st brough gand actions, a thorough UNDERSTANDING of how ! - the PLANili$iSIT.MS and 'ENTS OPERATE AND INTERACT (including set da i iticactions)? N;yst . /poity)$M) 44Y.;: 4 2' 1 l De,mogstrated thorough Minor errors ber.ause of Inadequate knowledge uddetstanding of how gaps in knowledge of of system and com-h,Nins and compo, s .how systems and ponent operation ' M a nd interact components operate resulted in serious $3 . mistakes or I plant degradation [.45 X - ] = NUREG-1021 23 of 29 Revision 8 g.: ES-303 4- Form ES-303-4
- 4. COMPLIANCE WITH AND USE'0F PROCEDURES 1
DID THE APPLICANT: I .(a) REFER to' correct procedures and' procedural steps when appropriate? A f,; 3 2 [g. A g( 1g , p("f ' Requested or readily Hinor lapses in a p, Failed to cocrectly located all appro- referring to or refer to', procedures s /r priate procedures locating appropriate ; in japor' tant'y . <'?*4 ,, g/ f%[ *e97s v '/ as necessary procedures instances .x p g,y 5@X::U25 x - Y (b) h ' USE PROCEDURES CORRECTLY. including following procedur,al; steps,1n correct 'b. sequence, abiding by procedural caution 6W~11mitations/fielecting correct paths on decisions blocks. and to fy Fins'1'tfoning b M *en procedures? 3 2. ff,, V// Yp" 1 -Accurately and Minor M rs, but [ Significant errors promptly executed made"tiecessar gf impeded or slowed procedural steps corrections ,,, i } recovery or degraded #4 bt'nely fa ' plant unnecessarily 4, AfBJ:4
- L 4:4%
g%y^y4ff W' ,? W'% > g 5g y . 3 ' w;'4(P>, ,p' t^t % "iidA '(c) heis Ensure:;$yu,sfe, eff : ENTATION of procedures BY THE CREW? 4.s a. 3 /[ formed k '<$h,,, WWFA 2 of Hjfu,' Dew occasionally had 1 Kept crewin/., Read procedures to proeg'luralstatus: got l'T$o question SRO him/herself:, failed ac%ledgment-from ' regarding status: to coordinate or ~edstwhenreading c J allowed lapses in verify crew's use of iddd$dures'- d' implementation by crew procedures [.25 X - ] NUREG-1021 24 of 29 Revision 8 7 ES-303 5 Form ES-303-4
- 5. OPERATE THE CONTROL BOARDS DID THE APPLICANT:
(a) . LOCATE CONTROLS efficiently and accurately?
- 3 2 g3/
Promptly located Some minor hesitancy or- ! Unable EI S $b appropriate controls- difficulty in locating . contropy@iihdsdhs j;f in all instances controls .. assisi!ance NE F %D N , (b) MANIPULATE. CONTROLS in an accurate and timelygma ~ h3 @ . d)m. Manipulations were M .h Minorshortcomig Imprppermanipu-consistently accurate- butanyresghnb K"" 13gjonscausedmajor and timely consequences *were 2 stem perturbations readily tigat ' [.25 X -- ] (c) ACT appropriately in response to UMENTsREADINGS? g % ! ??-N <4,4 2 4 9 ,,V ResponseI*'serR% dd Genin 11y, responsive. i Failed to react . appropriate'snd,O timely N but-soit'einor 'j;q'"Q;/dA' + So@*1' apses appropriately to instrument readings [J4yjfd,2" err %'$' without assistance
- b. [.25 X -
] '(d) Take MANUAL CONIROL of automatic functions when appropriate? j N'( [ 1 h1T Minor delays some Depended on automatic as g g jate prompting necessary actions: required < before overriding prompting to take automatic functions manual control [.25 X - ) , 1 NUREG-1021 ' 25 of 29 ' Revision 8 i i i ) i .g a 1 7, .ES 303 6 Form ES-303-4
- 6. COMMUNICATE AND INTERACT WITH THE CREW AND OTHER PERSONNEL DID'THE. APPLICANT:
( a' ') - : Communicate in a clear. easily-understood manner? f .:s 3 -- 2 / 1,, A', " m ) ' , ', ' Comun1 cations were At times, communi- ; s Communicatiorisfwere [/ 111-timedf < vagueU'p , timely.: clear. and ' . cations were ,' f ;' easy to hear and : confusing hard to ,(, or difficult'to ]'!,1,%gp understand : hear or understand ,' hear'or 'qm, e understand , S '* i, 1 y, .,s * ~+*97 i. Ny D4,5, X ) .. g'?% 'h,, x,3.% (b) Keep crew members and those outside the control room informed of plant status? [ ,5,;::l,, f " ' ',:' ' 6 '^ h '% , , "y 3' [p 2 , d/1 Provided others with Had to beprimpted jdr M'lhh3Ied to provide accurate, pertinent-information throughout: :instandesj gavej^orheinformation in some,, minor [l' d m scenarios incosp)ete or ,1naccurate / idgrmationffg f , :fx ?If,}A / [.35 X = ] g',,"/;> ^>:*>l,1,,,y s s ,A d ,' ' (c) ENSLRE'1tECEPT of lepr',, . k, assily-understood comunications from crew and ']p$ othersij[';6pf , yN :w Reque 3 $[o Q' pa w:w% ys.<p : sx information ^% < Ea)Ted to require %V2 Failed to request 1 orCNfificationwhen M acknowledge infor- needed information: nepessary: understood # mation from others inattentive when conmunications from , information was # provided: failed to 6th6fs i correct serious If I'Fyd['4hF' # misunderstand 1ngs %QtF among crew members 1 [.20 X - ) NUREG-1021 26 of 29 Revision 8 n I -ES-303 7 Form ES-303-4 g 7. DIRECT SHIFT OPERATIONS DID THE APPLICANT: (a)- Take TIMELY and DECISIVE ~ ACTION when problems arose?' i 3 2 1 g Took early remedial Minor instances of Failed to?take ti iy action when necessary failure to take action action:fresulted in within a reasonable 4eterioration of period of time M[, plant conditions, ' rf /~~ f,f [ .20,,X : y y*'AqL,v, h, s,, y s, ~ j (b) Provide TIMELY. WELL THOUGHT OUT DIRECTIONS {that facilitated CREW PERFORMANCi SAFETY;,6f the plant. sfaff'dfrid ! and demonstrated appropriate CONCERN for the(4 f:'h~i public? s- 3 (pM 'y%. nb p'g 2 1 ; l Directives enabled Directives i' inhibited I safe, integrated Minor incorrect.instances trivia @ofc~Jf , m ), safe' performance: crew performance or difficult-to'carr,$ d?f'gtrew h'ad+to explain out orders /' Id T'pN"'.:s 'why orders 'h~oi'Jsh6uld not becould not {f6110wed ,$1, # [.40 X - 3 ~, > s g (c) Sta in a position of OV . DIRgCTIONandGUIDANCE?,ERSIGHTand' y ' provide, ' APPROPRIATE AMOUNT of 34 /r h>d 2 hj *~4 , 1 Stayed 1p'vo15$'but Crer' occasional had to Lost the big picture; notintrusive!'fh request assistance, crew had to repeat-anticipatedtreW',5r73 which' interfered edly request or , i needsandprovidedgyh with h i l provide guidance; guidancewhenyd necessar,y/F'
- 8,garry(t e t/abi ity to y actions failed to verify that yf qP$dp%, directives were yys 1'p correctly implemented h *cp [.20 X -
3 ( SOLICITandIN3kPORATEFEEDBACKfromthecrewtofosteraneffective, team- ,s oriented approach to problem solving and decision making? '[g,?,?t 5', 3 kl* g 1 1 Ni virig process At times, failed to Made decisions with-involve crew in decision out crew participa-prot .as $en appropr g )iate leading .. - making when it would tion or consultation: .to effective team .have been appropriate, crew divisiveness was decision making detracting from team- counter productive oriented approach [.20 X - ] NUREG-1021 - . 27 of 29 Revision 8 ES-303 8 Form ES-303 4
- 8. COMPLY WITH AND USE TECHNICAL SPECIFICATIONS DID THE APPLICANT:
(a) RECOGNIZE when conditions were covered by technical specifications (TS)? , }, Gr' 3 2 }}':; ,<f f(' / ' ;' ' , , .+ Recognized TS Minor errors and f,,g Failed to, correctly limiting conditions for misunderstandings f'(, ze situations operation and action with respect to f,, recogn)d by TS a'nd': Y .. covere ^ ~ statements without use TS applications gn action statements: ; of references ,,' '%;4.#; ^ *(77(( n'. ' '6',% ] [+' sqT.40'5x Gn;' \s (b) LOCATE the appropriate TS quickly and efficiently? %' O' E % >G , :, 1'gs 't "f);l 9* 3 2 >t .c'g .~,ah 'b,, e ,Jy ;4' Located applicable- Had diffic ty locat ,' hYd not locate TS quickly and TS: had,tdsearchjf/g QJ@propriate TS accurately througli,16dex and; body ' ofd6cume*nt ly' ,y p.<A g [.20 X - ] g& A6;;:[k> s hl^h' it :n/M'yf i/ (c) En)s'ifefc,,o, treet C$+PLIANCE, ith TS"andLCO action stateme ~gp 4 3 1 Applied incorrect TS Directives p on correctFunder-(- ,eede ome assistance crew to ensure to situation: standjanfof TS allowed crew to actdEstatements 3,[MsI[iliance violate TS ,p [.40 X - ] , h NUREG-1021 28 of 29 Revision 8 / } Attachment 3 NUREG ' 021 "Oaerator Licensing Examination ; Standards for Power Reactors" Revision 8 NUREG-:.02:. Revision 8 l Operator Licensing Examination ~ Stancarcs for Power Reactors Manuscript Completed: Month 1998 Date Published: Month 1998 k .' Division-of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation U4S..! Nuclear' Regulatory Commission l Washington, DC 20555-0001-i ). Intentionally Blank . NUREG-1021 Iv Revision 8 9 TABLE OF CONTENTS
- ABSTRACT-TABLE OF CONTENTS EXECUTIVE
SUMMARY
ABBREVIATIONS ~ ~ ES-101,
' Purpose and Format of Operator Licensing Examination Standards ES-102 Regulations and Publications Applicable to Operator Licensing ES-201 initial Operator Licensing Examination Process ES-202 Preparing and Reviewing Operator License Applications ES-203 -
[ Deleted] . ES-204 Processing Waivers Requested by Reactor. Operator and Senior Reactor Operator Applicants ES-205 Procedure for Administering the Generic Fundamentals Examination Program ES-301 Preparing initial Operating Tests' ES-302 Administering Operating Tests to initial License. Applicants ES-303 Documenting and Grading initial Operating Tests
'ES 304 lO;:;;;d]
ES-401 L , Preparing initial Site-Specific Written Examinations ES 402 Administering initial Written Examinations - ES403 Grading initial Site-Specific Written Examinations 50 4^A [0;;;;;d] ES-501-Initial Post-Examination Activities ES-502.'
. Processing Requests for Administrative Reviews and Hearings After
' initial License Denial NUREG-1021 v Revision 8 6
c TABLE OF CONTENTS ES-601 Conducting NRC Requalification Examinations ES-602 Requalification Written Examinations ES-603 Requalification Walk-Through Examinations ES-604
' Dynamic Simulator Requalification Examinations ES-605 License Maintenance, License Renewal Applications, and Requests for Administrative Reviews and Hearings
- ES-701
' Administration of initial Examinations for Senior Operators Limited to Fu Handling -
ES-702 Administration of Requalification Examinations for Senior Reactor Operators Limited to Fuel Handling APPENDIX A Overview of Generic Examination Concepts APPENDIX B Written Examination Guidelines APPENDIX C Job Performance Measure Guidelines APPENDlX D Simulator Testing Guidelines APPENDIX E Policies and Guidelines for Taking NRC Examinations
- APPENDIX F Glossary I
i NUREG-1021 vi Revision 8 I
EXECUTIVE
SUMMARY
Title 10, Part 55, of the Code of federa/ Regulations (10 CFR Part 55) requires applicants for reactor operator (RO) and senior reactor operator (SRO) licenses to pass a written examination and an operating test that are developed and administered in accordance with 10 CFR 55.41 and 55.45 or 55.43 and 55.45, respectively. Although license examiners from the U.S. Nuclear
' Regulatory Commission (NRC) have historically prepared all of the licensing examinations using facility-provided reference materials. fJ;=th;:s;, th; r;gu:::ba; do n;; preh bn , the NRC has now amended Part 55 by adding a new section licensees frem-to developing and submitting, upon lapprov(a$55.40) that requires by an authorized representative of facility the facility licensee, proposed examinations for NRC review and approval. ";; 7:n0 b;'::c;;
. thd in!',Ri ' a .nn h;;; th; sp-tb; te, papa dien s;;nbdis; thd pa;b; en n;;p%5% trh for bnbl cprd;,7 lbenbg. Th; !?RO t:x; thi entdba = the, n;;;;;
;;f th; %;nnd epider ag;;::'.; tbn s;m:stbn pm . n ad the rewt ;f the p::d
=;mbd:s; th;t ;; ,7; ;dmbbi;md" b ;;;:rdux ;;Mh 0;nrb L;1.T (OL) 05 00, "Ch;..
b the, O p i d er U c x b ; I m 15,1005. Fu;th;=;;;, th;'tinO a; ;g;; ;;. thd th; %;;;.tj na ns';;;gr%m, im'dd;d ,^.;;st Sty vinh pbat ;ydeme ad preadarn (abC;;is that
;f !?CO Sanx :nm =m) ;;;ll shs;; th; ;;nt: =l:d%y ;f th; :smbdb .; sd th;
;"; bay ;f the:r d;;;bpmst.
I:::::j % sus ;; :nsac;;d t; d;nbp th; Unneb; n;m rin 67 th:: ;;;n RO :nd ERO ;pp:: sat (bebd ..; ths; : mn;d t; E;; hadts); h;;;;;;r, . ..., ... ~ . ..b; th;
;uthedti t; d;;;bp th; :nmbdbne v;h;n damed san;ri. In lieu of requiring a specific facility licensee to pre wntten examinations, pare the examinations and tests or to proctor and grade the site-specific the NRC may elect to perform those tasks in order to maintain the proficiency of its examiners or if it has reason to question a licensee's ability to prepare acceptable examinations.
Facility licensees are expected to develop and submit their proposed examinations based on the guidelines and instructions contained herein. Section ~107 of the Atomic Energy Act of 1954, as amended, re operators. Therefore, quires the Commission to prescribe uniform licensing conditions for the NRC discourages facility licensees from using testing methodologies
. that do not conform to the policies, procedures, and practices defined in this NUREG.
Nevertheless, facility licensees may propose d;;bt:s; fem attematives to specific guidance in NUREG-1021, and the NRC will review and rule on the acceptability of the dewatenealtematives. The NRC will make a reasonable attempt to administer all license examinations on the dates requested by the facility licensees. At times, however, resource limitations may compsi the staff to pnoritize its examination review and development activities based on need and safety considerations. Facility licensees are strongly encouraged to schedule their initial license
. examinationsand to resolve any applicant efigibility questions with their NRC regional office before commencing a license training class.
For Revision 8, NUREG-1021 was reorganized to more clearly identify the various orgaruzational responsibilities, it incorporates the methodolo pilot examination program described in GL 95-05, n ;;" s;herlmpr;;;m ;gy and lessons leamed from the sb ;nd eierifieste= .- xmme.-ded by bdadri gr;;p, %;med epid; ;, sd !?CO s;.mb;,a ad 1 managere and changes made in response to the public comments on interim Revision 8 solicited in connection with the regulatory amendment noted above. This revision also formally , implements Revision 1 of NUREGs-1122 and 1123, the " Knowledge and Abilities Catalogs" for 1 pressurized and boiling water reactors, respectively. NUREG-1021 vii I Revision 8 i
EXECUTIVE
SUMMARY
In addition, Revision 8 of NUREG-1021 supersedes Revision 5 of NUREG/BR-0122,
" Examiners' Handbook for Developing Operator I.icensing Written Examinations," dated March 1990.
This document reflects the following significant changes from Revision 7: Abbreviations This list has been added to provide a central location for defining the acronyms and abbreviations used throughout this NUREG. ES-101 - No significant changes. ES-102 - Me ; gr.l':c.;nt cher.;;;.NUREG-1560,' " Individual Plant Examination Program: Perspectives on Reactor Safety and Plant Performance," and NUREG-1600,
" General Statement of Policy and Procedure for NRC Enforcement Actions," have been added to the list of documents applicable to the operator licensing program.
ES-201 Resources permitting, the NRC will give each facility licensee an opportunity for one initial operator licensing examination per reactor type..or site per fiscal year. Additional examinations may be scheduled as the woridoad permits. Each NRC region will prepare at least one examination per calendar year. Facility licensees th;; ;-ll; th; :nic.:n:tl;r.; dll generally conduct the following activities based on the guidance in NUREG-1021: Observe various examination security and integrity criteria, including restrictions on which personnel can participate in developing the examinations, physical security expectations and considerations, and limits on the use of examination banks.
~
At least 60 days before the scheduled examination review date, prepare -
- and submit for NRC review and comment an integrated examination
' outline, in accordance with ES-301, ES-401, and the associated quality assurance checklist.
At least 30 days before the scheduled examination review date, prepare and submit for NRC review and comment the complete examination, in accordance with ES-301 and ES-401, along with a statement indicating the source of each test item proposed for use on the examination. Make examination changes as agreed upon with the NRC. Facility lik:ensees shall designate a point of contact to work with the NRC chief examiner, n ;; : ;; . An authorized .T.;r.;;;.T.;r.: .epresentative of the facility licensee shall feview eneFapprove the submittals before sending them to the NRC for review and comment. The standard also discusses the restrictions on the
' activities of instructors and supervisors who have knowledge of the examination
' contents.
NUREG-1021 viii Revision 8
E I; I EXECUTIVE
SUMMARY
ES-201 i Instructors are no longer restricted from preparing written examination questions l based solely on the amount of time they spent training the applicants. ; The amount of reference material requested from the facility licensee will be adjusted based on the NRC's level of involvement in the examination development process. ES-202 The eligibility criteria for senior reactor operators limited to fuel handling (LSROs) have been moved from ES-701 to ES-202. To make the standard conform with 10 CFR 55 and current practice, ES-202 now l includes a provision requiring that facility licensees submit a written request to have a license examination administered to an applicant. - If more than six months pass since an applicant's medical examination, the facility must certify that the applicant has not developed any condition reportable under 10 CFR 55.25. The regions will verify that an applicant's name does not appear on the " Restricted Individuals List" before accepting the application. The requirement for five significant reactivity manipulations has been clarified. ES-204 The provision for LSROs to be licensed at more than one site has been moved from ES-701 to ES-204. The regions may, under certain circ" umstances, waive the requirement for an examination for applicants that were previously licensed at the same facility. The regions may, under the circumstances specified in 10 CFR 55.31(a)(5), accept an application and administer an examination before the applicant completes the required control manipulations. The regions may waive the requirement for a new medical examination for up to two years from the date of the last examination if the facility licensee certifies that the~ applicant has not developed any medical condition reportable under 10 CFR 55.25. ES-205 Facility licensees should notify the Headquarters Operator Licensing Branch (OLB) if they must modify their previously submitted registration letter for the generic fundamentals examination by adding or deleting a person. ES-301 Dominant accident sequences, as determined by the facility licensee's probabilistic risk assessment or individual plant examination, should be considered for sampling during the operating test. l A site-specific task list may be used to supplement or override, on a case-by-case basis, selected individual items in the NRC's knowledge and abilities catalog. NUREG-1021 ix Revision 8
EXECUTIVE
SUMMARY
ES-301 The instructions for developing the operating test outline and the final test items have been separated to facilitate their sequential preparation, review, and approval. Generic guidelines (i.e., those that apply to both initial and requalification examinations) for developing the walk-through and dynamic simulator tests have been relocated to Appendix C, " Job Performance Measure Guidelines," and Appendix D " Simulator Testing Guidelines," respectively. No more than 80 percent of any applicant's walk-through test may be taken directly from the facility licensee's item bank without significant modification, and no more than 30 percent of the walk-through may be repeated from the last NRC license examination at the facility. Closed-reference questions should not exceed 30 percent the total questions on either walk-through test category. A quality assurance checklist has been included as an attachment to this standard to highlight various criteria and promote consistency. Each applicant's dynamic simulator test shall include at least one new or significantly modified scenario that the applicant has not had the opportunity to rehearse or practice. A quality assurance checklist has been included as an attachment to this standard to promote consistency by highlighting and suggesting target ranges for various criteria, including simulator critical tasks. The target ranges are based on a study of simulator scenarios used during past initial operator licensing examinations. No dynamic simulator scenarios orjob performance measures will be repeated on successive days, :nd n; .T.;;; then 30 p;c;;n ;f th; j.. y.........._ .......... th;n #;; . th; Mak ;;"' k . ag rM #;.T. ;n; d;y :: th; n;;;. ES-302 NRC examiners may use additional surrogate operators to augment the simulator crews if the technical specifications require the facility licensee to operate with more than two ROs in the control room. A shift technical advisor (STA) may also be used consistent with facility operating practice. SRO-upgrade applicants in an RO position do not have to be monitored individually by an NRC examiner during the simulator test. The facility licensee and NRC chief examiner should confirm that the simulator instructor's staten, programmers' tools, and extemal interconnections do not compromise the integrity of the operating test. Appendix D briefly describes a number of vulnerabilities. The practice of conducting an exit briefing with the facility licensee after the operating tests are complete has been adopted as policy. NUREG-1021 ' x Revision 8
EXECUilVE SU!tiARY ES-302 : The operating test briefing for the applicants has been moved to Appendix E, i
" Policies and Guidelines for Taking NRC Examinations."
4 The regions may ' delay the operating tests for up to 30 days after the written I
> examinations if necessary to achieve a quality product.
ES-303 The simulator operating test grading guidelines for errors having serious safety consequences (including critical tasks) have been clarified. Missing a critical task does not necessarily mean that an applicant will fail the simulator test, nor does success on every critical task prevent the examiner from recommending a failure if the applicant had other deficiencies that, in the aggregate, justify the failure based on the competency' evaluations. The ~ simulator operating test documentation requirements have been increased; examiners must now briefly describe the error that the applicant made to justify a ; grade of"2" on any rating factor. i The applicants' responses to prescripted JPM follow-up questions will be evaluated
. based primarily on safety-significance.
ES-401 - This standard now includes instructions,'an example method, and forms for use in
.' systematically developing the written examination outline. References to
- NUREG/BR-0122, " Examiners' Handbook for Developing Operator Licensing
' Written Examinations," have been deleted.
References for guidance in developing multiple choice test items have been changed from NUREG/BR-0122 to Appendix B, " Written Examination Guidelines." Facility licensees shall submit an outline at least 60 days before the examination review date, followed by the " ready-to-use" examination at least 30 days before the
- examination oview date.
A site-specific task list may be used to supplement or override, on a case-by-case 4
; basis, selected individualitems in the NRC's knowledge and abilities catalog; a site-
, specific task list may not be used in place of the entire catalog.
This standard n' ow includes several new criteria developed to ensure the integrity of examinations developed by facility licensees. These criteria include limits on the number of questions that can be taken directly from the facility licensee's item bank a-7 .or can be repeated from earlier quizzes and examinations.
~ ln an' effort to maintain examination quality and consistency, eHeme&50 to 60
- percent of the questions on the examination shall tesFbe written at the
. comprehension / analysis level.
NUREG-1021' x1 Revision 8 .m
EXECUTIVE
SUMMARY
n ES-401_ As a final check for technical accuracy, facility licensees should consider administering the examination to one or more previously uninvolved licensed personnel (under security agreements). ES-402 Facility licensees will generally administer the written examinations after they are approved by the NRC. The region may delay an examination for up to 30 days after the operating tests if necessary to achieve ,a quality product. The facility licensees will document for subsequent review by the NRC any questions posed by and answers provided to the license applicants during the examination. If NRC examiners are on site, they may periodically monitor the administration process. The guidelines for briefing the applicants who will take the examination have been moved to Appendix E. The time permitted to take the written examination has been extended from four to five hours.
' ES-403 Facility licensees should collect and consider any questions and comments made by the applicants after the examinations are administered. Facility licensees that prepare and administer the written examinations will grade the examinations, review the grading, evaluate the applicants' performance, and submit the results to the NRC for review and approval. The facility licensee shallJustify all recommended question deletions and changes to the answer key.
The discussion of examination grading quality assurance has been moved to ES-501. {
. ES-501 This standard summarizes the documentation that facility licensees are expected to provide to the NRC if they develop and administer (in the case of the written) the license examinatens.
' in addition, the standard now summarizes the post-examination quality assurance review process that was previously contained in ES 403.
1 if a facility licensee recommends deleting or changing five percent or more of the questions on a written examination that it developed, it may be asked to explain why the changes were necessary. The regional offices ws-may delay issuing the licenses for applicants who pass the written examination with insufficient margin to ensure that the licensing decision will be sustained if additional questions are deleted or changed upon appeal. TheIxamination report shall address any significant problems that the region or
' facility licensee encountered in developing the examination.
NUREG-1'021 - xii Revision 8
EXECUTIVE
SUMMARY
ES-501 The record keeping requirements, including the submittal of proposed examinations to the public document room (PDR), have been revised to reflect the new examination process. ES-502
' For those denials that are sustained by the regional office on preliminary review,
' the Chief of the Operator Licensing Branch will determine whether to convene a panel or evaluate the appeal intemally. Appeal panels, when required, will normally consist of three certified examiners, one of which will be designated chairperson.
ES-601
.The NRC will continue to monitor licensees forindications of undue stress during requalification examinations, however the stress feedback forms have been eliminated.
ES-602 .- References to NUREG/BR-0122 have been changed because the guidance for developing multiple choice test questions is now in Appendix B. Because the guidelines'previously documented in Attachment 1 to ES-602,
" Policies and Guidelines for Taking NRC Written Examinations," are generally the same for initial and requalification examinations, they have been consolidated in
_ Appendix E. i No'more than 50 percent of the questions on the examination may be duplicated t from any past examination or combination of examinations during the current requalification training cycle. ES-603 Attachment 1 to ES 603, " Guidelines for the Development and Use of Altemate ' Path JPMs"; Attachment 3, " Walk-Through Evaluation Guidelines"; Form ES-603-1, i "JPM Quality Checklist"; and Form ES-803-2, "JPM Worksheet," have been moved to Appendix C because they apply to both initial and requalification examinations. Attachment 2 to ES-603, " Briefing Checklist - System Walk-Through," has been
. moved to Appendix E.
1 ES-604 Attachment 1 to ES-604, " Critical Task Methodology," and Attachment 3,
" Quantitative and Qualitative Scenario Attributes," have been moved to Appendix D because they apply to both initial and requalification examinations. Attachment 2,
" Dynamic Simulator Briefing Checklist," has been moved to Appendix E.
ES 605 The policy on standing proficiency watches and renewing inactive licenses has been clarified.
' The regions may, under certain circumstances, authorize an operator to temporarily
^ suspend participation in the facility licensee's requalification training program.
ES-70[ ' The eligibility criteria for LSROs have been moved to ES-202, and th LSROs to be licensed at more than one site is now discussed in ES-204. NUREG-1021 xiii Revision 8
EXECUTIVE
SUMMARY
ES-701 The standard has also been edited to clarify the differences between the full-scope SRO and the LSRO examinations. The number of systems tested in Category B of the operating test has been decreased from six to five, and the requirement to test a normal evolution during each of the two discussion scenarios in Category C has been eliminated. The number of subject areas to be evaluated with questions when it is not practical to conduct or simulate a job performance measure has been decreased. ES-702 This standard has been edited to clarify the differences between the full-scope and the LSRO requalification examinations. . Whenever possible, the facility licensee should include an LSRO on the requalification examination team.
. Appendix A This new appen& discusses the generic examination concepts that play a role in the operator licensing process. It includes much of the information that was previously contained in NUREG/BR-0122 as well as discussions of new topics that have a bearing on the level of difficulty of an examination.
I Appendix B This new appendix incorporates the guidance for developing written test questions I that was previously contained in NUREG/BR-0122. It focuses primarily on multiple-choice questions, the only type currently permitted on the initial operator licensing
)
examination, and includes examples to illustrate various psychometric concepts. ' Appendix C This new appendix summarizes the guidelines conceming job performance measures that apply to both initial and requalification examinations. Much of this infermation was previously contained in Attachments to ES-603. There are no
. significant policy changes.
Appendix D This new appendix summarizes the dynamic simulator scenario guidelines that apply to both the initial and requalification examination programs. Much of the information was previously contained in ES-301 and Attachments to ES-604. Appendix D also describes a numbsr of simulator security vulnerabilities (related to features of the instructor's station, programmers' tools, and extemal intercontmetions) that NRC examiners and facility personnel should consider when preparing and administering operating tests, i t Appendix E This new appendix summarizes all of the policies and guidelities applicable to examinees who will be taking an initial or requalification examination. The information was previously contained in ES-302, ES-402, ES-602, ES-603, and ES-604; there are r.o significant policy changes. The policy on examining senior reactor operator upgrade applicants on the control boards has been added to the simulator test bnefing list.
' NUREG-1021 xiv Revision 8
r L EXECUTIVE
SUMMARY
. Appendixthis F This NUREG.
new appendix provides a centrallocation for defining terms used th Any reference to the plant's technical specifications includes the plant's other technical requirements documents, whether stated or not. l
/ \
1 1 k l 3 i i i
}
f
- NUREG-1021 xv. Revision 8 :
*=g l
l l l-I L.
w ABBREVIATIONS y
- AC. attemating current '
' ADS : automatic depressurization system AFW- auxiliary feedwater ANSl/ANS' AO American National Standards institute /American Nuclear Society
' auxiliary operator -
;AOP! abnormal operating procedure APRM : average power range monitor ARP alarm (or annunciator) response procedure .- i j
ATWS[T) . . anticipated transient without scram [ trip) B&W Babcock and Wilcox-BWR' boiling water reactor
' CAL- confirmatory action letter ~
CCW component cooling water-
'CFR. Code of FederalRegulations' CRD- control rod drive .
'CRT'- criterion-referenced test CT. critical task
,CTMT i containment CVCS. : chemical and volume control system DAS. dominant accident sequence
,DC. . direct current DHR- . decay heat removal ,
DRCH .j Division of Reactor Controls and Human Factors ' EAL' emergency action level '
-ECA emergency contingency action (proce. dure)
ECCSq emergency core cooling system
. ECP estimated critical position "
EDG. . emergency dieselgenerator EHC electrohydraulic control EOP.
- emergency operatmg procedure j
EPIP; . j emergency plan implementing procedure - EQB '. examination question bank ES- examinston standard . ESF !
,. engineered safety feature FHE. 4 fuel handling equipment FRP, functional recovery procedure i- c .FSAR-
' final safety analysis report GFE (generic fundamentals examination GL: '. < genericlatter *
- GUIL graphic userinteiface; HP. health physics -
L HPCl< high pressure coolantinjection HPCS- high pressure core spray ' lHVAC . heating', ventilation, and air conditioning IC instrumentaten and control NUREG-1021 - xvi l; ' Revision 8 i
l I ABBREVIATIONS INPO Institute of Nuclear Power Operations IP inspection procedure IPE individual plant examination IRM intern-~jiate range monitor JPM job performance measure ; JTA job task analysis K/A knowledge and ability , KSA knowledge, skill, and ability l LCO limiting condition for operation , LER licensee event report LOCA loss of coolant accident LPCI low pressure coolant injection ' LPCS low pressure core spray LPRM local power range monitor LSRO limited senior reactor operator MlP masterinspection plan MSIV - main steam isolation valve '
. NEl Nuclear Energy Institute NRC Nuclear Regulator Comrnission ,
NOP normal operating procedure NRR Office of Nuclear Reactor Regulation NRT norm-referenced test ; NWPA Nuclear Waste Policy Act(of 1982) OJT on-the-job training OLA operatorlicensing assistant t OLB Operator Licensing Branch-OLTS operatorlicensing tracking system OMB Office of Management and Budget PCIS primary containment isolation system PDR public document room PORV power-operated relief valve PPR plant performance review PRA' probabihstic risk assessment PWR pressurized water reactor i .QA quality assurance RBM rod block monitor RCA- radiologicar y controlled area RCIC reactor core isolation cooling RG- ~ Reguiatory Guide RHR: residual heat removal RMCS reactor manual control system RO reactor operator ROI report on interaction RM radiation monitor RPIS rod position indication system NUREG-1021 xvii Revision 8
ABBREVIATIONS
~RPS reactor protection system.
RPV reactor pressure vessel
, RWST- refueling water storage tank S
satisfactorf SALP systematic assessment of licensee performance SAT systems approach to training
.SGTS standby gas treatment system SD standard deviation SGTR steam generator tube rupture SI. safety injection .
.SLC standby liquid control
.SME subject matter expert SRO senior reactor operator
-SRP Standard Review Plan SRV safety relief valve STA shift technical advisor '
TDAFW(P) turbine-driven AFW(pump) TS-U technical specification (or other technical requirements document) unsatisfadory UPS uninterruptible power supply W/T walk-through NUREG-1021 xviii Revision 8 k L..
ES-101 PURPOSE AND FORMAT OF OPERATOR LICENSING EXAMINATION STANDARDS T A.L PURPOSE-L Title 10,'Part 55, of the Code of Federa/ Regulations (10 CFR Part 55) requires that applicants i for reactor operator (RO) and senior reactor operator (SRO) licenses pass written examinations and operating tests (both initially and for requalification). Moreover, the regulations mandate that the license examinations must be developed and administered in accordance with 10 CFR
~ 55.41 and 55.45 for ROs, or 10 CFR 55.43 and 55.45 for SROs.
The " Operator Licensing Examination Standards for Power Reactors"(NUREG-1021) establish the procedures and practices for administering the required initial and requalification written examinations and operating tests. These standards describe the provisions of the act and regulations on which the program is based. They also ensure the equitable and consistent administration the r'egulations.' of examinations to all applicants and licensed operators at all facilities subject to B. FORMAT Each standard explains the rules, procedures, and practices for a particular aspect of the program. For ease of reference, each examination standard (ES) is assigned a three-digit number, and related standards are grouped together in the sense that standards beginning with the same digit apply to related aspects of the program, as follows: ES-1xx- General ES-2xx - Initial pre-examination activities
. ES-3xx -Initial operating tests ES-4xx - Initial written examinations
- ES-5xx - initial post-examination activities ES-6xx - Requalification' examinations ES-7xx - Fuel handling examinations
.w .;
i t, NUREG-1021 1 of 1 Revision 8
f ES-102 REGULATIONS AND PUBLICATIONS APPLICABLE TO OPERATOR LICENSING
~ A. PURPOSE !
i This standard lists the U.S. statutes and the regulations of the U.S. Nuclear Regulator Commission (NRC) that establish the requirements. for conducting operator licensing examinations. ' It also identifies the regulatory guides and NUREG reports that establish the procedures for implementing the regulations and administering the examinations, as well as ; standards of the American National Standards institute /American Nuclear Society (AN that may provide additional guidance. Regulatory guides (RGs), NUREG reports, and industry standards are not requiremen as specified in Commission orders or as committed to by the facility licensee. The appropr! revisions should be consulted as referenced in the facility's FSAR or approved training p The following paragraphs summarize the latest revisions of the 3e documents. B. STATUTES-
- 1. Atomic Enemy Act of1954 l
t Section 107 of the Atomic Enemy Act of 1954 (42 U.S.C. 2137), as amended, requires that the NRC prescribe uniform conditions for licensing individuals as operators of production and utilization facilities, determining the qualifications of these individuals, and issuing licenses to such individuals.
- 2. Nuclear Waste PolicyActof1982 L
Sechon 306 of the Nudear Waste Policy Act of 1982 (42 U.S.C.10226, 96 Stat. 2201 at 2262 - 2263) directs the NRC to establish requirements goveming (1) simulator trainin 1 for applicants for operator hconses and for operator requalification training programs, '
. NRC administration of requalthcation examinations, and (3) operating tests at civilian nuclear power plant simulators.
/ ,
C. REGULATIONS
- 1. 10 CFR Part 2. Ruta = of Prmreien 1
l The regulations in 10 CFR Part 2 govem the conduct of all preceedings under the Atomic Enemy Act of 1954, as amended, and the Enemy Roomanization Act of 1974 with ringard to (a) granting, suspending, revoking, amending, or taking other action with respect to any license; (b) imposing civil penalties; and (c) public rulemaking. 10 CFR 2.103(b)(2) establishes the applicant's right to demand a review of a proposed license denial, and defines the applicant's appeal and hearing rights. l NUREG-1021 1 of 6 1 Revision 8 a
y . ES-102 I Subpart G, " Rules of General Applicability," govems all adjudications initiated by the issuance of an order to show cause, an order designating the time and place of a hearing requested by a person charged with a violation, and a notice of hearing.
- Subpart L, " Informal Hearing Procedures for Adjudications in Materials and Operator Licensing Proceedings," govems proceedings for the issuance, renewal, or licensee-initiated amendment of an operator or senior operator license.
- 2. - 10 CFR Part 9. Public Records The regulations in 10 CFR Part 9 prescribe the rules goveming the NR,C's public records that relate to any proceeding subject to 10 CFR Part 2.
Subpans A and B describe and implement the requirements for balancing the public's ' rights to information under the Freedom of /nfonnation Act and the NRC's responsibility to protect personal information under the Pr/vacy Act. Subparts C and D implement the provisions of the Sunshine Act, conceming the opening of Commission meetings to public observation. They also describe the procedures goveming the production of agency records, information, or testimony in response to subpoenas or demands of courts or otherjudicial authorities in State and Federal proceedings. 3. 10 CFR Part 20 Standards for Protection Aasinst Radi=hn The regulations in 10 CFR Part 20 establish standards for protection against radiation hazards arising from licensed activities. Some of the materialis appropriate for inclusion in the examinations administered to candidates for RO or SRO iicenses.
'4: 10 CFR Part 50 I !ceneins of Pradm* ion and Ui;;estion Fad:h 10 CFR 50.34(b)(8) requires that the final safety analysis repod (FSAR) include a description of the operator requalification program. That description forms the basis for the inspection, audit, and approval of requalification programs.
10 CFR 50.54(6-1) requires facility licensees to implement an operator requalification program that meets the requirements of 10 CFR 55.59(c) within 3 months after receiving a facility operating license. Notwithstanding the provisions of 10 CFR 50.59, the licensee may not decrease the scope ofits approved requalification program without
. authortzabon from the Commission.
10 C' FR 50.54(k) - (m) contain regulations restricting control manipulations to licens operators. These regulations are conditions of all facility licenses issued under 10 CFR Part 50. NUREG-1021 2 of 6 Revision 8
ES-102 110 CFR 50.74 requires facility licensees to notify the Commission within
. is a change in the status of a licensed RO or SRO.
5. 10 CFR Part 55. Ooerators' Licenses 4 10 CFR Part 55 is the implementing regulation that establishes the requirements and the regulatory basis for licensing and requalifying ROs and SROs. , D. REGULATORY GUIDES 3 1, i Raoulatorv Plants." Rev. 2.Guide April 19871.8. " Qualification and Trainina of Personnel for Nuntaar Po Section C.1 of this RG currently endorses, with exception, ANSI /ANS 3.1-1981, "American National Standard for Selection, Qualification, and Training of Personnel for
' Nuclear Power Plants" (effective March 31,1988).
The NRC is. currently reviewing, and is expected to endorse, with exception, the 1993 version of ANSI /ANS 3.1. Section C.2 endorses, with exception, ANSI /ANS N18.1-1971, "American National Standard for Selection and Training of Nuclear Power Plant Personnel." 2. Reaulatorv Guide 133. "OnaMv Assurance Proui-n, Raa*edents -Onerations" Appendix A to this RG contains a list of typical procedures for pressurized water reactors and boiling water reactors. 3. Reaulatorv Pc;;;c Plant" Guide 1.114. " Guidance on %ino an Operator at the Centicis of a Nuclear 1 This RG describes a method a.cceptable to the NRC staff for complying with the
' Commission's regulations in 10 CFR 50.54(k)- (m), which require the presence of an '!
RO at the controls of a nuclear power unit and an SRO in the control room from which ! the nuclear power unit is being operated. )
- 4. - !
Raoulstorv Gn* 1.134. "*.'. dicel Evalustlen of Licer.ud Personnel for Nue!==r Pc;;c- ; Plante." Rev. 2. Anril 1987 ' This RG currently endorses ANSI /ANS 3.4-1983, " Medical Certification and Monit of Personnel Requiring Operator Licenses for Nuclear Power Plants." The RG is beii 1 revised to endorse the 1996 version of the standard.
'v
- 5. . e I e
RaaMMuida 1.149. "Nur'
' Uc-n=e 1:-=ninstiens." Rev. 2. Anril 1000 st Power Plant Simulation Fadh for Use in O
' This RG currently endorses, with exception, ANSI /ANS 3.5-1993, " Nuclear Power Plant Simulators for Use in Operator Training and Examination." i NUREG-1021. 3 of 6 Revision 8
ES-102 E. - NUREG REPORTS
- is NUREG-0660
' May 1980 Vol.'1. "NRC Action Plan Develooed as a Result of the T item I.A.4.2 of this document describes the guidelines for long-term simulator upg 2.
hIUREG-0737. " Clarification of TMl Action Plan Reauirements " Novembe This document clarifies the following action plan items which are intended to upg the training, licensing, educathan, and experience of operators on the basis of experience gained from the accident at Three Mile Island, Unit 2: Item I.A.2.1, "Immediate Upgrading of RO and SRO Training and Qualifications" item 1.A.2.3, " Administration of Training Programs" - Item 1.A.3.1, " Revised Scope and Criteria fer Licensing Exams" Item 11.B.4, " Training for Mitigating Core Damage" 3. NUREG-0800. " Standard Review Pleri for the Review of Sefete Analysis Resorts for l
- Nuclamr Power Plerai LWR E* on " July 1981_
- Section 13.2, " Reactor Operator Training," describes the training and licensin operators and identifies information to be submitted by applicants for construction permits and operating licenses. 4. NUREG-1122 pra .d.w W :.-p:2.".. "Kr.c:/- 'w arid Armha Catsisc for Nue!=== Power Pls i" Rev.1. Anauet 1995 This docun nt provides the basis for developing content-valid licensing examination for operators at pressurized water reactors (PWRs). It contains knowledge a (K/A) statements that have been rated for their importance to ensuring that thei operated public. in a manner consistent with the health and safety of plant personnel and the !
- 5. - NUREG-1123 "Knc='
r'a= ar.d Atake n=+=! -: for Nor'==r Power P:sid Ossrmiers:
" Rev.1. Anao*+ 1005
+
1 This document provides the basis for developing content-valid licensing examinatioi for operators at boiling water reactors (BWRs). It contains K/A statements that have! ! been rated for their importance to ensuring that the plant is operated in a manner consistent with the health and safety of plant personnel and the public. I i NUREG-1021.
. 4 of 6 i Revision 8 Js .
i 1 ES-102 6. NUREG-1291. "BWR and PWR Off-Normal Event Descriotions." November The reactor event descriptions in this document provide a reliable, performance-based source of information that examiners may use to design simulator scenarios that will be a valid test of an applicant's ability to safely and competently perform alllicensed duties and responsibilities.
.7.
NUREG-1560. " Individual Plant Examination Procrem: Persnectives on Reactor Safety and Plant Performance" This report provides perspectives gained by reviewing 75 individual plant examination (IPE) submittals pertaining to 108 nuclear power plant units. Chapter 13, " Operational 6 Perspectives," is of particular interest because it identifies a number of important human actions that should be considered for evaluation on BWR and PWR licensing and
- requalification examinations.
l 8. NUREG-1800. " General Statement of Policy and Pro = dure for NRC Enforcernent A*.qs-
- This report addresses the NRC's expectations regarding compliance with 10 CFR 55
" integrity of Examinations and Tests," and possible enforcement actions against parties subject to that regulation (i.e., Part 55 license holders and applicants and Part 50 licensees).
9. NUREG/BR-0122. "Fareiners' Handhaok for D==lsoino Ocerator Licensino W Examinations." Rev. 5. Merch 1990 This document, which presented a procedure for systematically constructing content-valid licensing examinations for nuclear power plant operators, has been incorporated into the examination standards in NUREG-1021, Rev. 8. It may be used for historical perspective, but is no longer used for developing examinations. ,
' F. INDUSTRY STANDARDS 1.
ANSI Peresanel/ANS for 3.1. Nir" ="Arnerbn National Standard for E:l:*.n. Qualificatian and Trainin Pc;;r P ente" i This standard provides criteria for selecting and training nuclear power plant employee performing a variety of functions at various levels of responsibility (e.g., managers,
' supervisors operators, and techni i c ans). RG 1.8, Revision 2, endorsed, with exceptions, the 1981 version of the standard; the 1987 version was never endorsed by the NRC; the 1993 version is currently under review and is expected to receive a qualified endorsement from the NRC.
NUREG-1021 5 of 6 Revision 8 4 I e
E j 1 ES-102 2. ANS of 3.2Power Nuclear (ANSI N18.7-1976L Plants" " Administrative Controls and QA for the Ooer This standard provides guidance and recommendations for administrative rules of prac tice 1.33, and related subjects and for preparing procedures and audit programs. See RG
- 3. '
ANSI /ANS 3.41996. " Medical Certification and Monitorina of Personnel Reauirina Ooerator Licenses for Nuclear Power Plants" This standard is'the basic document covering the general health and disqualifying conditions applicable to license app licants and licensed personnel. RG'1.134 current endorses the 1983 version of the standard but is being revised to recognize the newer document. 4. ANSI /ANS 3.5-1993. " Nuclear Power Plant Simulators for Use in Ooerator T This standard establishes the minimum functional requirements and capabilities for nuclear power plant simulators for use in operator training. Revision 2 of RG 1.149 endorses this standard,'with exceptions. ~ NUREG-1021 6 of 6 - Revision 8
-- ~ ~
y l b L ES-201 INITIAL OPERATOR LICENSING EXAMINATION PROCESS A.: . PURPOSE-This standard describes the activities that must be completed to prepare for initial operator
' licensing examinations (including written examinations and operating tests) at power reactor
- facilities. it includes instructions for scheduling and coordinating examination development, assigning NRC examiners and facility personnel, maintaining examination security, and obtaining reference and examination materials from the facility licensee.
- B. ' BACKGROUND-
- Title 10, Part 55, of.the Code of Federal Regulations (10 CFR Part 55) requires'that applicants for reactor operator (RO) and senior reactor operator (SRO) licenses pass a written 1
. examination and an operating test. The regulation requires power reactor facility licensees to:
- (1) prepare the site-specific written examinations and operating tests, (2) upon approval by a! '
' authorized representative of the facility licensee, to submit the examinations and tests to the NRC for review and approval, and (3) to proctor and grade the written examinations Moreov the regulation requires that the license examinations be developed and administered in accordance with 10 CFR 55.41 and 55.45 for ROs or 10 CFR 55.43 and 55.45 for SRO' L nn;; ;:-zmir; ; fam 0; 72".O h;n h:d;.-:;.l y pre;;:ed ;;; ch; ::xn;; .; nem nd:w;
; n; f::"2j p ;;if af;;na md.fi. tiendhin, ia - ;i'_ x; de ad pahl' f;d:. i
- sans farc. wil. ;;rd :d- :fc; - m:nd:er; f;r ??"O 7;^;;;;; ;r.d ; , e;;l. Th; 7 ".O td:;;;'.hd_<__e_,__,_m,_.____._.,___.__
fs:::j faennx raw h;n th; ex; f..; ^; pr:;::; enm:n:;::; .;'hd pre;;d; a ___m
,m _ m . <____.m<_e_,__
______.,,....._,__....__...... . .., , , _ _ _ _ _ ... _. ,,. . .m _ _. ._ _ _ _ _ _, .m.. _ ll-x x 2 - rder agr"'.s :a :: zm:r;;;;.., .pce;;; . ....=_m_
- c. md 'h; raz": ;f 74 p :d s;m 7.d a; 2.2 m;; .d...:i'_ red n ;-r:.-/ r.x ;/ h C=;d; L;;; (OL) 05-00, "Obzgs in :he Op;rder L:sra:n;
.,_______.=_m,_-....
.";;,i.cc.," Ii I .";;;z 15,1000. "_~it'er cea, th; "O ;x;n ; th 24 fx::.t ,
m _ - _ ..........,... .r ....,.. .... .m _ _ m,,..._._ v,_._,.._.. ___._.._.___ i
...v.,,,,,
.. _ ,_ .<_._,m.m e_____
... . ..,..... . ... ~ .. - .. -
g;;{ es}};;':: stenx 0; sc/,.nP;:'!f::j ;f th; :nm:nd:a; nd 'h; ;" deray ;f 'hdr
....,r.......
i Facility licensees ^'.;; ;'- x ^; p; "^';: ;h;ll are expected to develop and submit the examinations in accordance with the instructions contained herein. The NRC retains the authority to develop the examinations on a case-by-case basis if it loses confidence that a facility licensee will develop examinations upon which the NRC can base'its licensing decisions.
. The NRC will also props; :iif xic.:n:2.; n ; ar.ds. hn:; at least one examination
; por region per calendar y' ear to certify new examiners, as required, and to maintain examiner proficiency. '
.v'
- y
. . +7 The NRC's goal is'to provide each facility with'an opportunity for at least one initial operator licensing" examination per fiscal year. Additional examinations may be scheduled on a case =by- . case basis if NRC resources are available. Examinationr for fewer than three applicants should be scheduled only under extenuating circumstances such as a shortage of licensed Ros or SROs at the facility. if a facility licensee has fewer than three license applicants, the examinations may be delayed until more applicants are trained.
NUREG-1021 1 of 26 Revision 8 i
7 ES-201
' Other pre-examination activities, such as submitting and reviewing license applications and . eligibility waivers and administering the generic fundamentals examination program, are addressed in ES-202, ES-204, and ES-205. Specific instructions for developing, administering, and grading the written examinations and operating tests are found in ES 401 through ES-403 and ES-301 through ES-303, respectively. Post-examination administrative activities, including management review of the examination results and preparation of examination reports, are discussed appropriate. in ES-501. Cross-references to each of these standards have been made where C. . RESPONSIBILITIES Facility licensees and NRC staff should use Form ES-201-1, " Examination Preparation Checklist," to track the examination preparations. As noted on the form, the target due dates can be adjusted as necessary to accommodate a given situation. The NRC chief examiner will initial the items as they are completed and ensure that the original form is retained for the master examination file (refer to ES-501).
- 1. Facility Licensee
'a. The facility licensee shelHs expected to apprise its NRC regional office of changes in its examination requirements.
W.; 'sl:P.i henz: 2;;ld r::;:-nd h ;...;...; t; th; !?CO'; :nna;l :d.T.b: .. ..v b;;;r x::::T:.; ;::: . if ;;; ;.ter l;sa;b; nxf: ;nd ne;.'j h !?CO r;;bn;l J
;Os. i' h ;n:r.b;.;ba 7:g::;;.;nt 2.;ng; :lgr.:';;.r.;:j t.T. th;n ;;;2d b h re;pr.n. Facility licensees are strongly encouraged to schedule their examinatxms with their NRC regional office before commencing an initial license training class.
If the NRC determines that a facility is unable to develop acceptable examinations, the examinations could be delayed significantly until sufficient NRC resources can be scheduled to develop and conduct the examinations, or until the facility licensee can develop an acceptable examination.
- b. l In accordance with 10 CFR 55.49, facility licensees s'nd applicants shall not i engage in any achvity that compromises the integrity of any application, test, or examination required by 10 CFR Part 55. Attachment 1 of this ES summarizes several examination security and integrity considerations. NUREG-1600,
" General Statement of Policy and Procedures for NRC Enforcement Actions,"
~
addresses possible enforcement actions against parties subject to the ; requirements in the regulation (i.e., Part 55 license applicants and licensees and I
' Part 50 licensees).-
- c. All facility and contractor personnelinvolved with an sxamination are subject to NUREG-1021 2 of 26 Revision 8
E L i ES-201 the restrictions stated in Section D of this ES. Any questions regarding those restrictions should be resolved with the NRC chief examinsr before granting an individual access to the licensing examination. The facility licensee shall designate a point of contact to work with the NRC chief examiner and assign additional personnel as required to ensure that the examinations are developed, reviewed, administered, and graded in accordance with the applicable examination standards. The faci!ity licensee may use contractors or other outside assistance to develop the examinations, but the licensee bears full responsibility for the product, including conformance with the examination criteria and maintenance of examination security and integrity.
- d. The facility contact shall submit the required reference materials, examination outlines, and examinations, as applicable, based on the level of facility participation. Form ES-201-1 specifies target due dates for the various materials; the actual dates may be adjusted with prior agreement from the NRC regional office.
All examination-specific materials (i.e., the examination outlines and examinations) shall be controlled and protected as sensitive information (refer to l ; Attachment 1). e. The examination outlines and the examinations shall be prepared in accordance with the guidelines in ES-301, ES-401, and ES-701, as applicable. The proposed outlines and examinations shall cover all portions of the license L I examination (written, dynamic simulator, and walk-through) at all license levels relevant to the applicants (RO, SRO, and limited SRO) to be tested.
"c. =^h; dad t"li aprenn;;..;; ( .:., A knowledgeable facility supervisor or manager h;;;;;; c'_tef.;i ;; ;;;;h :n bh;";f th; fad:.;j ::amn) ; hall independently review and approve :ll =;m =.;:a ;ps:T; mM;f;l: the examination outline (s) and the proposed examination (s) before they are sut mitted to the NRC regional office f;r av ;;; nd ;ppr;;;lper item (f) below. !
Ot;:r;;sa ;;h; :;;;;;d 2.; nas; m;;;fe.. .. ... .... ,......... :strad:m;; n;;;;t:s M 0; ::s .a a;;':x..;. m;y av :;; nd appr;;; th; s; c. m%s metenels: In conducting this review, the autheneed-facility span..;;;.J: ;upervisor shall 4 use Forms ES-201-2, " Examination Outline Quality Assurance Checklist"; ES-
?301-3, " Operating Test Quality Assurance Checklist"; ES 301-4, " Simulator
- Scenario Quality Assurance Checklist"; and ES-401-6, " Written Examination Quality Assurance Checklist."
f.
. Pursuant to 10 CFR 55.40(a)(2), an authorized representative of the facility q
licensee shall approve the examination outline (s) and the proposed ! NUREG-1021 3 of 26 Revision 8
ES-201 examination (s) before they are submitted to the NRC regional office for review and approval. The outline (s)~and examination (s) should be forwarded to the regional office with a cover letter signed by the facility representative. i
- g. . ire its examination submittal to the NRC, the facility licensee shall indicate the source of each test item proposed for use on the written examinatica and the
. operating tests. The following information shall be included for each item: )
+
State the source of each item (e.g., is the item taken directly, without changes, from the facility licensee's bank, another facility's bank, the
)
j NRC's bank, or an old NRC exam; is the item a modified version of a ' bank item; or is the item new?). Items that the facility licensee has obtained from another bank and deposited in its own bank may be i treated as " bank" items provided they have an equal chance of being l selected for use on the examination. Items from another bank may be l treated as new items if they have not been made available for review and study by the license applicants and there is no basis (e.g., historical precedent or reciprocal arrangements with the other facility licensee) for the applicants to predict their use on the examination. For those items that are taken directly from the facility licensee's bank, state if and when the item was used on the last two NRC license examinations at the facility or to evaluate the applicants' performance - during their current license training class. For those items that are derived by modifying existing bank items, note
, the changes that were made or submit a copy of the item from which it originated.
- h. The facility licensee shall make its simulation facility available, as necessary, for NRC examiners to prepare for and administer the operating tests. The NRC will take reasonable efforts to minimize the impact on other training activities.
Before developing or administering an initial licensing examination, facility licensees are encouraged to review the simulator examination security considerations in Appendix D to NUREG-1021 for applicability to their facility. Because facility licensees are more familiar than the NRC examiners with the _ unique capabilities, limitations, and vulnerabilities of their simulators, it is i expected that the licensees will take responsibility for determining and implementing whatever measures might be necessary to ensure the integrity of I
, the operating tests.
- i. The facility licensee shall meet with the NRC in the regional office or at the facility, as necessary and appropriate, to review the examinations and discuss l
potential changes. ; NUREG-1021, 4 of 26 Revision 8 I
ES-201 If the examination was prepared by the NRC, the facility reviewers should make their comments and recommendations on a copy of the written examination
. and operating test (s) provided to them by the NRC examiner. Simple editorial changes that do not change the intent of the question require no justification; however, every substantive change (e.g., deleting a question, replacing a distractor, or revising an answer) must be supported by approved facility reference material before the NRC ~will change the examination.
if the facility licensee has significant disagreements with the content or difficu of the NRC-prepared examination or the changes that the NRC has directed the facility licensee to make in its proposed examination, the facility licensee is encouraged to communicate those concems to the NRC. If the NRC chief
. examiner is not responsive to the stated concems, the facility licensee should contact regional management, and, if necessary, the Chief of the Operator Licensing Branch, Division of Reactor Controls and Human Factors, Office of Nuclear Reactor Rogulation, for resolution. ,.
J. If the facility licensee developed the exarninations,'it will generally make any necessary changes as agreed upon with the NRC; however, the NRC retains final authority to approve the examinations. k. In accordance with ES-202, the facility licensee shall submit the license
, applications along with a letter requesting that licensing examinations be administered. ' '
- 2. NRC Realonal Manaamment. Suoervin' + 1 and Desior.;;;
a. The regional office'shall schedule the NRC's initial operator licensing examinations and shall arrange for the development, administration, and gradi of those examinations as discussed below. The regional office shall periodicall review each facility licensee's examination requirements and shall negotiate with the facility licensee's training representatives as necessary to schedule specific examination dates ;; i:: ;; p:::"i ; th; Ct: :::12 n th; n:On;' n.:n;%.. ;;tei';, consistent with operational safety requirements and NRC resource availability. Each regional office shall plan to prepare at least one complete examination per calendar year, ) j b.
- Approximately six months before each anticipated examination date, the regional
. office should contact the facility licensee and confirm the examination date(s)
- ~1 and the expected numberof applicants to be examined. The regional office
- cshould use that information to estimate the required number of NRC examiners and to make preliminary work assignments. ,
c. The regional office should contact the facility licensee by telephone at least four months before the scheduled examinations to reconfirm the expected number of
' NUREG-1021 5 of 26 i
Revision 8
ES-201 applicants and the examination dates, and to make other preliminary arrangements for developing the examinations. The person who contacts the facility licensee shall discuss the following examination arrangements: the guidelines for ensuring examination !ntegrity and security (refer to Attachment 1) the requirement for an authorized representative of the facility licensee to approve the examination outlines and examinations before they are submitted to the NRC for review the need to have the examination outlines delivered to the NRC at least 60 days before the scheduled enemmetion-review date
. the need to have the reference materials necessary for the NRC to .
develop the examination (if applicable; refer to Attachment 2) delivered to ' the regional office at least 60, but prMerably 90, days before the scheduled eneminateen-review date the guidelines for developing, administering, and grading the written examinations, as applicable (ES-401, ES-402, and ES 403, respectively) the need to have the simulator available and the guidelines for developing and administering the operating tests (ES-301 and ES-302, respectively) the need to have the examinations and the supporting reference materials (refer to Attachment 2) delivered to the regional office at least - 30 days before the scheduled ewemination-review date the requirements (refer to 10 CFR 55.31) and guidelines (refer to ES-202) for submitting the license applications The regional office may negotiate earlier due dates with the facility contact but
~
should refrain from advancing the dates if it is unlikely that the review will begin promptly after the material arrives in the regional office. The region should also keep the facility contact informed of the dates by which the region expects to provide its comments regarding the licensee's submittals.
'd. The regional office shall normally issue a letter confirming the verbal arrangements no later than_120 days before the examination begins. The letter should be addressed to the person at the highest level of corporate management who is responsible for plant operations (e.g., Vice President of Nuclear Operations). Attachment 3 is an example of such a letter; the exact wording may be modified as necessary to reflect the situation.
. e.
Approximately four months before the scheduled examination, the regional office will assign the required number of examiners to develop, prepare for, and
- L administer the examination as arranged with the facility licensee. The regional office will also designate a chief examiner to coordinate the examination project with the. facility licensee andother examiners assigned to the examination.
When making assignments, the region should consider each examiner's certification status, other examination commitments, possible conflicts ofinterest NUREG-1021 6 of 26 Revision 8
1 l i ES-201 (as discussed in Section D of this ES) and general availability. The region should try to assign a sufficient number of examiners so that no examiner will have to adm,inister more than four operating tests per weak. f. The regional office will evaluate each examination assignment to determine if some or all of the assigned examiners should make a separate preparatory site visit. The purposes of such a visit may include providing examiner orientation, retrieving additional reference material, or reviewing and validating the examinations. When making a decision, the region should carefully weigh the costs and benefits associated with each additional trip to the facility. The region 4 should also consider such factors as the experience of the assigrted examiners, the quality of the facility licensee's examinations (if applicable), the number of written examinations and operating tests to be validated, and the status of the i simulation facility (e.g., is it new or recently upgraded?). In addition, the region i should consider the attemative of reviewing the written examination (s) and operating test (s) with the facility licensee via telephone (if the examination quality 1 is high) or in the regional office, as well as the alternative of validating the operating test (s) on-site at the beginning of the examination week, g. Upon receiving the preliminary license applications,'approximately 30 days before the examination date, the regional office shall review the applications in accordance with ES-202. In addition, the regional office shall evaluate any - waiver requests in accordance with ES-204 to determine if the applic' ants meet the eligibility criteria specified in 10 CFR 55.31. After reviewing and approving the preliminary license applications and resolving all waiver requests, the region will prepare an examination assignment sheet (in the format of Attachment 4) as far in advance as possible, but at least two weeks
' before the scheduled examination date. The region will review and revise the assignment sheet as necessary after receiving and evaluating the final license applications.
The' assignment sheet will identify the chief and other examinars by name and list the applicants by name, docket number, and type of examination (e.g., SRO upgrade, RO' written only) to be administered. All applicants listed on the assignment sheet should be administered complete examinations (written and operating) as indicated under " Examination Type" unless waivers have been
~ granted in accordance with ES-204. A copy of the assignment sheet will be
" distributed to all assigned examiners, the Operator Licensing Branch (OLB), and regional distribution.
- h. The responsible regional supervisor will review the examination outlines s.nd the draft examinations and evaluate any recommended changes and corrections noted during the chief examiner's review The supervisory review is not intended NUREG-1021 7 of 26 Revision 8 L
ES-201 to be another detailed review, but rather a check to ensure that all applicable administrative requirements have been implemented. If the outlines, examinations, and recommended changes are acceptable, the supervisor will . authorize the chief examiner to resolve any noted deficiencies with the author or facility contact. If any of the facility-developed examination materials (written, walk-through, or simulator) require substantive changes and cannot be made to conform with the examination standards at least five working days before the scheduled examination date, regional management shall consult OLB and make a decision { t whether to proceed with the facility-developed examinations or d,evelop the examinations in-house. If the region does not have the resources to ensure that I acceptable examinations are prepared by the scheduled administration date, regional management shall negotiate with the facility licensee to reschedule the examinations as necessary. It is generally easier to postpone the written examination and focus on the operating tests so that they can be administered on schedule and without affecting other examinations. F ;x;m:n;;;;n pr;b:;m;
;;; ;d;n:T.;d ;t th; '::t mh:t;, M ;:::m r:S:-n; ;Mdd M p;Q;n;d;
;d :; . ;; ;;;m:n;;;;n ;Mai;;;; .;"l net M m:i dr:n, M ;;;mln;;;;n ;;;d
. The region may delay the written examinations or the operating tests for up to 30
' days without OLB approval. However, the region ~should keep OLB informed and consider notifying the facility licensee in writing if the delay is significant.
The responsible supervisor will also ensure that any significant deficiencies and problems are addressed in the examination report in accordance with ES-501, i . After the chief examiner has verified that the necessary changes and corrections have been made, the responsible supervisor will review and approve the t examinations for administration. Before signing the applicable quality assurance form (i.e., Form ES-301-3 and/or Form ES-401-6), the supervisor must be satisfied that the examination is acceptable for administration. 1 J. If there is an indication that an examination may have been compromised, the i responsible supervisor will take action as necessary to ensure and restore the i integrity and security of the examination process. Actions may include not giving the examination, making additional changes to the examination, voiding the results if the examination has already been given, and possibly imposing i enforcement action in accordance with NUREG-1600. The supervisor shall keep regional management and OLB informed of any concems regarding examination integrity.
- 3. Assioned NRC Framinars
- a. When assigned to administer operating tests for the first time at a particular facility, the examiner should inform the chief examiner and the responsible
- NUREG-1021 8 of 26 Revision 8 E
ES-201 supervisor so that arrangements can be made to conduct an orientation trip to j the facility as described in item C.2.f. if deemed appropriate. j
- b. NRC examiners must monitor and ensure the integrity of the examination process. If they perceive that a compromise has occurred, they must , !
Immediately report it to the responsible regional supervisor so that the necessary actions can be taken to restore the integrity of the examination. Attachment 1 summarizes several examination security and integrity considerations.
- c. The assigned examiners shall review and inventory the reference materials received from the facility licensee in response to the 120-day corporate notification letter. The purpose of this review is to determine if the materials are complete and adequate to enable the regional office to review or develop the examinations, as applicable, if it is not, the reviewer (s) shall inform the chief examiner and the responsible supervisor and request that the facility licensee !
send any additional materials that might be required. If necessary, an examiner l may review and select additional reference materials during a site orientation trip l (refer to item C.2.f). ! i d. The chief examiner will work with the assigned examiners and the designated i facility contact, as applicable, to ensure that the examination outlines and i examinations are developed in accordance with the applicable examination
- standards. The chief examiner should adapt the level of oversight and coordination based upon the experience of the individuals who are preparing the i
examinations. Facility employees are generally less familiar with the ; examination standards and will require more oversight to ensure that a quality examination is ready on time,
- e. The chief examiner will review the examination outlines using Form ES-201-2,
" Examination Outline Quality Assurance (QA) Checklist," as a guide. A thorough and timely review (i.e., within 5 working days) will minimize the potential for i significant problems with the examinations. 1 The chief examiner will note any necessary changes and forward the outlines to the responsible supervisor (or a designated alternate other than the chief examiner) for review and comment before resolving any deficiencies with the !
author or facility contact. If the outlines are significantly deficient, refer to item C.2.h for additional guidance. 7 , s
' f.
1he chief examiner will review the written examinations and operating tests for
" quality in accordance with the applicable QA checklists (refer to ES-301 and ES-401) forwarded with the examination. If the chief examiner wrote the operating
; tests, another NRC examiner shall perform the independent review. The regional office may conduct additional reviews at its discretion if resources permit.
NUREG-1021 9 of 26 Revision 8
ES-201 It is especially important that facility-developed examinations and tests be reviewed promptly because of the extra time that may be required if extensive ; changes are necessary. The QA reviews should be completed within two weeks ' after the examinations and tests are received from the author or facility contact. The chief examiner will note any necessary changes and forward the examinations and tests to the responsible supervisor (or a designated alternate other than the chief examiner) for review and comment before reviewing the examinations with the author or facility contact. There are no minimum or maximum limits on the number or scope of changes the NRC may direct the facility licensee to make to its proposed examinations, provided they are { necessary to make the examinations conform with established ecceptance criteria or to attain an appropriate level of examination difficulty. Chief examiners shall exercise their experience and judgement to ensure that the level of difficulty remains consistent with that expected on NRC-prepared examinations. If the examinations are significantly deficient, refer to item C.2.h for additions guidance.
. g. Upon supervisory approval, generally about two weeks before the examinations are scheduled to be given, the chief examiner will review the written examinations and operating tests with the facility licensee. If 'h; :n.T.:n;;;s; J.;; d;;;l:;;d by th; N^O, th; r;^;:;; ;h;" 5; sadd:d in ;-::::dsa .;;;h th; :n;;;;;;;; .; :n .^. :;C. ..st 5.
' The chief examiner may conduct the examination review via telephone, in the , regional office, or at the facility, as appropriate to the circumstances, depending i on the extent of the changes, and as approved by the responsible regional supervisor (referto item C.2.f). i If the examination was prepared by the NRC, the regional office will provide a copy of the written examination (s) and operating test (s) to the facility reviewers only after they sign the security agreement (Form ES-201-3). The facility reviewers should make their comments directly on the examination, retum the marked-up copy to the NRC chief examiner, and ensure that he or she ; understands their comments and recommendations. The facility reviewers may ! retain a copy of the marked-up examination, subject to the physical security ! guidelines in Attachment 1. ' If the facility reviewers have significant disagreements with the chief examiner,
' the chief examiner will inform the responsible regional supervisor so that the ;
- disagreements can be resolved before the examinations are administered.
l
- h. After the examination corrections have been made, the chief examiner shall !
verify that the changes are appropriate and route the examinations and the mark-up drafts to the responsible supervisor for fina! approval. NUREG-1021 10 of 26 Revision 8
1 ES-201 i.
' As soon as possible after the responsible supervisor has approved t tests for administration, the chief examiner shall distribute copies of the scenarios, job performance measures (JPMs), and questions to the other assigned examiners so that they can familiarize themselves with those materia and be better prepared to probe the applicants' deficiencies if required.
j..
. The chief examiner should work with the designated facility contact to schedule the operating tests to optimize efficiency and the mix of RO and SRO applicant in the crews assembled for the simulator examinations. The number applicants on a crew shall not exceed the number of assigned examiners (i.e.
one-on-one evaluations are mandatory), except as noted below. However, if the facility licensee's technical specifications routinely require more than two ROs to be stationed in the control room, OLB may authorize the use of additional surrogates. Only one individual (applicant or surrogate) is allowed to fill a shift supervisor or manager position during the simulator operating test. If a three-person operating crew consists entirely of SRO-upgrade applicants (who do not have to be evaluated on the control boards), the region may as
' only two examiners to observe the crew. Although the applicants in the RO and balance of plant positions may not be individually evaluated, they will be grad and held accountable for any errors that occur as a result of their action (s) or inaction (s).
Normally, for purposes of test integration and continuity, the same examiner should administer all three operating test categories to an applicant. Howeve under certain circumstances, the operating test (excluding the dynamic simulator
~
scenarios) may be divided among different examiners. Such division is appropriate if a facility licensee's simulator is not located near the plant, because of limitation's in examiner resources or scheduling, or if a facility licensee requests examinations for an unusually large group of applicants. Refer to ES-302 for specific instruebons regarding administration of the operating tests. j Operating tests will normally be administered on regular work days. If weekend or shift work is required to administer the operating tests, the chief examiner will coordinate licensee. the arrangements with the assigned examiners and the facility l As a general rule, the operating tests should be scheduled after the written examinations; however, other sequences are permissible if agreed to by the !
~ u facility licensee. Normally, the written examinations should be administered no more than one week before the operating tests. However, under extenuating circumstances and with prior approval from OLB, the written examinations may be given as soon as the license applications are accepted, any applicable waiver requests are resolved, and the examinations are approved.- if n;;;n:ri, OLS m;y C:- ;dhd ; th; ;# en =;rn:n;;;;n; i: b; diy;d unt;; ;%; the NUREG-1021 :
11 of 26 Revision 8
ES-201
. ger;t:ni; t;;;; ;;; ;;m+%;;.
If, as an efficiency measure, the facility licensee prepared the written examination in conjunction with another facility, then the two examinations must be administered at the same time. If the examination schedule has to be changed on short notice, the chief - examiner will work with his or her supen/isor and the designated facility contact to reschedule the examinations to a time when examiners are available and other examinations en th; n;tba;.l :nT.b;tbn ;;h;ik are not affected,
- k. If the facility licensee will administer the w:itten examinations, th'e chief examiner shall review the ES-402 requirements (e.g. , proctoring and responding to applicant questions) and confirm the applicant's status on the assignment sheet j (i.e., examination type and waivers) with the facility contact before the examinations are given. {
1 D. PERSONNEL RESTRICTIONS
)
It is impossible to define criteria that anticipate every possible conflict-of-interest issue. Supervisors must apply sound Judgment to the facts of each case. ' If any doubt exists regarding a particular case, the supervisor should consult with regional management and/or OLB to resolve the issue. i 1
- 1. NRC Examinars !
]
. a.
i The regional omos shall not assign an examiner who failed an applicant on an ! operating test to administer any part of that applicant's retake operating test.
- b. )
. If an examiner was previously employed by a facility licensee (or one of its contractors) and was significantly involved in training the current license applicants, the regional office will not assign that examiner any direct responsibilities for developing or administering written examinations or operating tests at that facility. Regional management will control other in-office examination activities concoming the facility, such as technical consultation and quality assurance reviews of examinations.
- c. If an examiner is assigned to an examination that might appear to present a conflict of interest, the examiner shall inform his or her immediate supervisor of
~ ' the potential conflict. Such notifications should include the following information:
the nature and extent of previous perscnal and professional relationships with the applicants anything that could affect the administration, performance, evaluation, or NUREG-1021 12 of 26 Revision 8
q l ES-201 results of the examination anything that could create the appearance of a conflict of interest 1 1 1
- 2. - Facility Personnel i a.
Facility employees or contractors who had any direct involvement in training the ll;;r.n applicants during their site-specific license training program shall not prepare the outline (s) for the operating test (s) but may prepare the outline (s) for the written examination (s) er th; sp; mig 0;d; using a systematic process, as I described in ES-401, 1 Or.ly =; p;ran ;;he prided r== th;n 40 :: d h;;;; cf ;;h[il:d ;ln;mem I
- nd im;;;ter nMrud;r ;nd n; es ?;he pa?lid 15 p;ma cc mem of th; )
nh;il:d dner;;m nCr;;On er 20 p;reen er m;m ;f th; ;;;d ;;h;ilcd T'::: ram ;nd im;l:ter ndr;;; = m;y p;d:C : t in i v;';p:ng th; ;;rden n;;; son qc:%s. I;dh;rmere, p;d::lps ; m;y r;;t iv;lep ;;rden qus%s fer th;;;p;; th;y t;;;ht. There'are no restrictions on test item developers based upon their time spent 1 i
. instructing the license applicants. However, in the case of the written l examination and prescripted walk-through questions, personnel who were directly involved in training the license applicants may develop new questions, i
select questions from the test banks, and modify test bank questions for only ' those topic areas in which they did not provide instruction. Instructors (and supervisors) may review questions on all topics, as necessary, in order to verify i their technical and psychometric validity. Moreover, instructors may develop and bank questions in their area of expertise provided those questions have no j greater chance of appearing on the examination than other bank questions. Simulater-Instructors may prepare the implementing documentation for the dynamic simulator and job performance measure (JPM) portions of the operating test without regard to their instructional specialty, once the outlines (i.e., the simulator event sequences and walk-through task lists) are defined. As a means of NRC verification, the facility licensee may be requested to identify (1) which test items were developed and reviewed (including the effects of such reviews) by individual examination developers and (2) which content areas were taught by the examination developers. yn i If the facility licensee considers these restrictions too burdensome, it shall define and discuss with the NRC chief examiner the process it proposes to ensure that ; exam integrity is not compromised. The NRC chief examiner must approve the' facility proposal before it is implemented, and the regional office shall inform OLB l of any such situations before commencing examination development. i NUREG-1021 13 of 26 Revision 8 i l
<\
ES-201
- b. The facility licensee shall minimize the number of personnel who have detailed knowledge of the NRC licensing examination.
- c. All personnel who will receive detailed knowledge of any portion of the NRC licensing examination, incieding the examination outline, must acknowledge their responsibilities by reading and signing Form ES-201-3, " Examination Security Agreement," before they obtain detailed knowledge and again after the examinations are complete. Examples of prohibited activities include the following: -
the design and administration of any classroom and simulator instruction (including scheduled sessions, individual coaching, and remedial training) specifically for the license applicants (Simulator booth operation is . I acceptable if the individual does not select the training content or provide direct or indirect feedback. Continued participation in requalification training for groups including SRO upgrade applicants is also acceptable, as long as it is documented on Form ES-201-3 and is limited to areas in which the instructor has no examination knowledge.) all on-the-job training, practice, coaching, and sign-offs the preparation, review, grading, and evaluation of periodic quizzes, examinations, and simulator exercises, including the audit examination Supervisors and managers having knowledge of the examination content may continue their general oversight of the training program for the license applicants, including the review of examinations, quizzes, and remedial training programs. However, those' supervisors and managers may not provide feedback regarding the content of those examinations, quizzes, or programs.
- d. The facility licensee will provide a copy of Form ES-201-3 (listing the expected signatories) to the NRC chief examiner at the time the examination arrangements are confirmed. The facility licensee shall also inform the chief examiner if additional personnel need to be added. The chief examiner shall review the list of personnel, consider the nature of each individual's association with the license applicants, and evaluate the pros (e.g., increased technical accuracy and validity) and cons (e.g., increased security risks) associated with each proposal. The regional office (i.e., the chief examiner in consultation with the responsible supervisor) may deny the facility licensee's proposal to use certain individuals (e.g., the applicants' supervisors or coworkers ) to develop
~and review the examinations, but should discuss such decisions with OLB before informing the facility licensee.
The original security agreement forms must be submitted to the regional office for retention after the examinations are complete.' NUREG-1021 - 14 of 26 Revision 8
- p. i
)
l q l ES-201 L j , E. ATTACHMENTS / FORMS - l
- Attachment 1, " Examination Security and Integrity Considerations" l
; Attachment 2 l
" Reference Material Guidelines for initial Licensing Examinations" j Attachment 3,- " Sample Corporate Notification Letter' '
- Attachment 4, - " Sample Examination Assignment Sheet"
^
. .t;M.;r.: 5, "Ould;llr.;;
.-_e__o___ fdr I;;ll:i Pr;r;;:;;; cf NPO 0;;;lc;-d lr.ltl:' Ll;;r.;lr.;i !
Form ES-201-1, " Examination Preparation Checklist" - ; Form ES-201-2,' . Examination Outline Quality Assurance Checklist" - , Form ES-201-3,- j
" Examination Security Agreement" i
a I
]
]
4
- NUREG-1021 15 of 26 Revision 8 6
l ES-201 Examination Security and Attachment I integrity Considerations NRC and facility licensee personnel must be attentive to examination security measures to ensure compliance with 10 CFR 55.49. At the time the examination arrangements are confirmed, an NRC examiner shall review with the facility licensee the following guidelines covering physical security and limitations on the use of examination banks as well as the facility licensee's specific plans for ensuring examination s6CJrity. Additional restrictions covering the assignment of personnel are addressed in Section D of ES-201 and shall also be reviewed. ) Physical Security Ernectations 1. Th; N",0 ;;put; th;t th: fa;::M i lles.n '.;': s;ie:x th; um; phy;4;: narny preaut:en; ;;nh th; :nn:;' ;;;m =t:s; n n ds; '. inh n; aqu;:lf.stlm ear :7.;t:en;. The NRC expects that personnel will be made aware of the facility licensee's physical security measures and requirements, sign the NRC's examination security agreement, and understand their security responsibilities, including the limits on their interaction with the license applicants, before they are given knowledge or custody cf any examination materials. 2. All examination-specific materials (i.e., the examination outlines and final examinations) shall be positively and continuously controlled and protected as sensitive information (i.e., under lock-and-key or in the custody of someone who has signed the security agreement). Drafts, copies, and waste materials must also be controlled and disposed of properly. The NRC expects that the examinations wi;; NOT be developed and stored on a computer network to which the license applicants enuld gain access. 3. The examination outlines, written examinations, and operating tests that are mailed to the NRC regional omce shall be placed in a double envelope. The inner envelope shall be conspicuously marked "FOR OFFICIAL USE ONLY" and "TO BE OPENED BY ADDRESSEE ONLY " Furthermore, the cover letter forwarding the examination materials shall direct that the materials be withheld from public disclosure until after the examinations are complete. The examination outlines and examinations shall not be transmitted via non-s electronic means (e.g., the Intemet); they may be transmitted via the NRC's " AUTOS" local area network in the resident inspectors' office. 4. The facility licensee shallimmediately report to the NRC chief examiner any indications or suggestions that examination security may have been compromised. The NRC will evaluate such situations on a case-by-case basis and determine the appropriate course
'of action.
I i NUREG-1021 16 of 26 Revision 8 1
+
l {
ES-201 2 Attachment 1 5. The facility licensee and the NRC should determine if examination security problems were noted in the past and ensure that corrective actions have been taken to precludo recurrence. 6. The facility licensee and the chief examiner will review the simulator security considerations in Appendix D to ensure that the instructor station features, programmers' tools, and extemal interconnections do not compromise examination integrity. The primary objective is to ensure that the exam material cannot be read or recorded at other unsecured consoles, and that examination materials are either physically secured or electronically protected when not in use by individuals listed on the
. security agreement.
Examination Bank Limitations 1. The facility licensee and chief examiner shall ensure that written examinations and operating tests conform with the guidelines in ES-301 and ES-401 regarding the use of items taken directly from the bank, modified items, a' red new items. 2. If the facility licensee has an open bank, it will not place any new or modified items to be used on the examination (written cuestions, job performance measures, or simulator
. administered.
scenarios) in its examination bank until after the last examination has been
, Other Considerations 1.
The NRC will consider an examination to be potentially compromised and a possible violation of 10 CFR 55A9 to have occurred if a situation exists or existed for applicant to gain an unauthorized advantage as a result of the facility licensee's failure to exercise continuous positive control over the integrity of the examination. If the NRC determines that a facility licensae has manipulated the scope, content, or level of difficulty of an examination to enhance the chances that its app!icants would pass the examination, the NRC will utilize Es sniorcement authority including, as warranted, civil penalties, orders against the individuals involved, and, charging the individuals involved with deliberate misconduct pursuant to 10 CFR 50.5. 2.
<, The license applicar'.s should not be able to predict or narrow the possible scope or
_ content of the licensing examination based on the facility licensee's examination practices'(other than those authorized by this NUREG or in writing by the NRC). 3. Facility licensees are responsible for the integrity, security, and quality of examinations prepared for them by contractor personnel. A 1 NUREG-1021 17 of 26 Revision 8 -
i ES-201 Reference Material Guidehnes Attachment 2 For Initial Licensing Examinations This attachment discusses the reference materials that facility licensees are expected to provide for each NRC initiallicensing examination. The regional office will customize the list of reference materials as required to support the specific examination assignment; additional materials may be requested at a later time if necessary to ensure the accuracy and validity of the examinations. In determining the need for reference materials, the regional office will consider the facility licensee's level of participation in the examination developraent process. If the facility li.'ensee will be preparing the examinations, it may be sufficient to obtain only those references necessary to review and validate the items that appear on the examination, plus a set of key procedures and other documents required to prepare for the operating tests. The regional office will duly consider the administrative burden it places on facility licensees and request only those materials that are actually necessary for the NRC examiners to pre pare for the examinations. All reference materials provided for the license examinations should be approved, final issues and should be so marked. If any of the materialis expected to change before the scheduled examination date, the facility licensee should reach agreement with the NRC chief examiner regarding changes before the examinations are administered. The reference materials may be submitted on computer diskettes (in a format compatible with the NRC's word processing software), as hard copy, or a combination as arranged with the NRC chief examiner. If the facility licensee prepares the examinations, the hard-copy i ' references should normally be limited to those materials required to validate the selected test items. All procedures and reference materials should be bound with appropriate indices or ' tables of contents so that they can be used efficiently; a master table of contents should be provided for all materials sent. Failure to provide complete, properly bound, and indexed reference material may prompt the NRC to retum the material to the person at the highest level of corporate management responsible for plant operations. The retumed reference materials ; will be accompanied by a cover letter explaining the deficiencies in the material and the basis ' for postponing or cancelling the examinations. I Unless otherwise instructed by the NRC regional office, the facility licensee is expected to j provide the following reference materials for each NRC initial licensing examination:
- 1. Materials used by the facility licensee to ensure operator competency
- a. The following types of materials used to train applicants for initial RO and SRO licensing, as necessary to support examination development:
leaming objectives, student handouts, and lesson plans system descriptions of all operationally relevant flow paths, components, controls, and instrumentation NUREG-1021 18 of 26 Revision 8
i e* ES-201 2 Attachment 2 mPc a used to clarify and strengthen understanding of normal, abnormal, and emergency operating procedures
\
complete, operationally useful descriptions of all safety system interactions and, where available, balance-of-plant system interactions ; under emergency and abnormal conditions, including consequences of anticipated operator errors, maintenance errors, and equipment failures, as well as plant-specific risk insights based on a probabilistic risk analysis (PRA) and individual plant examination (IPE) These materials should be complete, comprehensive, and of sufficient detail to l support the development of accurate and valid examinations witHout being a redundant.
- b. Questions and answers specific to the facility training program that may be used in the written examinations or operating tests
- c. Ccpies of facility-generated simulator scenarios that expose the applicants to abnormal and emergency conditions, including degraded pressure control, degraded heat removal capability, and containment challenges, during all modes of operation, including low-power conditions (A description of the scenarios used for the training class may also be provided.)
d. All JPMs used to ascertain the competence of the operators in performing tasks j within the control room complex and outside the control room (i.e., local t operations) as identified in the facility's job task analysis (JTA) (JPMs should l' evaluate operator responsibilities during normal, abnormal, and emergency conditions and events, and during all modes of operation including cold ; shutdown, low power, and full power.) '
- 2. . Complete index of procedures (including all categories sent)
- 3. All administrative procedures applicable to reactor operation or safety
- 4. All integrated plant procedures (normal or general operating procedures)
- 5. . All emergency procedures (emergency instructions, abnormal or special procedures)
- 6. Standing orders (important orders that are safety-related and may modify the regular procedures)
- 7. ' Surveillance procedures that are run frequently (i.e., weekly) or that can be run on the simulator 4
- 8. '
Fuel handling and core loading procedures (if SRO applicants will be examined) NUREG-1021 19 of 26 Revision 8
ES-201 ' 3 Attachment 2
- 9. All annunciator and alarm procedures 10.- . Radiation protection manual (radiation control manual or procedures)
- .1 1.
. ~ Emergency plan implementing procedures -
- 12. Technical Specifications (and interpretations, if available) for all units for which licenses are sought '
; 13. . System operating procedures
~
l
- 14. Technical data book and plant curve information used by operators as well as the facility precautions, limitations, and set points document
- 15. The following information pertaining to the simulation facility:
- a. - list of allinitial conditions
- b. -
list of all malfunctions with identification numbers and cause and effect information, including a concise description of the expected result or range of results that will occur upon initiation and an indication of which annunciators will be actuated as a result of the malfunction , c. * - a description of the simulator's failure capabilities for valves, breakers, indicators, and alarms
- d. ' the range of severity of each variable malfunction (e.g.', the ' size of a reactor coolant or steam leak, or the rate of a component failure such'as a feed pump, turbine generator, or major valve) d.
a list of modeling conditions (e.g., simplifications, assumptions, and limits) and problems that may affect the examination
- f. a list of any known performance test discrepancies not yet conected g.
. a list of cafferences between the simulator and the reference plant's control room h.' simulator instructor's manual . - 16.--
L Any additional plant-specific material that has been requested by the NRC examiners to develop examinations that meet the guidelines of these standards and the regulations NUREG-1021 ~ 20 of 26 - Revision 8 ( 4
ES-201 Sample Corporate Attachment 3 Notification Letter (date) (Name. Title) ~ (Name of facilitv) (Address) - (Citv. State. Zio code) .
Dear (Name):
s-In a telephone conversation on [ data) between MrJMs. (Name. Title) and MrJMs. (Name. Title). arrangements were made for the administration of licensing examinations at (facility name) during the week (s) of(date) [As agreed during the telephone conversation, your staff]((The NRC)) will prepare the examinations based on the guidelines in Revision 8 of NUREG-1021, " Operator Licensing Examination Standards for Power Reactors."[ The NRC regional office will discuss with your staff any changes that might be necessary before the examinations are administered.](( fn nard;n;; ;; th th; guid;: na in Attnhment 5 ;f CO 20h-Your staff will be given the opportunity to review the examinations during the week of { data).]} To meet the above schedule, it will be necessary for your staff to fumish the [ examination outlines by (date).' Th t written examinations, operating tests, and the supporting] reference materials identified in Attachment 2 of ES-201 [will be due] by (date) [ Pursuant to 10 CFR 55.40(a)(2), an authorized representative of the facility licensee must approve the outlines, examinations, and tests before they are submitted to the NRC for review and approval.] Any delay in receiving the required [ examination and] reference materials, or the submittal of inadequate or incomplete materials, may cause the examinations to be rescheduled. in order to conduct the requested written examinations and operating tests, it will be necessary for your staff to provide adequate space and accommodations in accordance with ES-402, and to make the simulation facility available on the dates noted above. In accordance with ES-302, _ your staff should retain the original simulator performance data (e.g., system pressures, temperatures, and levels) generated during the dynamic operating tests until the examination results are final. Appendix E of NUREG-1021 contains a number of NRC policies and guidelines that will be in effect.while the wntten examinations and operating tests are being administered. To e ly NRC review and evaluation, your staff should submit preliminary reactor
' tim operator and senior reactor operator license applications (Office of Management and Budget (OMB) approval number 3150-0090), medical certifications (OMB approval number 3150-0024), l ' and waiver requests (if any)(OMB approval number 3150-0090) at least 30 days before the first 1 ' examination date. If the applications are not received at least 30 days before the examination '
1 NUREG-1021 - 21 of 26 Revision 8 9
' ES-201 2 Attachment 3 l date, a postponement may be necesr,ary Signed applications certifying that all training has been completed should be submitted at least 14 days before the first examination date.
This letter contains information collections that are suyect to the Paperwork Reduction Act of 1995 (44 U.S.C. 3501 et seq.). These.information collections were approved by the Office of Management and Budget, approval number 3150-0101, which expires on April 30, 2000. fdahtt-The public reporting burden for this collection of information is estimated to average fembed
. [500) ((50)) hours per response, including the time for reviewing instructions, gathering and
- maintaining the data needed,~[ writing the examinations, ]and completing and reviewing the collection of information. Send comments on any aspect of this collection ofinformation, .
' including suggestions for reducing the burden, to the Information and Records Management Branch (T-6 F33), U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001, or by -
Intemet electronic mail at BJS1@NRC. GOV; and to the Desk Officer Office of Information and
. Regulatory Affairs, NEOB-10202, (3150-0101), Office of Management and Budget,' Washington, DC 20503. '
The NRC may not conduct or sponsor, and a person is not required to respond to, a collection of infonnation unless it displays a currently valid OMB control number. Thank you for your. cooperation in this matter. (Name) has been advised of the policies and guidelines referenced in this letter. If you have any questions regarding the NRC's examination procedures and guidelines,' please contact (name of ranianal contact) at (teleohone numbert or (name of r==aannihla ranional anaarvisor) at (telephone number). Sincerely, (Anorooriate raaional reoresentative.
- Tiutl Docket No.: 50-(Number)
Distribution: Public - NRC Document ControlSystem Regional Distnbution [).. Include ordy for examinations to' be prepared by the facility licensee. i (()) include only for examinations to be prepared by the NRC.
; 3, ..
l L NUREG-1021 22 of 26 Revision 8 L
V i
; ES-201. Sample Examination .
Attachment 4 Assignment Sheet l l l l . MEMORANDUM TO: (List NRC examiners bv name) FROM: (Reaional Suoervisors Name. Title)
SUBJECT:
EXAMINATION ASSIGNMENTS APPLICANT DOCKET NO. EXAMINATION TYPE I l Facility andlocation i Facility contact
. NRC chief examiner Written examinations to be prepared by (RO)
(SRO) , Dates of Examinations ' i NRC Supervisor cc: Resident inspector Project manager (Standard regionaldistribution) ' OLB t NUREG-1021 23 of 26 Revision 8
F'~ t t l'* e S A 4 w V di. w a r% . . : s _ s . _ . E . _ r* _ m :s.A. h__m.:... gas hs s w ge u s s s w air uws s A6A_.L w%ru ss by 4 aw w w sw wy -__6 f" _ f L i M #'t P'\ _ . . . f J I ._ ! A ! _ t t
- _ _ _ _ *__ r=* . , _ ! _ a ! _ _ _
E T h6 e be e a a a s h, s # 6 g/ wa ais W se w w w ~p.rww i s a s u ss u was,w a swes sy unus a ses assovs se 4
- s. "PL . _ _ _ _ _ . ! _ . . . _ f L f M f's J .
serw ys u s s, w s w. vu vs _f___ J I_11t_1 t?_.__!_ _,,__L_1?__. s i a t v ww. w w s wyw w u n uus -_.,L_ __,J.._A_J _aAL_ _ !a m .- !_ A L _ L f f"% #% _.*.__t _ ff _ u wws sess ig wnwi s uss as,s uvu se c a sey ww we swwk,6ww u n u sw w a n t, ws ss s u rw __ J:-...__) L e ._ _1 S _. _fALf_ t ! _ _ ,_ _ ! _ m..__*__A!___ u is gv s wysws uess w a s s ww , u w vswwgseww w sus w wwuw._ #% is v.v.y wa u ss w w ass a uw sis u se . _A_.J_.J "f*L . _ L _ It L_ m..!_....J .! I s sV, saws,s swas sg wnes s ris su uws se ws rosas us, a %, y sw w,w w _____.._J L.. AL_ _ _. _ _!Lt. _ . _*___I
" . . .___!___L.f.._ AL._.. __ f____ff.. m___.,:_..._J, es sus uyys w w wws wy hasw aw wy vsswawsw avyawssus wwyws u sews ww s va w ts sw y pa w s ws s i stsuy yn g,1 we r sw vv w w wwsu s 68 ~ _
- t :1. . 9:_.____ .!AL AL . f.
l s p hauss47 erww s sww w. ' Wl *T"L _ f_?f'A.. .._..!_..._-i .tti L_ _I.,_
- a. . sasw s eswsuu sy s w w ww ww w s e
___.._:A.. ______.___a ______ &_AL, _._!mAI__ vu sus w%, ga wwa s w www eir e 6v u st wnes s s sss es.s uwu s s t restws sususeir ass
- _A J_t_ _faaw _ _s A L _ . . _!__ _
fr __- r e agru m 6u s%,y wag u t u www6ss sty usgu wws a sw it gs vs a s a w s.r e.w e wf. e3 af iL _ _...L...A_t_.
- w. _ _ _ _ at _ . __ _ r_.. AL_ k t M r% _ L f _ f as u~ s w w sw ww teer3w w s a swa w u samu a vs she wuy , u as, aisgv %,serws genu s a s us twi we swwwws ws swa u sw
. . ._ f. _ _ . L . . .t r L ! _ f A L _
_______Le__.__.!__._aL. _i.i.._ _ f iL _ s wwyws swswww whsyws v swws was Er sw w a sis tbe w we usw__..!_...it_ u se r sw ww g s. w9%rwyw _f_L_____
. , ___ .vs wu susur sgww mes ._J
..-___f._J!__.,__\ _A AL_ ,_J _f-L sws l J_.. M__*--t--__--_A ve rs te wws y w w swerugr eirf Es t u ste ws sba vs ba ss bes a !_ -u--AJam IA _ _
AL, f. -___ILt- ts u y . esw gswssws sisussteg wserwuet soir gre ygewtwgs tw
._ m
- _... .... .--_._. t _f ? - A t _ _ - _fI_
s u ivs sabws J__.i.A. f. _ *f !A. . _f___._ te st s w u sw ww g.rt wgaw naar swu yw w w s www se sbsuvais tsws sw ws s u s www hg hs a ng, a simbasus ty s w s ws ge s sww
- _ A .! I _ _ I _ . . .ff* ? _ _1 ...ita.. ._m..____ L , A L _ 19 M #%
u u sustw s asas vs as sepu u a swsw a at gg tsmess ty esserbst aims sww wy u aw, e is 1W.
,J. *PL __..!_...___ .L_..fl__
ss~ fs _wsusk
. ? t1 A,.aww~.wwse
. ws ~wsw a s srm l _uAL_!____--_.JA*___ __J
~ u s a t ww. s s e s ~ s s.s.suus sw -s ~ wwa s as s ~ s ste._
__-._._*.A. &L- . _ . . wa s um wwry l _f AL we u es,_ ..
- LA _ _ - . _ __ ! . _ A ? _ _ f . \ __mi ___...._1,_../.\ __..*J- t&_ AL . - L..AL. i vu a attges a wnus a sus seim pws sg eirf sins sw us sw ww wi E f f"% r'4 _..__t.,_ 5 L"*!_ _f. . l: A _I _ l .L_.-__ AL _ A J_ uTwJ gej yi w w swww tw u sws a s wy 6s sw a is tv g,nus a sss sh I
%sysssysw wws twe usius
__A L -_. AL_ !_A_ A _f AL . _.._1**. _._...!__ .
.._Aff__A!--. L...-..-.ws amas agges u put ww a swt ws seist syv usw sisawsen wt usw ngtewwerws s a wwgbsse w asw
._-._.L_A__&I... .L,_ _ #_ _ J _1 1 - _ -.._.1? .mfm**. , _ _ _
J%seusivss J: _ A - __A_. tswu __ _s,. s.:awww_ t _ .w._ w br s , w w wsy wwwwtens su w w we semu sgw gst .g. , newsw ese sy as tgtoweirtswa s, s wysphess ___..._s_..aL. wuwns uurwsws , ws mi..__f______ _ _ i . _! _ s af aLa_w __Lt_t__f______ w serus ry use s sus uwww ws f s a shown ww ernsyyvs tww wy muyys ww wts..._.__a__sL..___...._Jf s eswass47 s w s w s te s sww s s suu twa russ. su u s t, _mi_ J , . . L _ 4.a _ f J _ _ __A _..____a we agia sus a en a w e ws sww a s s ustwv sus aL _ _L_ __ aL _ ___A_J&_ __o,t J . AL _ wbswu u sattama tswwtir a swt eiremsyywu 4 ts swa we UEss rgwe 6s sw s te r sw ww ws e ums w genyw wnww tw yt w w sww a,a ste __. .___, J__.._.__1_A*_ L_f___ AL _ k l M #'4 . .Ilt _L_ . &L - ..!_A!__ s u%ewge nrwgiinn y iswwsmi s awu stustrwi s ww s ws w u ste s i s D gsr ww sse has saws agw u she wnnes s a ss sam uwt s. E' *PL _ ___- A?__ &__a-
- w. sasw [_T_.t_i___.___J__ .D fL o,,J_ _ ____
wyne s estas ____..?_... J ag tw erteir gerus i s hs sustwa www s sums sww ums sw jww yws swa ie resi swg, ._____.___\ __.,L-a a sweswws ge wf a a sesy ww _._f ..._f._A.J f _ &L _ m ff" . _ _ _ . . _ I? J m A _ J yi ws & w sw ww w ts uuss su w w unswoutwts se a tu sgp su rs s syg, wi w usn esissow as us _A A L _ f _ _
- t 11. , If A L _ _______tLt_
t u sw _*__.t.._.!_m J_A-
- s muwsssty ss p sga e tw eryvi serswrge s wgvws sums eatsyea s y swwu_ tsw twu s s are rise eisAL A _ _ !A _ . .! _ ?A 'm _._.,___.
asswk usu wetge wswsg se a ~wwerwes y , IfAL. ___ m&!__ A_mA. ___ __A . _ f ! J _ i _ _s s . _* _ _ AL _ ._____..r_.. ea nasw wyVers as us eg tw erteir se u g, s swt u eng-us t w w __sftL_...._J____ AL_ wtse ss sg ta sw yu w s w w s w vv , we su _ t_ _. te _ _ _ i _ Li _ _ _ i! _ __ _ _ _ _ . . ia _f na swy gsu swswa gv wrgs sus usages sg mo u tw s as trwa a u w us s viirtsa t vs A?_L_. ..t?J_A?._ AL_.. _Lm..1J____?.,_ f* , _ g , t; J _ A ' _ .A AL . ts tw yr wees
- a sus rus J w ;---- _ _ _ s y , as swy ers ~tssgs a wgrart v w g rs sess ymerggsupwse es t L*__*_. . f A L __
.- w- i ssa -. a s
-. A ? _ _ ..._i_ # _ _ f - Am f= f't a3 pg di g na sw wwgus su sus sg wu n a s%,
ww wwsw gs wsws tw miosa m wai.f. f* A_ k t MJ"b .._~L__.
- we a us e s is vier waeus s suu sws ww.itt ussL ww
_ _.ga. v_ t _ L t _ A L _ _ . . _ L _ . .A AL _ _..?_. .__J J :- m ..._. AL _ _..__L_A!__ _.___ uusw w gaswwt Es aw s te w sw ww- ess ets . . t! J A ? _ _ _______ A_ __A_ _*_1_ . . ,,,,9 A L _ -...?_...___t . . . _ _ _v essutstimpws e ys wwteww tw De =PLa .ser, f. _
- t
- A p..*_ . . . . _ _
_ L s_ tyv esJ w-.Aeir su sw _- -- u sus s uur erws e i s russows sumummy esu uns u sw a w w sw ww wu gir gews awrws .
. .t a a rge
. _ AL _ _ _ _t . J _.__!__A?__ a su wussty a w w sw wu w s e ers swwyts a wetsu a r u sw a s runs sTwtm ... ,, .A. ?_l_ A. AL _ k g f's #"h _L*_f any weeps a sus agusuws s a s sustw a stenw .
____!_J
.. _ . - AL _ A L_ ___L- ...Jm A__J_
tv u sw u i s g ha we erws wasas a sus sws sist sgs AL_t __ _ _ __A_ _.D,____JA?.-- wu wwww w uswt ssw we ws rise eri states eromas sgsty mz agrus wgri a is a swa stair usus vis L _ f_ - AL_.. r ____m..J_ iL _ _____..r_...___i?__ a wgs wu a ss s stra rwststrwa sair pg,s gry ame u swy ff sk _ f_ _ mi.. __..u...___ wwu swwwwgp La sw ys ws sur w sw ww s a sww tsu rg. rs u r%s a tswf us 67 s W v sw ww we e s stim L _.u %, _ _ ? u r _ _ _ i _ - _ _ _ __ _ _____JL_ AL _ _. , . teFugt as s r%rWu st brVs shrists s siiB f1A. . , _ _f ? fr _ . ta . _f &L _ _.._t.A?_ ,__A _?_1- IA Im AL*__-____?L!f r wgues tsrs eg u sgr tgtemuusty wrs %sst u swwsty ws as sw genuius a sus ues uws e 1.. A_ ___......_L _ _ _ _ _ _ _ a Lema % e a reenwa sesser, at stir u rw us u wqiiryvs seirswissty tw wVs u v try erbsVu u wws svwu a sair __________A f______f.A:__ tw a W s 11er a 5 sums #4sgw s s sws a t sWs a he qupws ess tr wu 4.
"P "t"L _ f _ _ ?t ?A. . _ ..?_..._
- s. _ _ _ . . _ _ _ _ _ . , _f AL _
E a sin rurgressay s w w sw ww we e s a susy a w_mesus A_ t a um ._L_A!__ _ _a__? $_ m ..._If. AL__.._L% wwyy wr tu sw wnus u sus uu trwu a a a eenages runsw I .
- AA - _ _!_..t_A_,
. - Jir J J.._*_ AL .,?_...___ g rus abstes s, erus s s taru nwr ,
ws ww usurs . .__ ma ss wwgs sf une s s swinus u sww news us eg u rw s te r sw ww ys vgaw tire. G '9"L _ f_ _ *
- A R*______
- w. _L_tf __-I. AL _ _L.
st~ s .~.rs t ,. s ~,. s u ,..a w wa s.ssa w.s s u re,. . .w s uL s as ~ ris,._!
. .!A
_t ___.._A., *
. a AA _ _L _ _ _ A a A. r- en sw tr~ e erw,~s st, s y n ? _ _ s_s _~ s_sA.,
.s w te ,Jssw !__ . . _ _ mwJas.
w.serem I_ s , m. rs e swa st u LW wv. NUREG-1021 24 of 26 Revision 8 4-
ES-201 Examination Preparation Checkhst Form ES-201-1 i Facility; . Date of Examination: Examinations Developed by: Facility / NRC (circle one) Target Date* Chief
- Task Description / Reference Examiner's initials
'180
- 1. Examination administration date confirmed (C.1.a; C.2.a & b)
-120- 2. NRC examiners and facility contact assigned (C.1.c; C.2.e)
-120 3. Facility contact briefed on security & other requirements (C.2.c)
-120 4. Corporate notification letter sent (C.2.d)
[-90R] [5. Reference material due (C.1.d; C.3.c)]
-60R 6. Integrated examination outline (s) due (C.1.d & e; C.3.d)
-55R 7. Examination outline (s) reviewed by NRC and feedback provided to facility licensee (C.2.h; C.3.e) -
-30 8. Preliminary license applications due (C.1.k; C.2.g: ES-202) l
-30R - g. Draft examinations, supporting documentation, and reference materials due (C.1.d, e, f, & g; C.3.d)-
-14 '10. Finallicense applications due and assignment sheet prepared (C.1.k; C.2.g; ES-202) '
-14 11. Examination approved by NRC supervisor for facility review
' (C.2.h; C.3.f) -
-14 12. Examinations reviewed with facility (C.1.i; C.2.f & h; C.3.g)
-7 13. Written examinations and operating tests approved by NRC supervisor (C.2.i; C.3.h)
-7 14. Final applicatons reviewed; assignment sheet updated; waiver
. letters sent (C.2.g)
- 15. Proctonng/wntten exam administrabon guidelines reviewed with
;-7, facility licensee and authorization granted to give written exams (if applicable)(C.3.k)
- 16. Approved scenarios, job performance measures, and questions distributed to NRC examiners (C.3.1)
Target dates are keyed to the exam date except where noted with an "R" for review date. All dates are for planning purposes and may be adjusted on a case-by-case basis. () Applies only to examinations prepared by the NRC. NUREG-1021 25 of 26 Revision 8
ES-201 Examination Outhne Form ES-201-2 Quality Assurance Checklist
' Facihty*
Date of Examination: item Task Desenption Initials a b c
- 1. a. Venfy that the outline (s) fit (s) the appropnate model per ES-401,
- b. Assess whether the outline was systemmatmally prepared and whettier all em-know; edge and g few. ability categones are appropriately sampled.
- c. Assess whether the outhne over emphasizes any systems, evolutions, or generic topics.
E N d. Assess whether the repetition from previous examination outlines is excessive. .
- 2. a. Using Form ES-301-5, venfy that the proposed scenario sets cover the required number of normal evolutions. instrument and component failures, and major transsents.
S l b. Assess whether there are enough scenano sets (and spares) to test the protected number and M mix of applicants in accordance with the expected crew composition and rotagon schedule without compromising exam integrity; ensure each appleant can be tested using at least one new or significantly modified scenario and scenarios will not be repeated over successhe days.
- c. To the extent possible, assess whether the outline (s) conform (s) with the quahtative and cuantitative enteria specified on Form ES-3014 and described in.*;;: 4rx D.
- 3. a. Verify that the outhne(s) contain(s) the required number of control room and in-plant tasks and wenfy that no more than 30% of the test matonalis repeated from the last NRC examinaten.
W
/ b. Vertfy that the tasks are distnbuted among the safety function groupings as specified in ES-301; T one task shall require a low-power or shutdown condition, one or two shall require the apphcent to implement an altemate path pr*e. and one should require entry to the RCA.
- c. Venfy that the required adminletrative topics are covered, with emphasis on performance-based actwilies
- d. Determine if there are enough dlSerent outhnes to test the projected number and mix of appbcants and ensure that nowness then 80% of4he items are; dE r ' on successive days.
- 4. a. Assees whether plant-specific priorilles (including PRA and IPE insights) are covered in the appropnote exam sedian G
E b. Assoas whether the 10 CFR 55.41A43 and 55.45 samphng is appropnete N-E c. Ensure that K/A r; : _- ^ 2 febngs (except for pient *M priorities) are at least 2.5.
- d. Check for dupbcation and overlap emong exam sections.
L
- e. Check the entre exam for balance of coverage
- f. Assess whether the exam fits the appropriate job level ('A O or SRO).
- a. Author --
- b. FacultyReviewer(')
- c. ChiefExaminer
- d. NRC Supervisor
(*) Not applicable for NRC developed examinations. NUREG-1021 26 of 26 Revision 8
3-f s 1 f o E ot esly . t 0 o i i T 8 2- o s a ent ane de c. A n E S eddot b aisi ias i ii d d i n p i p D o is t o Iv h t ne v i o6 i v n ;mcom ip i m r sa t e e o eto eim-s in nio s
)
2 R vef rhl n ( F ic o l amh t E acst hnnf w sax t
, R f o ao noI ne d-ie nf U
T ohmha o e. be ao x d; A
)
(s sf wromb ae s ev e o n ; b' N k ne ml r eopaieh xv c on a gt mle ;. GI es r eet spt S w e d gha y a iy l nm ::; epvmt i t m eo he c i ; h yo l t nr sdicy .
'orapnnai r ct eaf w t
Ceh . E f o d t oisee c Rt C. , T A es eeet he
,l r Nt l i ^ .
nt sdr s e n :.' D deu alat oauhu enon l hu h eio t t gn , hmvdermt c a m m e1 smeeht r nsi me; , . sa nx nte osfW. n m ee:c. C
)
1 iNR t ia a cr ( oeii it ecmI gx ng d a t
.a e cym e oa E R
w n iet mhs sumCna r d t ntnM o u y h t U t n aRoh Nt i t T at xu ie c s ha ecL db A
. n mss N g
i eo g b =i oht b tyese ni ao r e G A m a t t o i L.ozn fnth I S t y sn n o td _le deg u ecgu bme i yt o~ ,th n u ceha nm U drr et os nnou ai t a c rf r sd ky 6 S e Co" Rf n r
. cshehon t o
s n recl oebcaa l 2 f o n Ni ;~ t s aa un n st r ndf i o y c n o eeei c 6 i e nt alyrh pieef 2 - t a hat t cl /o c a dt p m tu ge n oliadi pi cnd e aes zhc Y T pfian m bol vumac esy i nt otea aB n s I L a x aia epn n hamtp I E edI gyt o s soia t udr e S I dlgh ahs ar e t aef o rc x N l e mt t ,ta m m nhee ut p O
~--
wowd teapm m yme s , P o an adxe cx x a n odno ari S E kn knt s ae e ~oFvh R dt i eo znd n - ifh r e t ' n, hefe ot i t ge
. os r pin rm o
/
E L a oc l l R aiu~ .t
. u v ,a T L iwI c
eI r. ;ibseR pt
;t nC r d
i te ex I T ag B sa en ;. ni e N l t o l u n O dthi" .mle e n" f vn ais J - e e m dch do a d)ste ,ie r ant r i ie ur x qge 1n o ca to I
,( cl c -
caf ek aI e i inn
,taie gerues t n e .h . nlt m n dese E t
o vecZ.ms.iu e ar huC ;aeg i o l ewnh it M A _ s n I t aR^ .eyn xr a t a n wethod ot n er N n agnN ,..f a a i i m t m k gd e D a omm a yi ni rd st E x t sht x i T nt y muI ,in hied ey _ E- E- N e g dmb y oni iei t t l f odod d nm I emca t r s R 1 2 P l efod
,l pef o tsrebaa P 0 1
0 w ez' _meo ot e P eer e btss t r 1 mdo ar' ort i cgt eiie - hi nnw G 2- i h _ l aro S c usp t mmo i E _ a et E 1. 'l hu# t a ni uthe r 2
.- oddh Taaw . . .
R U 4 5 6 7 8 9 1. 11 12 1 3 4. 5 1 . 2 3.
. . . . . . 0 .
4 1 1 N p
' ES-202 PREPARING AND REVIEWING OPERATOR LICENSING APPLICATIONS A. PURPOSE This standard provides instructions for facility licensees and applicants to prepare and the NRC to review initial licensing applications. It also discusses the experience, training, education, and certification requirements and guidelines that an applicant should satisfy before being allowed to take an NRC reactor operator (RO), senior reactor operator (SRO), or limited senior reactor L operator (LSRO) licensing examination. B. BACKGROUND
. The Commission-approved licensed operator training programs at most power reactor facilities are based on a systems approach to training (SAT) and use simulation facilities that have been either certified by the facility licensee or determined to be acceptable by the Commission under 10 CFR 55.45(b). In accordance with 10 CFR 55.31(a)(4), these facilities are not required to include details of the applicant's qualifications, experience, and training on the NRC license application form, in lieu of these details, the Commission will generally accept certification by an authorized representative of the facility licensee that the applicant has successfully completed the facility's Commission-approved training program.
If the facility licensee does not have a SAT-based licensed operator training program that uses a simulation facility acceptable to the Commission, the NRC will not accept the license application unless it includes the details of the applicant's qualifications and training. Detailed license
" eligibility requirements and guidelines are derived from 10 CFR Part 55, Subpart D,
. Applications," and Regulatory Guide (RG) 1.8, Revision 2, " Qualification and Training of
- Personnel for Nuclear Power Plants," respectively. With respect to license applicants, RG 1.8 endorses, with exceptions, the' guidance in American National Standards institutel American Nuclear Society (ANSl/ANS)-3.1-1981, " Selection, Qualification and Training of Personnel for i
Nuclear Power Plants." NRC examiners should refer to those documents as necessary when evaluating the eligibility of applicants at facilities that do not use an NRC-approved or facility-
- certified simulator as part of a SAT-based licensed operator training program. j I
C. RESPONSIBILITIES. The regulatory requirements as'sociated with the license application process are detailed in Subpert D, " Applications," of 10 CFR Part 55. The medical requirements for license applicants and licensed operators appear in Subpart C, " Medical Requirements," of 10 CFR Part 55. l These requirements should be referred to as necessary when preparing and reviewing license l applications. 3 a w
- 1. Apolicant/ Facility Licensee a.
To apply for an RO or SRO license, an applicant must submit an NRC Form 398,
" Personal Qualifications Statement - Licensee," and an NRC Form 396,
" Certification of Medical Examination by Facility Licensee." (Computer-NUREG-1021 1 of 12 Revision 8
i ES-202 generated duplicates are acceptable.) The application is not complete until both forms are filled out, signed by the appropriate personnel, and received by the NRC. Detailed instructions for completing NRC Form 398 are provided with the form. Additionalinstructions regarding waivers of training, experience, and examination requirements are provided in ES-204. If the applicant is reapplying after a license denial,10 CFR 55.35 applies, and the applicant must complete and submit a new Forme 398 end-396 The applicant may file the second application two months after the date of the first final denial, a third application six months after the date of the second final denial, and successive applications two years after the date of each subsequent denial. Each new Form 398 shall describe the extent of the applicant's additional training since the denial and shall include a certification by the facility licensee that the applicant is ready for reexamination. If the applicant previously passed either the written examination or the operating test, he or she may request a waiver of that portion of the licensing examination. Such waivers are limited to the first reapplication and must be requested within one year of the date of the failed examination. Refer to ES-204 for a more detailed discussion of this and other waiver criteria. The medical data in support of NRC Form 396 are normally good for six months from the date of the medicalcramination. If, because of a delay in administering the examinations, more than 6 months have passed since the date of an RO or SRO instant applicant's medical examination, the facility licensee shall certify in ! writing that the applicant has not developed any physical or mental condition that would be reportable under 10 CFR 55.25. If it is likely that the time since any applicant's last medical examination will exceed 24 months before the licensing action is completed, the applicant shall be reexamined by a physician and the facility licensee shall recertify the applicant's medical fitness on NRC Form 396. However,-if an applicant is reapplying after withdrawing a previous application or accepting a final license denial,' he or she may request a waiver of a medical reexamination by checking item 4.f.4 on NRC Form 398. The time since the last medical examination can not exceed 24 months and the applicant must certify Ch;; prei.Cr.;; :n r7;r.;O . in item 17, " Comments," of the form that he or she has not developed any physical or mental condition that would be reportable under 10 CFR 55.25 er ;d:c.: ..;; ; ::;;:d; ::";; .dh th; ;pp::e_O .. b. Each applicant (except those applying for an LSRO license) must satisfactorily complete the NRC's generic fundamentals examination (GFE) section of the w _ ritten operator licenting examination for the applicable vendor. Refer to ES-
-205 for more information on the GFE program.
Applicants do not need to take the GFE if they were previously issued an RO or I
. NUREG-1021 2 of 12 Revision 8 L.
p ES-202 E SRO license'or an instructor certificate based on a site-specific written l examination (on the same type of facility) that was administered between February 1982 and November 1989 and included the material covered by the GFE. Enter the date of the examination in item 4.g on NRC Form 398 and an explanation in item 17; a waiver is not required. c. As noted in ES-201, the facility licensee should submit preliminary, uncertified license applications and medical certifications for review by the NRC regional office at least 30 days before the examination date. This will permit the NRC to make preliminary eligibility determinations, process the medical certifications, evaluate any waivers that might be appropriate, and obtain additional information, if necessary, while allowing the facility licensee to fiftish training the applicants before the certified applications are due, d. The facility licensee's senior management representative on site must certify . when an applicant has completed all of the facility licensee's requirements for the I desired license level (i.e., experience, control manipulations, training, and medical). Such certification involves placing a check in item 19.b of NRC Form 398, signing the form, and submitting it to the NRC regional office at least 14 days before the examination date. l The facility must also submit a written request that the written examination and ! operating test be administered to the applicar t. e. When the NRC regional office denies a license application, the applicant may not accept the proposed denial. In such instances, the applicant may request that the Director, Division of Reactor Controls and Human Factors (DRCH), Office of Nuclear Reactor Regulation (NRR), review the application denial or request a
. hearing in accordance with 10 CFR 2.103(b)(2). Further action will be taken in accordance with ES-502, 2.- . NRC Rea_ lonal Offirm o .. -
a. The NRC regional office shall review the preliminary applications as soon as possible after they are received. In that way, the regional office can process the medical certifications, evaluate and resolve any waiver requests in accordance { with ES-204, and obtain from the facility licensee any additional information that
; might be necessary in order to support the final eligibility determinations.
~
J With regard to the medical certifications, the regional office shall forward the
^
applicant's NRC Form 396 and the supporting medical evidence to the NRC
" physician at the Headquarters Health Unit or the regional contract physician for evaluation any time the examining physician has recommended that the applicant be issued a restricted license or that an existing restriction be changed (by checking block A.4 or A.5 on Form 396). l
{ i
'NUREG-1021 3 of 12 Revision 8 !
l
ES-202 The NRC will not process a retake application if the applicant's request for reconsideration or a hearing on the previous license denial is still outstanding (refer to ES-502). Before entering the applicants' data in the operator licensing tracking system (OLTS), the region shall verify that none of the applicants' names appear on the
" Restricted Individuals List" found on the NRC's intamal web site at
' http://www.nrc. gov /OE/ rpr/ia.htmund;- th; A;pacy ?/i ;pp::e;;;;n; cf th; 7"'O': l;;;' ;re; n;Mes. The region shall check with the appropriate contact in the Office uf Enforcement by telephone or electronic mail to verify that the information on the subject individual is current before using the information on the list to deny a licensing action. ~
- b. The regional office will verify that the applicant has successfully passed the GFE, if required, and review the data on NRC Form 398 to ensure that it is complete.
Affirmative responses to items 12.a and 12.b of NRC Form 398, indicate that the ' applicant has successfully completed a Commission-approved, SAT-based training program that uses a simulation facility acceptable to the Commission under 10 CFR 55.45(b). If the facility licensee checks "yes" in response to these items, the licensee need not complete items 13, " Training," 14, " Experience," and 15, " Experience Details," of NRC Form 398. The region may accept the application without fusther review unless there is reason to request further information conceming the applicant's qualifications. ~ Occasionally, a facility licensee completes items 13,14, and 15 even though i
~ they are not required as explained above. In such instances, the region may review the information provided against the eligibility guidelines in Section D for the requested license level and resolve any deviations with the facility licensee.
New applications must still include the number of significant control manipulations in item 13.3; at least five are required on the facility for which the license is sought. This requirement can only be waived or deferred under the conditions specified in 10 CFR 55.31(a)(5); situations other than those specified would require an exemption in accordance with 10 CFR 55.11. For ROs applying for an SRO license, certification that the operator has successfully operated the controls of the facility as a licensed operator shall be accepted as evidence of having completed the required manipulations. If an applicant checks "no"in response to items 12.a and 12.b on Form 398, the (
~*
region shall review the application against the specific RO, SRO, or LSRO i eligibility guidelines described in Section D. if the applicant is reapplying after a previous examination failure and license
' denial, the region shall evaluate the applicant's additional training to determine if NUREG-1021 4 of 12 Revision 8
n 1 ES-202 the facility licensee made a reasonable effort to remediate the deficiencies that caused the applicant to fail the previous examination. c. The region may determine that the preliminary application is incomplete, that more information is necessary to make a waiver determination, or that the applicant does not meet the requirements in 10 CFR 55.31. In such instances, the region wil1 note the deficiencies and request that the facility licensee supply additional information when it submits the final, certified license application (or soonerif possible). Conversely, the region may determine that the preliminary application is - complete, and the applicant meets the eligibility requirements oris expected to meet the requirements pending the receipt of additionalinformation, in such instances, the region shall enter the applicant's name, docket number, and examination requirements on the examination assignment sheet in accordance with ES-201. d. Upon receiving the final, certified license application, the reviewer shall evaluate any new information to ensure that the eligibility criteria are satisfied, if so, the , reviewer shall check the " meets requirements" block at the bottom of Form 398, ! sign and date the form, if necessary, the reviewer shall add the applicant's name and other data to the examination assignment sheet in accordance with ES-201. The reviewer shall also ensure that the assignment sheet accurately reflects any examination waivers that may have been granted in accordance with ES-204. If the region determines that the applicant still does not meet the eligibility
. requirements, the regional licensing authority will discuss its decision with the Operator Licensing Branch (OLB) and notify the applicant in writing that the
.. application is being denied and identify the deficiencies on which the denial is based (AttachmeM 1).- The responsible regional supervisor, or designee, shall check the "does not meet requirements" block at the bottom of Form 398, and shall sign and date the form. ' The applicant's name shall be stricken from the examination assignment sheet; the applicant shall not be permitted to take the licensing examination until the region determines that he or she meets the eligibility criteria.
With prior approval from OLB, as necessary, in accordance with ES-204, the region may administer a license examination to an applicant who has not satisfied the applicable training or experience requirements at the time of the examination, but is expected to complete them shortly thereafter. Assuming that the applicant passes the examination, the region shall not issue the applicant's license until the facility licensee certif;es that all of the requirements have been completed. (Referto ES-501.) D. LICENSE ELIGIBILITY GUIDELINES NUREG-1021 5 of 12 Revision 8
l ES-202 Threugh lt; t;l;;f; j an; n;;;;na (O *,) pr;;r;.m, ;;y;;; r;pe.1 (IO? n), t;;hn ;;l ;p;;;' c;t!;n; UC), et qu;:lty
;h f;;;;;;illcn;n h;;;;mm:r;d t;;pa':; guld ::na for nu;::;r p=;;r p ;nt ^;;;rbr qu;:l':22n; (;.g., ?,N0! 10.1 10 't er ?,NO!/?,NC 3.11001).'.^!h;n r=:=;lng th; opgl cent;' :::g:b :::j, NCO :nm'n;;; mu;t n;ur; th;; th; prep;r f;;llty ! cenx; l
;cmmltm;nt l; und. Th; NCO r;;l;n;l c" n ;heuld r;f;r ;" qus%n; reg;rd:n; ::an;;
- g b lltj t; th; Oh ;f, OLO, f;r Ts;lunn.
' Revision'2 of Regulatory Guide (RG) 1.8, " Qualification and Training of Personnel for N Power Plants," describes a method acceptable to the NRC staff for complying with the Commission's regulations with regard to the training and qualifications of nuclear pow personnel. For the positions of shift supervisor, senior operator, and licensed operator, this RG generally endorses the guidelines contained in ANSI /ANS-3.1-1981: specific clarifications, additions, and exceptions are noted in Section C, " Regulatory Position," of RG 1.8. The lic eligibility guidelines in RG 1.8, Revision 2, and ANSI /ANS-3.1-1981 are summarized below.
{.y suji':as m;y n;t b; ;ppl:sbl; t: ;ll f;;;;;;j ::anns ;nd ;h;" net b; anstru;d ;;
.v......,.,.....
The for NRC regional resolution. office should refer all questions regarding license eligibility to the
- 1. Reactor Operator
- a. Experience (1)
The applicant should have a minimum of three years of power plant experience, at least one of which is spent at the nuclear power plant for which the license is sought (preferably in the performance of nonlicensed operatorduties). (2) The applicant should spend at least six months performing plant operational duties as a nonlicensed operator at the nuclear power plant for which the license is sought.
- b. Training (1)
The applicant should complete at least 13 weeks as an extra person on shift in training for the RO position. This training should include all phases of day-to-day operations and be conducted under the supervision oflicensed personnel.
,1(2)
.s #' The applicant should be trained in nuclear power plant fundamentals and plant systems, use of those systems to control or mitigate an accident during which the core is severely damaged, and operating practice.
1 (3). The applicant should complete at least 500 hours of lectures on the principles of reactor operation, design features and general operating NUREG-1021 6 of 12 Revision 8 I-L L
}
l l i ES-202 characteristics of the nuclear power plant involved, instrumentation and control (IC) systems, safety and emergency systems, standard and emergency operating procedures, and radiation control and safety procedures. (4)~ The applicant should satisfactorily complete an NRC-approved training program involving at least one week at a nuclear power plant simulator. The simulator training center should certify the applicant's ability during a rea'ctor startup to manipulate the controls, keep the reactor under control, predict instrument responses, use instrumentation, follow procedures, and explain annunciator alarms that occur during operation.
-(5) The applicant must manipulate the controls of the reactor (not simulator) during five significant changes in reactivity or power level (refer to 10 CFR 55.31(a)(5)). Every effort should be made to diversify the reactivity and power changes for each applicant. Startups, shutdowns, large load i changes, and changes in rod programming are some examples; these I changes could be accomplished manually using such systems as rod control, chemical shim control, and recirculation flow control.
- c. Education 1' i
The applicant should have a high school diploma or equivalent. , 2. Senior Reactor Ooerator
. a. Experience (1) The applicant should have a minimum of four years of responsible power plant experience, as defined in RG 1.8. At least two of those four years should be nuclear power plant experience.
(2) The applicant should have actively performed licensed RO duties for at least one year at the facility for.which the SRO license is sought. The NRC may accept any one or more of the following education or experience' qualifications to satisfy this requirement provided that the applicant supplieo sufficient details in the license application for the staff to make a. judgement regarding equivalence: A four-year degree in engineering or the equivalent (e.g., a degree in engineering technology or the physical sciences that includes course work in physics, mathematics, or engineering; a professional engineer's (PE) license obtained by passing the PE examination). NUREG-1021 7 of 12 Revision 8
ES-202 At least one year as an active licensed RO at a comparable facility (same vendor, similar vintage) or 18 months as an RO at a noncomparable commercial power reactor. At least two years in a position equivalent to a licensed RO at a military reactor.- Experience obtained in licensed positions (or their equivalent) on other large-scale reactors will be evaluated on a case-by-case i basis. Applicants must also submit a waiver request in accordance with ES-204 if they want this experience to apply toward the requirement. ~ (3) At least six months of the applicant's nuclear power plant experience should be at the site for which the license is sought. (4) During the two years of nuclear power plant experience, the applicant should participate in reactor operator activities at power levels greater than 20 percent for at least six weeks.
- b. Training (1) The applicant should complete at least 13 weeks as an extra person on shift in training for the SRO position. This training should include all phases of day-to-day operations and be conducted under the supervision of licensed personnel. Any portion of the 13 weeks that is spent at or above 20 percent power may also be used to satisfy the experience
_ guideline in Section D.2.a(4). (2): if the applicant has not held an RO license at the facility and one of the qualifications specified in Section D.2.a(2) is substituted for that experience, the training guidelines of Sections D.1.b(4) and D.1.b(5) should be met. The applicant should satisfactorily complete a training program that is comprehensive in its coverage of both RO and SRO knowledge, skills, and abilities and must take an SRO-instant license examination. (3) 1he applicant should be trained in nuclear power plant fundamentals and
' ~p lant systems, use of those systems to control or mitigate an accident during which the core is severely damaged, and operating practice.
(4)- The applicant should also complete the additional instruction specified in Section 5.2.1.6 of ANSI /ANS-3.1-1981 in subjects related to the duties of an SRO. NUREG-1021 8 of 12 Revision 8 J
ES-202
- c. ~ Education
' The applicant should have a high school diploma or equivalent.
. 3.' Limited Senior Reactor Ooerator
. a. Experience
- The' applicant should have three years of nuclear power experience that includes :
active participation in at least one refueling outage at the site for which the license is sought or at a similar facility. Six months of the nuclear power plant experience must be at the site for which the LSRO. license is so0ght or at a similar facility owned by the same facility licensee.
- b. Training The applicant is expected to have satisfactorily completed a training program
- that ensures that he or she is qualified to supervise fuel handling operations.
The program should be based on a systems approach to training and is expected to inc5ude instruction in at least the following ' areas: (1) nuclear power plant and health physics fundamentals and the principles of reactor theory and thermodynamics (2) design features of the nuclear power plant pertaining to fuel handling activities, including plant systems and equipment associated with fuel handling operations, pertinent IC systems, and features of the emergency i core cooling systems (ECCSs) associated with the refueling mode of l operation j (3) the use ofinstalled plant systems to control or mitigate an accident in
- which the core is damaged during refueling operations i i
(4) operstmg practices and procedures that pertain to refueling, including administrative,' operational, surveillance, emergency, radiation control, and safety procedures; the technical specifications applicable to refueling; and the requirements concerning communications and interfaces with the main control room
, The applicant should also complete a minimum of 80 hours of on-the-job
~ : training (OJT) in refueling activities, including manipulation of the refueling bridge or similar refue!ing equipment.
- c. Education NUREG-1021 g of 12 -
Revision 8 9
ES-202
. The applicant should have a high school diploma or equivalent. '4. Cold License Elioibility Cold examinations are those administered before the unit completes preoperational testing and the initial startup test program as described in the FSAR.
- a. Each applicant must satisfactorily complete the training programs described in
'Section 13.2 of the FSAR and approved by the NRC. The NRC's review and
- approval are based on information contained in Section 13.2.1 of the Standard -
Review Plan (SRP)(NUREG-0800). Note
- These NRC-approved training programs typically require ten startups on a research reactor. This requirement may be waived if the applicant has completed a plant-referenced simulator training program accredited by the Institute of Nuclear Power Operations (INPO). ,
- b. In lieu of the control manipulations on the facility for which the license is sought (per 10 CFR 55.31(a)(5)), the Commission may accept evidence of satisfactory performance of simulated control manipulations as part of a Commission-approved training progsm on a simulation facility acceptable to the Commission under 10 CFR 55.45(b).
E. - ATTACHMENTS / FORMS , " Sample Initial Application Denial from Region" NUREG-1021 10 of 12 Revision 8
l l l ES-202 Sample initial Apphcation Denial from Reaion Attachment 1 \ j NRC Letterhead
-(Anolicant's name)' (date) !
. (Street address) ;
(City. State. Zio code) l
Dear (Name):
This is to inform you that your application of { data) for a (reactor coerator. senior reactor operator) license submitted in connection with the (facility name) is hereby denied. ) (Reoion to discuss deficiencies and which oart of 10 CFR 55 31. ES-202 NRO. facility trainino orocram. or Reoulatorv Guide 1.8 was involved.) When you have met the requirements of 10 CFR 55.31, you may submit another application. If you the do not following accept actions: this denial, you may, within 20 days of the date of this letter, take onei You may request that the NRC reconsider the denial of your application by writing Director, Division of Reactor Controls and Human Factors, Office of Nuclear Reactor Regulation, U.S. Nuclear Regulatory Commission, Washington, DC 20555. Your request must include specific reasons for your belief that your application was improperly denied. If the NRC determines that the denial of your application remains as described below. appropriate, you still have the right to request a hearing pursu You may request a hearing in accordance with 10 CFR 2.103(b)(2). Submit your request, in writing, to the Secretary of the Commission, U.S. Nuclear Regulatory Commission, Washington, DC 20555, with a copy to the Assistant General Counsel for Hearings and Enforcement, Office of the General Counsel, at the same address. If you have any questions, please contact (name) at (teleohone number). Sincerely,
.(Regional branch chief or above)
Docket No. 55-(number) cc: (FaeW V renr-antative who aloned the anelicant's NRC Form 398) CERTIFIED Mall- RETURN RECEIPT REQUESTED NUREG-1021 11 of 12 Revision 8 i
ES-204 PROCESSING WAIVERS REQUESTED BY REACTOR OPERATOR AND 5ENIOR REACTOR OPERATOR APPLICANTS A. PURPOSE This standard provides guidance conceming the processing of waivers requested by. reactor operator (RO) and senior reactor operator (SRO) license applicants at power reactoi facilities. y B. a. ga .A ' BACKGROUND. j^ 69 T' di# \.RuF[ , e in accordance with 10 CFR 55.35, "Reapplications," and!10'CFR 55.47l" Waiver of ExaminatiorP and Test Requirements," an applicant may request to biexcused frohia writteh Asaiinination an operating test. The NRC may waive any or all of thejiaminatiddrequireme ~ determines that the applicant has presented sufficient justifidatidni in an effort to expedite the resolution of applicant requests, the NRC Operator LicerYsihgBfsnch (OLB) has delegated the authority to grant routine waivers of certain operator licensing requirements to the NRC regional offices. [ 'N '6 C. RESPONSIBILITIES m e/ h ,, .,
'c1*
% , ,
- b ',
. $j)'s ,y b
- 1. Aeolicant/ Facility Licensee
'%f a.
An applicant may request a waiverof a license requirement by checking the appropriate block injtern3 f on hlRC Form 398, " Personal Qualifications
}
Statement - Licenseep?The applicant shoi&i also explain the basis for requesting the waiver in itemM7, " Comments."
.b.
Ak h [b Td[
%The facility:8consee's seniormanagement representative on site must certify the j@dNnallicerjNM'ed!on, s Selverreque ther*S6yTubstantiating the basis for th MIh$kNA g c.
Facihtf*lidensees units designed by the same nuciear steam supply
/ stem %,and $khted dualli6ensingjopheir
~ at approximately operators. Similarly, if the units of a multi-unitthe facility same are po nearly id55bcadOIe facility licensee may request a waiver of the examination
.A [f requiremhipthe second and subsequent units.
p
=
In eitherdse, the facility licensee must justify to the NRC that the differences betwed[the units are not so significant that they could affect the operator's I abgIo operate each unit safely and competently. Further, the facility licensee must submit for NRC review the details of the training and certification program. he analysis and summary of the differences on which the applicants must be j trained willinclude the following, as applicable: i
' s
- 1 facility design and systems relevant to control room personnel j
. technical specifications
' NUREG-1021 1 of 5 Revision 8 l 1
I l
t ES-204 procedures (primarily abnormal and emergency operating) t a control room design and instrument location operational characteristics f administrative procedures related to conduct of operations at a multi-unit { ' site (e.g., shift manning and response to accidents and fires)g
^ the expected method of rotating personnel between units arid the refamiliarization training to be conducted before aifoperathiassumes responsibility on a new unit t $$iMW -
- 2. NRC Reaional Office a.
j ff i The regional office will evaluate waiver requests on a case-by-case:bd Is against the waiver criteria discussed in Section Dlofjthis ES. NggjP b. N$dfy$%. The regional office may grant routine waivgs identified in Section 0.1 without first obtaining OLB concurrence,
.m f
However, waivers of experience requirements, com f3{ ,mg,&e. of training, or completion of examinations (e.gfdheyshericfundamd$tAlsexamination) not specifically identified in SectionV.1 munk. approved b"y OLB. The regional office should evaluate theMaIver req recommendation to the CNIef, OLBgfo%@fo$la@its approval r concurrence. Abk Af Ik
' The region does notroquire written OLB concurrence to deny an applicant's
- waiver request, bd(shou!diijiEuss itsMik:ision with OLB before infomling the applicant; formal [6Encurreru:feI*
[55sirable in some cases. A@ dditiorist:ihf@ormation o reach ais reI on the waiver request, the decision 756Ior) shalfheg$fa$hgequest the necessary information from the facility licens *
- "acEcrdance w'disS1202. i
'3MM ' - 3pfy/
- d. _
1 to grsk6r deny a waiver, the regional office shall promptly notify the a. p m friting concoming the disposition of the request [includiryar) t expiaria, ion f6rkdenial. If time is too short to notify the applicant in writing before thElilIA$IIInstion Ute, the regional office shall notify the facility training g represenhivlby telephone conceming the disposition of the waiver request and i! provideJrfollow-up written response to the applicant. The regional office shall y includab OLB Branch Chief on distribution for all waiver disposition letters. jY
,, region shall document the disposition of every waiver request, whether granted or denied, by completing the block designated "For NRC Use" on the applicant's NRC Form 398 and by entering the data in the operator licensing tracking system (OLTS).
- f. .
NRC examiners assigned to a particular examination will be notified of approved
- NUREG-1021 2 of 5 Revision 8
ES-204 waivers by the appropriate regional supervisor and by an entry on the examination assignment sheet (ES-202, Attachment 4). g. If the applicant is determined to be ineligible to take the licensing examination, the regional office shall issue a denial letter in accordance with ES-202. D. WAIVER CRITERIA -
, S'
% ."~
g~p i d,)
\
- 1. Routine Waivers N Yk ,,
- a. ff
' If an applicant fails only the written exammation A b k la $inor one category of.thel g
. test, the region may waive those examinElon areasTeategories)'th5tMFEF .
passed. This is only applicable for the fiishetakfixamination andbhlYifit takes i place within one year of the examination ihit ihEhpplicant failed. y!!ges@A
- b. '
The region may waive training requirements si$iSfi$0Ih the final safety analysis report (FSAR) when the FSAR authorizSh waiver *dfMA$$s i and the applicant otherwise meets NR feguiremerifile.3)pecific Waiver of some requ training requirements for applica6 tim (viously licensed c.
.fV &Q$$% j
. The medical data in supporfof NRC Form'395NMnorm$* alygood for six months from the date of the medidl examin$ lion fMAM5n apply'ing for an RO or an SRO instant license.g Freapplichiions folledirig a license denial or withdrawal.
of an application, wsisers extenhing the sixYnonth period may be granted if the date of the originaIMidical ex%nination IE%ithin ca; y;;r 24 months;of the O.;d i d n z.
. ^pCform r 3#ce%Sa anildkated.filinsing datejand itemj7lfComm rtifies]ha(Qagplic~ ant. has;not; developed ;antphysicalfor of -!
Ihmental:co'n8iE6 Eat:would Sireportable u' nder10 CFR:55.25. For renewal and SR6Mgradfijihliisnt.s,"ihe rnedcal edminat' Ion"documerded on NRC Form
~g ~
MQQ yV
- d. . ballowe . ' Regulatory Guide 1.8, Revision 2, are not considered to
_be wWa.5hd#therefore, do not require approval. For example, substitution of
- related t$dii@de[ifalning for up to two years of experience for an SRO or up to one year (kisjhO is not a waiver. However, training for the examination applied fopmay not be counted as related technical training.
- e. If the ty licensee certifies that the applicant has successfully completed a tr BFygar% program accredited by the Institute of Nuclear Power
_@{' iacceptable simulation facility, the region may waive the requirement for ten pJffis#itartups on a research reactor typically required by NRC-approved cold license training programs.
- f. For those facilities unable to meet the requirement for six weeks on shift at greater than 20 percent power (because of extended plant shutdowns or other NUREG-1021 3 of 5 Revision 8 w-
l 1 ~ 7 l 1 l ES-204 extraordinary circumstances), this requirement may be waived upon application if the following criteria are satisfied: (1) Facility training objectives for the desired licensed position have been developed using a properly validated job and task analysis (JTA); , g?
.(2) The facility licensee's training ased.on i program is)A ;6 a systems approach to training (SAT) usinig~the five elemehts defined in 10
' CFR 55.4. f57 jK ig f X k %W8lQ (3) The facility licensee can ace'omplish thi training objsctive required for plant operation lit greati'r than 20 percent power using a plant-referenced ofNRCJspproved simulation fadity.
NinMMb g.~ If an operator was previously licensed at s'fai6ilitish$ reapplies for a license at the same facility and license level, the Egion maNp6Fsdiant to 10 CFR 55.47, waive the requirement for the applicant to pass a nitiedixamination and an 46diO[fjh cperating test if it kP finds that the app %^[7 y #%WM!hk (1) previously d safelyeand;is,ischarged.hisyfKsr
' capable %f co r < to"do so responsibilities co)
Ajh '
\
(2) terminated parti ' tion in t faci!ity licensee's requalification j progiiiWiess thiritwo yeadbefore the date of the license ' appDestion
- hAIf (3hgsuccessfully I$ -
" Additional Training," pursuant to 10 CFR j A,.%
~
%y557 ' b), andT Eility-prepared written examination and
' test which ensure that the applicant is up-t&date in the 1"** $^p erator requalification training progratri t l
ph. .. % %g 44%' Will:succe[ssfully completed at least 40 hours of shift) am Qer the direction of an operator or senior operator, as appropriate, and in the position to which the applicant will be f,g assigned (see 10 CFR 55.53(f));before;beinglassigned torlicensed duties 1,T 9
, ) complies with the requirements of 10 CFR 55.31
. ,an. applicanf {styfacilitythat.has"complet.ed praioperational testing is: unable.to Perform'the;five.'significant.'contro[manipulationsfrequir,ed, by;i1,0M}52t (a)(5) because;of an extended shutdown,Ethe regon maycprocess;theispplication and
. administer;the. examinat. ion l butLshall:not issue. the.;licenlseLunti[the[ facility Neensee; supplies thepquired; evidence that;theimpplicanthas successfully WC ithe control; manipulations. (refer to ES-501).1 Situations ;other. than NUREG-1021 4 of 5 Revision 8 i
- _ - - - - ~
g L I !~ ES-204 L those specified in the regulationfrequ. ire an exemption and must be processed
- through.OLB; i
2. 1 Examination Waivers for Previousiv Licensed Ooerators at Comoarable Facilitiei s Depending on the justification provided by the applicant and the facility lice $he, OLB will consider examination waivers for operators who were previouisly licerihid at a i comparable facility Pursuant to 10 CFR 55.47, the Commissiorihuiyssive any or all ! requirements for a written examination and operat t. 1
'Q{
- 3. Multi-Unit Examination Waivers kN a.'
Generally, personnel will not be examinadon.or allowed to hold li6ense "different units" simultaneously. "DiffereEUsitdhwned or managecibyi single facility licensee are defined for purposes 5stisifisindard as follows: N$
. units having the same vendor bO[Fsignific$$@a.
antly dif6trent age and/or power N i ' '? > level (e.g., Dresden / D I' Units 1 and 2)k,l s% ~$l) units having the same v'enderland uniiar design but different locations (e.g., Sequoyah and Watts Bar, Byrdirilind Staibwood) units having diffe[entQ'Y jf s vendors (PWR only) but located (e.g., Arkanssis,Onits 1 and 2 Mll Me Units 2 and 3) l l OLB may authoriate MjI limited./k , i stdI
~
senior reaclor operator (LSRO) to be licensed at I multiple site ed that of sid " . The @WMiltilisiNImanufactured Eust pass an examination that addresses by the sam l differences;, g designs, procedures, technical data, and administrative of thigeparatefacilities for which the license is being sought. t
.b. Ao the $kjuddation requirements for " identical" second or 1Uinits at tifsame site, OLB may waive any or all requirements for a
- written .
and operating test if it finds that the applicant meets the criteria s"" 10 CFR 55.47, as noted in item D.2 above. If the situation warrants, ^ mmission may impose other examination requirements, such as NRC ad istered operating tests and written examinations conceming the plant differences.
.Q l' ' N' UREG 1021 5 of 5 i Revision 8 i.
i ES-205 PROCEDURE FOR ADMINISTERING THE GENERIC FUNDAMENTALS EXAMINATION PROGRAM A. PURPOSE-This standard describes the procedures and policies pertaining to administration of the generi
' fundamentals examination (GFE) section of the written operator licensing examination at pow reactor facilities. . It describes how the examinations are scheduled and constructed, how to solicit facility licensees for applicants to take the examinations, and how to promulgate the examination results; B. BACKGROUND Sections 55.41 and 55.43 of 10 CFR Part 55 require that the written operator licensing examinations for reactor operators (ROs) and senior reactor operators (SROs) include ,
. questions on various mechanical components, principles of heat transfer, thermodynamics, and fluid mechanics. These regulations also require that the written examination address l fundamentals of reactor theory, including the fission process, neutron multiplication, source effects, control rod effects, criticality indications, reactivity coefficients, and poison effects.
The fundamental knowledge and abilities (K/As) required of an operator do not vary significan between license levels or among facilities of the same vendor type. As a result, the NRC implemented the GFE program to standardize the fundamental examination coverage for all applicants at pressurized and boiling water reactors (PWRs and BWRs). Having passed a GFE , as an RO or an SRO applicant, an operator will not have to take another GFE unless he or she transfers to a facility of the other vendor type. The GFE program does not pertain to limited senior reactor operator (LSRO) license applicants. Applicants do not need to take the GFE (nor obtain a waiver) if they were previously issued an RO or SRO license or an instructor certificate based on a site-specific written examination (on the same type of facility) that was administered between February 1982 and November 1989 and included the material covered by the GFE. Applicants who were issued a license before > 1982 will have to take the examination or apply for a waiver in accordance with ES-204. The GFE examinations for BWRs and PWRs are typically administered twice a year, on the Wednesday following the first Sunday in April and October. C. RESPONSIBILITIES 1.: Facility Limnsee
. a. -
The facility licensee must certify that all individuals who plan to take tne GFE are enrolled in a facility-sponsored training program that will satisfy the eligibility requirements for an RO or SRO license. The operator trainees need not complete all of the training required for the license before they take the GFE. i NUREG-1021 1 of 13 Revision 8 l
ES-205 The facility licensee may use the sample registration letter enclosed with the NRC notification letter (Attachment 1) or any similar format that contains the required information and certification If the facility 'icensee must add or delete an individual after submitting its registration letter, .. should inform the NRC Headquarters' Operator Licensing Branch (OLB) of the change, as specified in the examination cover letter, before the examinations are administered.-
- b. ~ When the examinations are received from the biFE contractor, the facility licensee shall reproduce and safeguard the examinations as described in the examination cover letter.
- c. On the designated examination day, the facility licensee shall administer and proctor the GFE in accordance with the instructions contained in the examination p6ckage.
The facility licensee will start and stop the GFE in accordance with the time zone map contained within the examination package. Late anivals will be allowed to
' take the examination; however, all examinees must hand in their examinations at the completion time designated in the proctor instructions enclosed with the examination cover letter (refer to Sechon C.2.d).
- d. No later than the day after the GFE is administered, the facility licensee shall send the following items via ovemight mail to the namo and address designated in the examination' package:
)
the original answer sheets the signed exam coversheets the signed security statements'
- 2. NRC Onorator 1 icensina Branch and GFE Contrar*ar a.
OLB will designate a coordinator to oversee the GFE activities with the regional i offices, the GFE contractor, and the facility beensees. l
- b. The NRC' $i send a notification lettor (Attachment 1) to each facility licensee 60 days before the GFE administration date. The letter will notify the facility licensee' of the date of the examination and request a registration letter listing the licensed operator trainees to whom the facility licensee plans to administer the examination. A sample registration letter is enclosed with the notification letter,
- c. The GFE contractor will prepare the examinations as described in Section D of C
. this ES. The examiner assigned responsibility for developing the GFE shall submit the examinations to the OLB GFE coordinator and any other designated ,
reviewers at least 20 calendar days before the scheduled administantion date. OLB will provide comments and recommended changes to the examination NUREG-1021 2 of 13 , Revision 8
ES-205
- author as soon as possible. The final examinations should be ready at leastI days before the GFE administration date.
I d. The' GFE contractor will assemble the approved examination pack described below, and mail the plages to the names'and addresses desig by the participating facility licensees. The examinations should normally b mailed one week before the examinations are scheduled to be a The examination packet will contain the following inform 9 tion, enclosures,1 attachments: cover letter (Attachment 2 is a sample letter)
- proctorinstructions security agreement single copies of appropriate exam, forms A and B exam time zone map sample answer sheet facility docket numbersheet applicant docket number sheet appropriate number of answersheets applicant answer sheet instructions e.
On the day that the GFE is administered, the OLB GFE coordinator and GFE contractor anses, shall be available to answer questions from facility proctors if the need f. When the examination answer sheets are received from the facility license GFE contractor shall score, grade, and tabulate the overall item statistics, and generate facility and regional grade reports for each GFE examination. The contractor shall forward the regional and facility grade toports, including individual scores and copies of individual answer sheets, and corrected answe keys to tt.s applicable regional office for distribution. The GFE contractor shall develop individual item statistics on all questions used on the GFE examinations. - Questions with acceptable statistical characteristics
. shall be moved into the " validated" GFE question bank.
The contractor will provide copies of all grade troorts to the OLB GFE coordinator, along with the following additional items: I
- exam-wide item statistics (PWR and BWR)
- analysis reports of specific items deleted or answers changed corrected answer keys originalanswersheets
~ originalsigned exam cover sheets signed security statements .
NUREG-1021 3 of 13 Revision 8
ES-205 g. The OLB licensing assistant will ensure that copies of the final master BWR and PWR examinations are placed in the NRC's Public Document Room.
- 3. NRC Reaional Office
- a. Regior.al management should assign an individual to coordinate GFE administration in the region.
b.
- The regional operator licensing assistant (OLA) shall assign a docket number to each individualidentified in the facility licensee's registration letter. The OLA shall forward the list of names and docket numbers for each facility to the GFE -
c ontractor, with a copy to the OLB GFE coordinator, no later than 20 days before the examination administration date. c. The regional GFE coordinator should keep the OLB GFE coordinator informed of any changes in the number of applicants scheduled to.take the GFE at any facility. d. The regional office shall distribute the GFE examinations to their respective I facility licensees. Sample cover letters for facility licensees that did and did not participate in the examination are provided in Attachment 3. e.- The regional OLA shall update the applicants' status (pass or fall) in the operator licensing tracking system (OLTR) and ensure that a hard copy of the GFE results is placed in each applicant's docket file. D. EXAMINATION SCOPE AND STRUCTURE Each GFE shall contain 100 questions covering the " Components" and " Theory" (including reactor theory and thermodynamics) sections of NUREG-1122, " Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Pressurized Water Reactors," or NUREG-1123,
" Knowledge and Abilities Catalog for Nuclear Power Plant Operators: Boiling Water Reactors."
The passing grade for the GFE is 80 percent. The kr-c;if-;+ and ability (K/A) topics applicable to the GFE for PWRs and BWRs have been M;-:--sed into various component, reactor theory, and thermodynamics groups as shown in Attachment 4. The' attachment also identifies the number of test questions required to evaluate each topic. The NRC wa use Institute of Nuclear Power Operations (INPO) Academy Documents 89-003,
" Generic Fundamentals Test item Catalog - PWR Operator," and 89-004, " Generic Fundamentals Test item Catalog - BWR Operator," as the primary sources of test questions when developing the GFE. The ratio of previously used (i.e., " validated") test questions to new
.or unvaldsted test questions will be adjusted as the size of the validated question bank l increases.
NUREG-1021 ' 4 of 13 Revis'Ca 8 ! 1
F l. I ES-205
- l. Theguidelines style questions u~ sed in Appendix B. on the GFE examination shall conform with the applica
{ E. ATTACHMENTS / FORMS Attachment 1, - " Sample Notification Letter' Attachment 2, " Sample Examination Cover Letter"- Attachment 3, " Sample Results Letter' Attachment 4, "GFE Test item Distribution"
)
e NUREG-1021 5 of 13 Revision 8
ES-205 Sample Notification Letter Attachment 1 NRC Letterhead (Name. Title) (Date) (Facility name) (Street address) (City. State Zio code) Dear (Namel-The NRC plans to administer the generic fundamentals examination (GFE) section of the written operator licensing examination on (date), To register personnel to take the GFE, an authorized representative of your facility must subm a letter to the appropriate regional administrator with a copy addressed as follows: Chief, Operator Licensing Branch Mail Stop OWFN 9 D25 U.S. Nuclear Regulatory Commission Washington, DC. 20555 Your letter should identify the individuals who will take the examination, and it should ce they are enrolled in a facility licensee-sponsored program leading to NRC operator or senior operator licensing and that they will have completed their fundamentals training by the date of the examination. The letter should also identify the personnel who will have access to the examinations before they are administered (e.g., prwctsis) and the address to which the examinations are to be sent. To allow the NRC to assign docket numbers, the letter should be received by both the NRC regional administrator and the Chief, Operator L; censing Branch, 3 days before the examination date. A sample registration letter is enclosed. Sincerely, (Aserssr;&te reaional representative) Dt.cket No. 50-(Number)
Enclosure:
As stated NUREG-1021 6 of 13 ' Revision 8
E i i l ES-205 2 Attachment 1 l (Name) Enclosure Regional Administrator U.S. Nuclear Regulatory. Commission - Region (Number)
~ (Street address) i (City. State Zio code)
Dear (blEDal. (Facility name) requests to have the following (number) individuals take the (BWR or PWR) . generic fundamentals examination (GFE) section of the written operator licensing examination J to be administered onIdatal '
' Nag}a Date of Birth Previous Docket No.
(Insert each the name. date of birth and orevious 10 CFR Part 55 Docket Number (if annliemhla) fo norson.) , All of the listed personnel are enrolled in the (facilitv name) operator licensing training program and will have completed the generic fundamentals portion of the program by the examination date. i
. The following personnel will have access to the examinations before they are administered; Heala Bla (Insert the name and title of amch oerson who will have access to the examinations before theI are Edininistered (e o.. orc -d=s).
Please address the examinations as follows: Name. Title Street address Citv. State 7in encia If you have~any questions, please contact (facility contact name) at (teleohone number). Sincerely, Name. title . cc' Chief, Operator Licensing Branch l NUREG-1021 7 of13 Revision 8
- ES-205 Sample Examination Cover Letter Attachment 2
{ (Date) (Name. Title of desianated addressee) (Facility name) (Street address) ) (Citv. State 7in carie) Dear (Namek Your facility is scheduled to administer the generic fundamentals examination (GFE) section of . the NRC's written operator licensing examination on Idata}. (Name of contractor) is authorized under contiact to support the NRC in the administration of GFE activities. Note: For security reasons, please open the sealed envelope now and page-check the examination using the enclosed checklist. . Then contact immediately and no later than idata) one of the persons listed below informing (him or_ hat) that you have received this package and noting any discrepancies: (Name),(Telenhone Number) (Name),(Telenhone Numbert This letter and its enclosures provide the instructions and guidelines for administering the GFE and retuming the completed exams and related materials to INaTL0icontractor) Please read this letter and follow the directions in the accompanying enclosures ricw. Enclosure 1. Security As. ;.T.e..t. Please refer to the enclosed NRC Security Agreement. A copy of this agreement must be completed by each and every exam administrator and/or proctor seeing or having krW of the GFE contents. For security reasons, the number of persons seeing or having krc;if-;+ of this exam's contents before the exam must be limited to three persons who have a need to know. The top portion of the security agreement is expected to be completed now, and the bottom portion immediately after the exam has been completed. Fill in the spaces for each individual's name and the name of the facility for both portions, and have the individual (s) sign the form (s). Please note: The signed security agreements must be retumed to (Name of contractor) along with the completed exam answer sheets before any scoring will be performed. j i i i 1 NUREG-1021 8 of 13 Revision 8 1
l~ I l ES-205 - 2 , Attachment 2 Enclosure 2. E-n Cooles. Two single copies of Forms A and B of the exam are provided.
' These altamative forms are identical in content; however, for security purposes, the test item sequence on each form is different to reduce the possibility of an applicant copying any
. answers from a nearby test answer sheet. (See the separate Proctor instructions in Enclosure f 3 for further exam administration instructions.) .
You are responsible for reproducing the number of exam copies required for the number of individuals taking the exam. Prior to the exam, store the original copies in a locked cabinet or i safe and reproduce the necessary number of copies only on the d::y immediately preceding exam; in this case, copies should be made on Idata). Please note: your total number of copie should consist of one half Form A and one half Form B. After making the necessary number of copies l secure the original and all copies from view of unauthorized persons, storing them in a t locked cabinet or safe until the exam date. Each individual taking the exam must sign the security statement on the exam cover page. This page must be removed from the exam copy and mailed to (Name of contractor) along the answer sheets and administrator / proctor security agreements. After the exam has been given, the exam copies become public knowledge and no longer need security.' Exam copies, therefore, may be kept or disposed of as desired. Enclosure 3. Proctor instructions. The proctorinstructions detail the guidelines for administering the exam. Please note that the specific instructions presented are designed to be adhered to and followed identically by each proctor at all facilities. This process will ensure uniform administration and equity of results nationwide. As noted in the Proctor Instructions, al *
- GFEthe which exams will facility is be administered located.- at the same time in accordance with the local tim Enclosure 4. Exam Answer Sheets. The appropriate number of answer sheets (extra copies included) is enclosed for the number of applicants you identified to take the exam. All applicants must use the original enclosed answer sheets for recording answers during the exam.
Summary of items to be Returned to (Name of conti cter)
' The following items are to be mailed via Overnight Dei /very Service to (Name of contractor j
- and postmarked no later than Idatal.
5 completed answer sheets
. applicant-signed exam cover sheets administrator / proctor-signed security statement (s)
NUREG-1021 g of 13 Revision 8 l L
y ES-205 3 Attachment 2 Mail all of the above exam-related materials addressed as follows: (Name) (Name of contractor) - (Street address) (City. State Zio code)
' For furtner questions regarding the specifics of this exam, please contact (Name) at fielenhone number). For_
(teleohone questions regarding the GFE in general, please contact (Name). NRC, at number), For matters regarding candidate withdrawals or cancellations l contact either (Namei or (Name at (telephone number) for specific guioance. (Name). Chief - - Operator Licensing Branch Division of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation
Enclosures:
As stated- ' Distribution: w/o enclosures Director, DRCH Chief, OLB ' OLB GFE Coordinator Project Manager Public OLB R/F l i NUREG-1021 10 of 13 - Revision 8
ES 205 Sample Results Letter Attachment 3 NRC Letterhead (Date) (Name. Title) (Facility name) (Street address) (City. State Zio coM)
Dear (Name):
(*) On (date), the NRC administered the generic fundamentals examination (GFE) section of the written operator licensing examination to employees of your facility. Enclosed with this letter are copies of both forms of the examination, including answer keys, the grading results for your facility, and copies of the individual answer sheets for each of your employees. Please forward the results to the individuals along with the copies of their respective answer sheets. A "P"in the column labeled RESULTS indicates that the individual achieved a passing grade of 80 percent or better on the GFE. Those individuals having an "F"in the RESULTS column failed the examination. (**) On (date), the NRC administered the generic fundamentals examination (GFE) section of the written operatorlicensing examination. (**) Your facility did not participate in this examination. However, a copy of the master (BWR or PWR) examination, with the answer key, is enclosed for your information. if you have any questions conceming this examination, please contact (Name of the OLB GFE coordinator) at (ohone numbert Sincerely, (Anorooriate reoional representative) Docket No. 50-(Number) (*)
Enclosures:
1. Examination Form "A" and "B" with answers
- 2. Examination Results Summary for Facility Name
- 3. Individual Answer Sheets
(**)
Enclosure:
As stated I [ Paragraphs marked (*) apply only to those facility licensees that participated in the examination, while paragraphs marked (**) apply only to those facility licensees that did not participate in the examination.) NUREG-1021 11 of 13 Revision 8
)
ES-205 GFE Test item Distnbution Attachment 4 i K/A Pressurized Water Reactors No. of Topic items Grouo I Comoonents 191001 Valves 191002 4 ) Sensors and Detectors 191003 10 Controllers and Positioners 191004 Pumps 5 191006 Heat Exchangers and Condensers 7 191008 Breakers, Relays, and Disconnects 3 7 Groun ll Comoonents ' 191005 Motors and Generators 191007 5 Domineralizers and lon Exchangers 3 Groun i Reactor Theory 192004 Reactivity Coefficients ~ 192005 Control Rods 4 192008_ Reactor Operational Physics 4 i 8 i Groun il Randor Theorv 192003 192006 Reactor Kinetics and Neutron Sources 2 Fission Product Poisons 6 Groun ill Reactor Theorv 192001 Neutrons l 192002 Neutron Life Cycle 1 ' 192007 Fuel Depletion and Burnable Poisons 2 ; 1 i Grout i Thermedw.emic.s 193009 Core Thermal Limits 193010 2 Brittle Fracture and Vessel Thermal Stress 5
)
Groun Il Thermodynamses l 193003 Steam i 193007 Heat Transfer 2 i 193008 Thermal Hydraulics 2 ' 8
~
Groun 111 Thermodw.emic.s 193001 Thermodynamic Units and Properties 193004 Thermodynamic Processes 1 193005 Thermodynamic Cycles 2 193006 Fluid Statics and Dynamics 1 5 ! Totalitems 100 NUREG-1021 12 of 13 Revision 8 4
ES-205 2 I Attachment 4 Boiling Water Reactors K/A Topic No. of items Comoorl1011 l 291001 Valves ' L 291002 5 Sensors and Detectors 291003 9 Controllers and Positioners 291004 Pumps 3 291005 8 Motors and Generators ! 291006 Heat Exchangers and Condensers 5 291007 G Domineralizers and lon Exchangers l 291008 Breakers, Relays, and Disconnects , 3 5 Groun i Reactor Theorv 292004 Reactivity Coefficients 292005 Control Rods 2 292008 Reactor Operational Physies 4 l 8 Groun Il Reactor Theorv 292001 Neutrons 292002 Neutron Life Cycle - 2 l 292003 2 Reactor Kinetics and Neutron Sources 3 ! 292006 Fission Product Poisons 292007 Fuel Depletion and Bumable Poisons 6 1 Groun i Thermodynamics 293007 Heat Transfer and Heat Exchangers 293009 3 Core Thermal Limits 7 Groun ll Thermodynamics 293003 Steam - -
- 293004z Thermodynamic Processes 2 i
293008 Thermal Hydraulics 2 293010 7 Brittle Fracture and Vessel Thermal Stress 2 Groun !ll Thermodynamk;g 293001- Thermocynamic Units and Properties ! 293005~ Thermodynamic Cycles - 1 i 293006 Fluid Statics 1 3 Totalitems 100 NUREG-1021 ' 13 of 13 Revision 8
ES-301 PREPARING INITIAL OPERATING TESTS A. PURPOSE All applicants for reactor operator (RO) and senior reactor operator (SRO) licenses at power
' reactor facilities are required to take an operating test, unless it has been waived in accordance with 10 CFR 55.47 (refer to ES-204). The specific content of the operating test depiinds on the type ofIhense for.which the applicant has applied.
g gn?%g % f This standard describes'the procedure for developing opera #kLtests fdst meet the -
% 1
-conduct requirements of 10 CFR 55.45, including the use of react 6$) ant simuld5ffasitiOAs.and '!
of multi-unit evaluations. $f Af W iP % # B. . BACKGROUND E. dF NW % 4 1 kgd To the extent applicaHe, the operating test will require the applicant to demonstrate an understanding of, and the ability to perform, the actions)iecesAM 36Mmplish a representative sampling from the 13 items identified in,j0 CFR 55$5(ajlill:13 items do not i need to be sampled.;orgevery operating test). In;mdSitiorQJhe conte $tj6fKoperating test will I be identified, in part, from leaming objectives.c$ntained;in the facility Ildensee's training program and from information in the final sakana NpMteysyMdescription manuals and operating procedures, the facility license and lic@ense ame and other materials requested M from 47the%gIcility licefdee by
- Id$51 mission.
; The structure of the operating test 4
- YtatedAn part, byjl0'CFR 55.45(b)(1).- It states that the
~ test will be administered in a plarilI$alk-throQ and in either a sM ulation facility (as defined in
- 10 CFR 55.pthat the Commiss% has apsfiiwod;crI5imulation facility consisting solely of a plant-referenced simulatori(siso!as definedin110CFR 55.4) that has been certified to the Commisdsn'6/ihe facil$lii$$Ee. #
The walk o t consists of two categories, each focusing on
. specific knowle j,est
- assigned dutid lities (K/Ajis tequired for licensed operators to safely discharge their ImeMthird category of the operating test is administered on an NRC-a%6ved or "" , d simulation facility. Unless specifically waived in accordshce with ES-20(. , mented on the Examination Assignment Sheet (Form ES-201 chment 4), all ..
k , tegories must be ceiwpieted for every license applicant.
%_ .. category of the operating test is briefly described below. S gr_adin,g,4_he opera,tirig test are contained in ES-302 and ES-303, respectively.
^
"Ad.TJrJeimiive Tosics"
~
This category of the operating test covers K/As that are generally associated with the
, , administrative control of the plant.' It implements items 9 through 12 of 10 CFR 55.45(a)
- and is divided into four administrative topicsi as described below. The depth of NUREG-1021 1 of 27 Revision 8 A
I ES-301 coverage required in each topic is based on the applicant's license level. The j applicant's competence in each topic is evaluated by administering job performance I measures (JPMs) or by asking specific questions. Topic A.1, " Conduct of Operations," evaluates the applicant's knowledge of_the daily operation of the facility. The following subjects are examples of the types ofinformation that should be evaluated under this topic: #
,, c p#4%f shift tumover g' < p*. r ~
dJE 3 c .F shift staffing requirements 4M h [.
%, ];sh.m !)%,
temporary modifications of procedures 4 ' reactor plant startup requirements Q@.ff Au htL 'a -
- mode changes (@h;[AP' %n position (ECP) plant parameter verification (estimated critical {
short-term information (e.g. . night and standigMders) key control security (awareness and familiarity) g 9{[%ght fuel handling Np k Topic A.2, " Equipment Control," addrdssEs~idsdmijdstrativejequirements associated with managing and controlling plapystemsjiiMiiequiprism6FThe following subjects exemplify the types of informatiorrthat should be evaluateh under this topic: { gy {b jf~ surveillance testingp*~ qf ff l maintenance f,y
- As /p tagging and cleasences [hymf7
~
yb[*;jlemporary/nodl6 cation of systemsg tamiliarity'withimduse of pipliiglia'nd instrument drawings yh'd+ %, N ; 9:;g TopidW respect'N,ddletion g .- to
;mg .
hazards.and.l protection (of pla
; , , g use and of portable radiation and contamination survey instruments and personneboYiiitoring equipment knowiedh of significant radiation hazards the atM to perform procedures to reduce excessive levels of radiation and to gua#against personnel exposure hym' p., on exposure limits and contamination control, including permissible levels 47 n excess of those authorized radiation work permits control of radiation releases Topic A.4 " Emergency Plan," evaluates the applicant's knowledge of the emergency NUREG1021 2 of 27 Revision 8 !
I
i l l ES-301 plan for the facility, including, as appropriate, the responsibility of the RO or SRO to decide whether the plan should be executed and the duties assigned under the p The following subjects are examples of the types of information that should be eva under this topic: I _ lines of authority during an emergency l - emergency action levels and classifications [^ emergency facilities f emergency communications i jf '"f
~ jgp xl ' ' % !
emergency protective action recommendations , y fy,j, v"<' %,Q ,9' gg /v Category A is administered in a one-en-one, walk through format and the foUr topi[7 areas are graded collectively (refer to ES-302 arni ES-303). ( j'
' 2. y :: h Cateoorv B " Control Room Systems and Facility Walk-0 'udah" 8 NE5NM4 This category of the operating test is used to determine if thiPapplicant's knowledge the area of plant system design is adequate'shifto determinillt.ttisipplicant is able to safely operate those systems. As such$Uili]fAtAgory implerrydithe requirements of items 3,4, 7, 8, and 9 identified in 1_0lCFR 55;45 of systems, including primary cool $li, emerg5ddy($211also encompasses s
;c$olaintydicay heat removal, auxiliary, radiation monitoring, and instrumentation'iihdlcosirol.
Category B is divided $ ubcatintog)s$ bV
' " Control Room Systems"ffocuses on,egories. The:Srst and larger subcategory (
those systems with which licensed operators are seco$d subcategorf(si[* Facility Walk-ThroUgh") ensure wlOEUMiesign afiid; $tiliibilities relative sh5ceitt'slknowledge
^
to each system valuated by admi Ms arid % [%[' follow-up questions. Su B.2 are. hdministered in a one-on-one, walk-through format and
. areffsided er t'o ES-302 and ES-303). I 3.. *Cateoorv C. "Inte ' tPlant Ossreisons" This category o(gg aMe operating test implements item i
, identified in 10[CFR 55.45(a). This is the most performance-based category of the
"' rating Aliifand is used to evaluate the applicant's ability to safely operate the
~6h6er dynamic, integrated conditions.
e' simulator test is administered in a team format with up to three applicants (or surrogates) filling the RO and SRO license positions (as appropriate) on an operating crew.- (Refer to ES-201 for additional guidance on crew composition and ES-302 for test administration instructions.) This format enables the examiner to evaluate each NUREG-1021. 3 of 27 Revision 8
+
ES-301 applicant's ability to function within the control room team as appropriate to the assigned . position, in such a way that the facility licensee's procedures are adhered to and that the limitations in its license and amendments are not violated (refer to 10 C Each team or crew of applicants is administered a set of scenarios designed.so that the examiners can individually evaluate each applicant on a range ofpompeterhies applicable to the applicant's license level. Appendix D describes.fhose <fofnpetencies, and Forms ES-303-3 and ES-303-4, the " Integrated Plant Operitions:d6mpetency i Grading Worksheets" for ROs and SROs, break dodisach c$mpetehdyJnto a numbee , of specific rating factors toff be considered g '% !$Rjg during js tidiradin Each applicant must demonstrate proficie,cy on every corppetency applicable
. her license level. The only exception is that SRO, Competency trol Number 5Mx.a
~
Board Operations,"is optional for SRO-upgrade appiscahts._
- 0. %
RESPONSIBillToS 48hh 81 4 m
- 1. Engility Licensee ~ @w%ng
.A "*
^:;f%
. The facility licensee is responsible for 3%
foil activities,,a -s applicable, depending upon the examination arrangementFconfirme*ci MNRC7egional office in accordance with ES-201 approxiiNately fousinonttEdhsMNithe scheduled examination date: ~
;f a.
Prepare proposed examinationoutlines)n accordance with Section D and submit them to the NRC' ional ofais and approvalin accordance with ES-01 CAA QNA
<fp ME6
. ag{
lSU tun;t the reihpaterials necessary for the NRC regional office to prepare g.g. ' . . . 'ew thei i
-s . examination (s)(refer to ES-201, Attachment 2). {
.bnaw c.
? review- . . ifinal operating tests in accordance with the pre iv ously
~
apprdddiiiEnination outline (s) and the instructions in Sections D and E, and I cubmit td " *the NRC regional office in accordance with ES-201. W v
- d. Make th
't g prepareMthe mutation operatingfa cility available, as necessary, ior NRC examiners to tests.
3@pgt the NRC in the regional office or at the facility, when and as { refer to ES-201).necessary, to review the proposed operating tests and f.
. Revise the operating test outlines and the final tests as applicable and as agreed upon by the NRC regional office'(refer to ES-201). The NRC retains final authority to approve the operating tests.
NUREG-1021 4 of 27 Revision 8 ! l
i 1 ES-301
- 2. NRC Realonal Office R ;
L The NRC regional office is responsible for the following activities: I
.p
- a. l Ensure that the $perating tests are developed in accordance with Section D.
p
- b. ff?"
Ensure that the operating tests are reviewed for, quality in_Qaccordance with Section E. A 40NF !
- c. th ki R-rereWew Meet with the facility licensee, when and operating tests in accordance with ES-20if A[. asMpr
. D. INSTRUCTIONS j& f9g #,pV %?)W NQ4 #
%QfMA[g
.n Prepare each category of the operating test in accordance withWfollowing general guidelines and specific instructions:
gjf 1 General Guidelines A.Qlh NE
- V7.
amm AA
- a. In an effort to reduce exami n preparationIeffort, th.e same operating test :
may be used to examine miiiiiple apMidsliE@$idiirfator crews. Depending on ; the number and license possible to use theasam [e' set of JPMs and scenarios to e ; applicants if the opersiw,gn testis: administer =ed in multiple segments (e.g., single
-a scenarios single daynThelicility
> or licana two-jdJPMs)p$e c nn r andM NRC chief exami RPtions and agreement'onihswe; process before developing the operating tests.
Jds3h QQ_fg T&W
%ppexq, kkkNbizep 4and maintain test integrity, varied subjects, systems, i
N5pei.ntio, w ~ ns siMGMiiluated ww with applicants that are not being examined
- samsiime applEFjihls.fD,- -unlessmeasures cf du;'!.r.br h;;;n are taken to
, ;;-:-:::::;; preclude
;p;;d...; 0;.3 interaction h;:: M among th 72.J.r; 3[JSA"dir;ne"JPMsiand: simulator scenarios shall not be JI t :$.$;f.??.3 successik~ d icdid.f.;;.
i 'r;.7.th.;;th; (;re terJ, lr. .T thei r-e ;'sps.ed; M rep;;;;d
. ., ra; JF;t OF,0;)
d;j 0; dei (l
.T j s'.M. 7:;;;td cr. : :::;;:v; d;y;).
~
^. ng tests written by the facility licensse may not repeat test items stor scenarios or JPMs) from the applicants' audit test (orn tests)f tre appliMis; retaking:the examination)lgiven at or near the end of the license
~ training c: ass, Simulator events that are sinn.ar to events that were tested or: the
- audit examination are permitted provided the actions required to mitigate the transient are significantly different from those required during the audit examination. NUREG-1021 5 of 27 Rv ision 8 4
}=
ES-301 Sufficient operating test materials shall be developed to ensure that all applicants can be telted with the available personnel according to the schedule agreed upon by the NRC regional office and the facility licensee (refer to ES-201). b. To the extent permitted for each category of the operating test, select and modify testing materials (i.e., JPMs, questions, and simulator scenarios) fr6siithe facility's examination banks. Every selected test item must satisfy the qualitative and quantitative criteria specified for the applicable sectidrE6f ths operating test t or be modified accordingly. ~@ % ;gdh
- c. Consider the K/As associated with normal bnormakand emerDency; ANNW l%. 4 tasks evolutions as a source of to,-ics for use in;e, valuaty applicant competency in each category of the operating test. $g g g%Mih q., -
. The knowledge and abilities associated withibe: tasks and questions planned for the operating test should have importancifaE6iiiE6fNieast 2.5. Tasks with importance factors ofless than 2.5 ma/ tie used ifihef$is.a substantive reason for including them (e.g., a recer,t* licensee event or'5Moant system modification). jg Ng/
The K/As should be appro
+
y e to thei Tequirements for the applicant's license level.ft$fer to thsiacil" j$tilnd task analysis (if available), leaming objectives, and biher referyhce materfafio confirm that the operating test is correctly orienddito the fdsflity and tin applicant's license level. The facility licen e-s c tasitid may be used to supplement or Isisi6ted individualitems in the NRC's K/A override, on4[ca~ai-by-caseEt@Aliiation catalogs.4tiUNdigito maintkhze: , consistency the
.I66all k!MEk not NhMhbM5edMg%ce ofilEentire K/A catalog.
- d. When ing matorials (JPMs, scenarios, and questions) for N ; selecting kh"that test, ensure the materials contribute to the test's I
,[differentiaYbetween those applicants who are competent to safely operat, tend those who are not. Any test items that, when missed, would raisii 6ns regarding adequate justification for denying the applicant's license slid 0 fot be included on the operating test.
e. SRO apiijlicants, whether upgrade or instant, will be examined for the highest on-y 6 shift, position for which the SRO's license is applicable (e.g., shift supervisor), j less of the position to be assigned when licensed. SRO applicants should strate their supervisory abilities and an attitude of responsibility for safe
#Ep.eration, and are expected to assume a management role during plant transients and upset conditions while taking Category C of the operating test.
The operating test briefing, discussed in Appendix E, ensures that the applicants are advised of this policy. NUREG-1021 6 of 27 Revision 8
i ES-301 l Differences in administrative controls and facility design will affect the SRO's responsibilities, but, in general, the following guidelines should be used to differentiate the SRO operating test from that of an RO. In directing licensed activities, the SRO must evaluate plant performance and make operationaljudgments accordingly. SRO applicants should, therefore, be more knowledgeable in areas such aspperattiig characteristics, reactor behavior, and instrument int ' retation. Ab in directing licensed activities, the SRO~must Nh have:a! $w x er and more 4*, 1 thorough knowledge of facility adr$histrative cohtfotAAWMhods, j@
' including limitations imposed by 15iregulatiohi and thi*fasilit/sTieclinical 1
specifications and their bases, g %gjP" l The SRO may be assigned resporikitEdfor auxiliary systems that are outside the control room (e.g., was 5W
. are not normally operated by licen) sed op%f[and handling sy erators$Because the SRO may j
have these additional responsibilities, the SR,0 license applicant should l 1 maximtm permissible con [centrations,pefflu radiologicalconsideniitl6ns. y a 8> ' %) e
- f. {
Incorporate facility-specid6 and industry-ge _ operating experience into the operating test whenessIpossiblif Docume$tation such as licensee event reports, significant!iNEEi repor6[and serviiIe information letters are readily available sourcosisioperatio5 ally orie.deisplant anomalies.
#valuatepih nt acci s (DASs) for the facility to determine if
)%;mlkthare suitableMiesting, on a sampling b a - roukO$is@DASs are those sequences that contribute significantly to ncy oftere damage as determined by the facility licensee's "Mkrisk aisissa 0Snt (PRA) or individual plant examination (IPE).
The P NME Y Chapter
.. dould also be used to identify risk-important operator actions.
"tional Perspectives," of NUREG 1560, " Individual Plant Examina ram: Perspectives on Reactor Safety and Plant Perfonnance,"
identifie . number of important human actions that may be appropriate for evaluatidiIon the operating test. In determining what actions te evaluate, do no overtofactions that are relied upon or result in specific events being driven to ;
. . Iowink contribution. This will help identify those human actions, assumed to be )
"reliab'e, that might otherwise not show up in a list of risk-dominant actions.
If the applicants at a facility qualify for dual or multi-unit licenses, the operating tests should evaluate their knowledge of the design, procedural, and operational differences between the units. ,
. NUREG-1021 7 of 27 Revision 8
ES-301 Divide the operating test coverage among the units and do not become predictable by conducting the walk-through tests on only one unit. Different applicants may be examined on different units, or each applicant may be asked to explain or demonstrate his or her understanding of variations in control board layouts, systems, instnJmentation, and procedural actions between the units at the facility. g Most dual- or multi-unit stations have a simulator that is'rhodeplafter only one of the units. Therefore, ensure that the applicants are properlyjested on the different systems, control board layouts, and any cther differences between the units during the walk-through portion of thE$perating tdi{ FReistiiple s administering Category C of the operatingfest on Browns Ferry tinitsthe9 control room systems portion of Category!B of;the"6perating test 'could f(ej ' administered on Unit 2 or Unit 3 or both. %m*. ' ,
'"^
i N?%
- h. The operating tes't should examine a broad rakfiQnggho.wledge and abilities, systems and components, and operations and events.g three categories of the test should not be redundant, nor should they duplicate' material that is covered on the written examination 301Isparticularly inpdrt'5nt that Categories B and C be developed and revievii5dNEjahN$ge to preclude the same tasks and events from appearing on b$ifEparts,6f tfEIdst3hg#
bq g wsy m qb sy i. Every facet of the operatihg testp ifEluding thsluestions and answers, JPMs, and simulator scenarWshouldlii planned documented to the#ris$$imumd$ tent possiSe$esearched, valida before the test is ad 71inistemd. Examiners)n[6 e admi perating tests but were not involved in J.M iheir devel6pnEnt are expe5$d$research and study the f
%htiE^examinisibhiidoperatinglest so that they are prepared to ask whatever Nfdli6$i-up que'shdiNiiiht be necessary to determine if the applicant is Nom [dE(di)t in thoIAMis($ Examination team members are stsongly encouraged 9"[et6m$fi5Y%jhyQeen eOshe
$$ proved group %th for administration by thethe chief responsible examine supervisor.
The discussensishould focus on those test items that might require extensive cueing bM$A3NA" miner and those that are unique to the facility and require a ; response diffAIent from what the examiner might expect based on past { experiery. '
. k.
I, JPM6hould include the elements identified in Appendix C (e.g., initiating and thating cues, critical steps, and performance criteria). The guidelines and c<r Iorms (or equivalents) in that appendix should be used when developing new JPMs. Facility procedures may be adapted for use as JPMs by identifying critical steps and entering comments on how to execute particular steps. 1. The prescripted questions for Categories A and B may include a combination of open- and closed-reference items. Open-reference items that require applicants NUREG-1021 8 of 27 Revision 8 l _o
I ES-301
- to apply their knowledge of the plant to postulated normal, abnormal, and emergency situations are preferred. Closed-reference items may be used to evaluate the.immediate actions of emergency and other procedures, certain '
automatic actions, operating characteristics, interlocks, end-set points, and routine administrative activities', as appropriate to the facility l howeverl such questio.ns:should not exceed 30 percent of the total in eithe.r^a test CaMgory. Minimize the use of closed-reference questions that rely,s6tely ori%emory, and cdd W,e do not.use eRpen-reference questions whosefarb4Er can simply be I looked up'if the applicant can7find the4$correctIdScume6dg$d% 4;jM% f, Refer to Section B.6 of Appendix C for mo[ts guida development and use of open reference uestions fofthe walk-ttiroU l
)
m. MNp Ifit becomes necessary to deviate from a test; ou y n approved by the NRC chief examiner in accordance with*8M20Rdiscuss the proposed deviation with the chief examiner and obdih A$icliN$ s:e 3before proceeding with the changes. Be prepared to explain why the oridiskt proposal could not be implemented and why the proposed riidiscement isEnsidedd an acceptable substitute. /*h,
- 2. Soecific Instructions for Cateoorv AMAdmini$r"$$dT I "
Although the administrativejopp N ~\$W may im$ it is preferable, examined; separately, whenever possible, to linkjessociate,,orintegrate th'em with tasks and events conducted during Categories B and C7sNowever proficiency in the admini$55tive topEN;kis houldSideliberately imporf$ht evaluated and not to keep inferred in mind so m observations %ade duriktiksiidDiator portion of the operating tett.
-4 a
$k *W$hh 5 '#
Nopeach of ttis$bministrative topics listed below, select the required number of , I
'subjects to benils'hedduring the operating test. Section B.1 provides
;of the typeiliWidbjects that should be evaluated under each topic; the
.- t.e,remot'all-incluslM, Iggig Number of Subiects A.1, "Co of Operations" ~ 2 A.2, "Eq% ment Control" 1 A.3, "l6idiation Control" 1 A rgency Plan" 1 or each administrative subject, determine the best method for evaluating the
' applicant's knowledge or ability in that area. Although a performance-based evaluation, using a single administrative JPM is generally preferred, two prescripted questions may be used to conduct the evaluation in each specific subject area selected for evaluation. The questions may be associated with NUREG-1021 9 of 27 Revision 8 4
ES-301 Category B JPMs (as additional questions) or they may be administered separately,
- c. In general, SROs have more administrative responsibilities than ROs, so SRO applicants should be evaluated in greater depth on the administrative topics. RO applicants need only understand the mechanics and intent of the relat'ed subjects, as they pertain to tasks at the facility. 8.
As mm%&g Qui n
- d. The following specific guidelines should be, applied whlnlseldcting or developing _
questions or JPMs to confirm the applicantTeompetence Wi6regird to each g i topic: jY
.[. %isiEk QQQ %V Toole A.1. " Conduct of Ooerations" (1.N(
ys 9 g yiF Many of these subjects can be covered wi _ _ ework of a shift tumover or by integrating examination. them into* other discussi&Is,'$i[Nis@$MDbtINy 4 NkbN, hIi can G[ cover @edio the control
- The subject of fuel handling cover this subject in the fuel h&ElIihdiresiN1he plariithenever possible. The RO applicant should be awd5 of his4fNsNk5esiriftNe control room during fuel handling. These duties irEiude monEoringInsiSNE$tation and responding to alarms from the fueldling agcommupIing with the fuel handling and storage facility, and 9aratingfystems from.the control room in support of (re) fueling operati alterations, new an$. For t%gRO appnt, evaluate topics such as core d spent fuststorage and movement, the design of the fuel
,Ahandling arsh,Mof the fuM$iddliN6 tools, and fuel handling casualties.
f(,~!X
>c
'A. Q?MTA +m 92&##
- The applicarts: security awareness should be evaluated by observing his or her N tielisivior duriniWdjieNati_ng test. However, passive observations, in and of
'themseNhs, are ir$UffidEidto justify al evaluation in that subject area. It is l@MM9td securi he operating crew's interactions with the secu supervisor; .'
y Tooic A 2}^~"Eauioment Control" These'seb
. evo40N6a. jects can be evaluated within the framework of a normal maintenanc For example, ask the applicant to demonstrate how he or she would a failed system or component out of service, initiate maintenance on the P, system, and test the system before placing it back in service.
l .. Tonic A.3. " Radiation Control" This topic is best covered in conjunction with the JPMs and questions prepared NUREG-1021 10 of 27 - reevision 8
E ES-301 for Category B.2 of the walk-through (i.e., local systems and operations). It is most appropriate to erWuate these subjects during an entry into the radio-logically conmSen area (RCA). The levels of knowledge expected of RO and SRO applicants in some radiation control subjects are significantly different. The RO's duties generalisequire knowledge of radiation worker responsibilities ind operation of pisht systems associated with liquid and gaseous waste releases. Therefdre, the depth to which RO applicants are evaluated should be limited 16 thAirie'sponsibilities and , the monitoring requirements before, duringEa6d after the release 4The SRO, however, may be involved in reviewing and"approvingfeleass permits?and , should be cognizant of the requirements l associated with those releases [as well as their potential effect on the health and safetycor the public. The SRO1 applicants may be asked to discuss or sirndiati(i.e., with a JPM) a pIanned release (e.g., liquid, gaseous, or containmekpdrse) when examining these topics. gi Mg.]pg. jg%Ih N~- Tooic A 4 "Emeroenev Plan" kgjP There are significant differences 4'between Y
. .h $$5(d.[:the~ knowledge req SRO applicants in this areaf RO ap emergency plan and with'their plar$plib5Ntish5did specifid)esponsibilities under the sie familiar w emergency plan implemnnting procedures (EPIPs). SRO applicants, however, must demonstrate.additidnal kn$ ledge baised upon their responsibility to direct and manage the irnplimentati6dof the ERIPs during the initial phases of an
~
, emergency. ,Because of this'l SRO apflidants should have a more detailed f/g(jrocedures/psceschve action recommendations, and com
( , requirements andsnethods. W' '& K;Sh Thisibhicis best evaluated'by integrating it into a discussion of a Category C 5liist requirdiniplementation of the emergency plan, or by conducting a JPM'M[Ase of the emergency plan. Such a JPM can be conducted immediat
, A or B)ananstion.
ex%;foloiIsing a simulator scenario or during the walk-throu e. k The planned administrative subjects should normally take no more than 1 hour and 14 hours to administer to RO and SRO applicants, respectively. N ES-301-1, " Administrative Topics Outline," briefly describe the
$ hministrative subjects selected for evaluation and the method (s) by which each subject will be evaluated. The method of evaluation should include the title of any planned JPMs and a brief summary of the proposed questions.
g. Forward the completed outline to the NRC chief examiner so that it is received NUREG-1021 11 of 27 Revision 8 y
.y ES-301 g by the date agreed upon with the NRC regional office at the time the examination arrangements were confirmed; the outline is normally due at least 60 days before the scheduled examination date. Refer to ES-201 for additional l instructions regarding the review and submittal of the examination outline. I The NRC chief examiner and responsible supervisor shall review thkest outline coverage as soon as possible in accordance with ES-20,1 and forward any comments to the originator for resolution. A L Mbb M',' . e h. After the NRC chief examiner approves the~ operating t,est outline, prepare the g in j ? final Category accordance A testoperating with the general materials (i.e., the tesfguidelirdidn JPMs, questions, and Sectiohh1/ih - reference question guidelines in Appendin B?andithe JPM guidelines ihW i.- Appendix C. i mym When the materials are complete, reviewithe qf511i[% f ofjhe final Category A walk-
; Assurance through testreview Checklist." This using shallForm ES-301-3, be perfoiidid "Op5 fating Tdst'QUnlitp$lh in conju'ri:liorE Category B walk-through and the"dynsisnic^ simulator o'perating test as noted in
' Sections D.3 and D.4. # P !
Submit the entire operatif10 test package ton disignated facility reviewer or the NRC chief examiner $as appreferiste, for!seYiew and approvalin accordance with Sechon E. TheMmust difreceivedh the NRC chief examiner at least 30 l days before the schieduled a$ilinistration#Este, unless other arrangements have been made. - i4 3~
* % k?hh[ k ~ .6 Room SvsMms and Facility Walk-Throuch" 1 b=t # % -f 7 = vB.
3. NMDMk This'5aliieryef,the o Nd N evaluates the applicant's systems-related K/As by having theM pe tasks and probing his or her knowledge of the l
. task anii$amiiEmim$inart systill)Aivith specific, prescripted, follow up questions. The CM B " , '
s are in addition to and should be ditbrent from the l j Avents and e ucted during Category C, " Integrated Plant Operations." 4[a. Refer to I .
]
on 1.g of the K/A catalog applicable to the type of reactor for which ;
' )^l the applisiint is seeking a license (i.e., NUREG-1122 for PWRs and NUREG-1123)erBWRs). From the nine safety function groupings identified in the jj ^
ce*Wei, select the appropriate number of systems (see the table below) to be i
, m #{j@gfdilduated for each emergency and abnormal subcategory'of plant evolutions the test (E/ APES) listed in Section 1.10 ofbased on the appropriate NUREG may also be used to evaluate the applicable safety funcbon (as specified for each E/ APE in the first tier of the written examination outhnes attached to ES-401). I I
NUREG-1021 12 of 27 Revision 8
i I !- j k' ES 301 License Level Subcateoorv B 1 Subcateoorv B.2 1g131 RO 7 3 10 i SRO-instant (l) 7 3 10 i SRO-upgrade (U) 2 or 3 3 or 2 5
- <1
. The 10 systems and evolutions selected for RO and SRO-l applicants should l i
evaluate at least 7 different safety functions. All of the; systems ~and evolutions in each subcategory of the test should be selecied from differentjafety function lists, and the same system or evolution shduld not be used to evaluate more 44 w ) than one safety function in each subcateh. m jy g7 %{e w ~ ,gf {' The 5 systems and evolutions selected fdEdi SRO-U applicant slGuld saluate at least 5 different safety functions. One*ofltliA hbntrol room systemE5r j evolutions (Subcategory B.1) must be an engmeered. safety feature, and the same system or evolution should not be ~used to; evaluate more than one safety function.
- g^
$$ggsn W%.
\&A Keep in mind that the systems an,k e$volutioits. selected for evaluation in Subcategories B.1 and B.2 ndNN citientid 55 ward cdNirol room operations and local operations, respectivejy'. FM f
- b. . For each system select or evaldation, select from the applicable K/A catalog or the facility licensnifssite-sgehific task list *one task for which a JPM exists or can be developed.%kview theissociatesl" simulator outline if it has already been prepared (rejer tdSection D[
tselected forevald,ation on tlN[Ispd sif$ki those tasks that have alread simulator test. N"Nha . kkhk in enter to prote$itseintegrity and security of the examination process, no more 70preerif(i JPIAs[as applicats$)yisyMe taken directly from the fac iidlinout siinilicant maillfication. A significant modification means that at least one 55Ndilledises been substantively changed in a manner that alters the course of actiodiif$$$M. Additionally, no more than 30 percent of the walk-through test may liifyated from the last NRC licensing examination at the facility. a= k
- At leastene of the tasks shall be related to a shutdown or low-power condition,
' and onEbr two of the tasks shall require the applicant to execute attemate paths withlihhe facility's operating procedures, facility,licenseeslmay; increase, the
.of loppoweriand altamate pathMPhis,'and .NRC . examiners .may; change
" . eledad@lds;to;requus;the;use:of;altamate:pathsfnecessaryto; improve their diu:lnminatory. validity 4]in addition, at least one of the tasks conducted in the plant (i.e., Subcategory B.2) shall evaluate the applicant's ability to implement actions required during an emergency or abnormal condition, and another should require the applicant to enter the RCA. This provides an excellent opportunity for
- NUREG-1021 13 of 27 Revision 8
-).
F' ES-301 the applicant to discuss or demonstrate the radiation control subjects described in Administrative Topic A.3. Ifit is not possible to develop or locate a suitable task /JPM for each of the
- selected systems, retum to Step (a), above, and select a different system or evolution.- After identifying a JPM for each system, list it and its associated safety function number on Form ES-301-2, " Individual Walk-Through Test Outline." Also indicate the type of JPM by entering the Shplidable code (s) identified at the bottom of the form. gj!p c.
by *e]yg%for
^asW tojhe:ap For each system and evolution selected for evaluation,;
section in the K/A catalog and select two sistem-spesific or generid K/Asi$ tie evaluated with prescripted questions. TQplresgripted questions squid ME diversified among the different K/A categones' associated with each system or evolution. List the selected K/A numbers description of the topic on Form ES-301[, Nisi&importan
%!4gg$A TCM
- d. Forward the completed walk-throughiisi. outline to the NRC. chief examiner so that it is received by the date agree (uk$with the NRC' ygional office at the time the 60 due at least examination days before arrang$e scheddlifesiwsi5eliMweview date ES-201 for additional #insthictions rodardindihiMAw and submittal of .
examination outfire // c Af The NRC chief examiner andsosponsib supervisor shall review the test outline ES-201 $$illorwanifEny comments to the originator for d in accordance,witN#$ 43 Y '"% M
,,$hMihe e NRC chiefixaminer approves the operating test outline, prepare th
~
%finslTCategory B iisihistorials (i.e., the JPMs, questions, and answers) in Eccishlairice with NE{ge,heril guidance in Section D.1 and the JPM guidelines in V
.# When theiiM are complete, review the completed walk-through test for j quality usN{$Nn ES-301-3, " Operating Test Quality Assurance Checklist," and make anyphanges that might be necessary. To minimize duplication, this review shall beperformed in conjunction with the associated administrative topics and the sirr$nator operating test (refer to Sections D.2 and D.4).
K i Aff
, hit the entire operating test package to the designated facility reviewer or NRC chief examiner, as appropriate, for review and approval in accordance with Section E. The test must be received by the NRC chief examiner at least 30 days before the scheduled admimetreteen-faview date, unless other arrangements have been made.
NUREG-1021 14 of 27 Revision 8
ES-301 4. Soecific Instructions for Cateoorv C. "Inteorated Plant Ooerations" a. Based on the anticipated crew compositions, determine the number of scenarios and scenario sets necessary to rotate each RO and SRO-l applicant into the lead reactor operator position so that he or she can perform a direct reactivity manipulation. For example, a crew consisting of two ROs and one SRO-l will normally require three scenarios to evaluate each applicant's performance on the reactor controls; however, a surrogate SRO will have to'filidsupervisory role while the SRO-l applicant is in the lead operstor position /Similarly, the crews and scenarios will have to be planned so t$sfevery SRO applicant (U and 1) fillsY_ the supervisory role for at least one scenario, , jfjp % l? . Q gF SRO-U applicants are given credit for their previous RO license %i" evalu l experience and are normally not requiredjo psnipulate the controls, 3 tion and 11 may be possible to significantly dlWhreduce th'e num&% l ber of simulator scenario sets ; required to examine a large group of aMlicants Madmihistering the same set of scenarios on the same day to two (or more different) crews of applicants. However, provisions must be made (Sionsu)re that thaferews' remain contact until all crews have completed the. set of scenarios (refer to ES 302). - j[ jbhMMfyd Additional or replacement l scenarios should:also tNprepared and available while administering the operatlhg tests iruccordahdeEth ES-302 in case one of the planned scenarios dciiiiihot worn 5s intenddkl. b. bV #& tests in scen)an,Y The simulator, operating I l 4and modifying;schiharios frobi(ams,tiddficility licensee or NRC scen ' a#9and -Qg by desistoping* pew scehariosP"
%ggpQ '~
- n order to mahntain test integrity, every applicant shall be tested on at leas hi$rWinignificaht 1NitEsrae or pra@ct6sMisignificant modification means tha f[SohditioriN5 vent hailiMeen substantively changed to alter the course of action the sceharlo8%irthermore, any other scenarios that are extracted from the facility lidoniesYtiank must be altered to the degree necessary to prevent the
/ #f applicant conditiorps {jor other cues.fr651mmediately recognizing the scenarios c.
I The [ l conditions, normal operations, malfunctions, and major transients be varied among the scenarios and should include startup, low-power, x Mi$dfull-power situations. Review the associated walk-through outline if it has Intready been prepared (refer to Section D.3), and take care not to duplicate operations that will be tested during the walk-through portion of the operating test. d. In order to maximize the qualiy and consistency of the operating tests, develop NUREG-1021 15 of 27 Revision 8 L
3.: ES-301 new scenarios in accordance with the instructions in Appendix D. Modify existing scenarios, as necessary, to make them conform with the qualitative and quaniitative attributes described in that appendix and enumerated on Fcrm ES-301-4," Simulator Scenario Quality Assurance Checklist." The quantitative attribute target ranges that are specified on the form are not absolute limitations; some scenarios may be an excellent evaluation tool but may not fi.t;within the ranges. A scenario that does not fit into these ranges shatI be evaluated to ensure that the level of difficulty is appropriateg Whenever{possible, the critical
~
tasks should be distributed so that each applicant is required to respond. aw jW n xeW@ m xn%nk j Each scenario set must, at a minimum, emereiserequire/each ; applicant-on to g respond to the types of evolutions, failurdi[and tranklints in tiie quantiSesi - identified for the applicant's license level'on Form ES-301-5, " Transient and Event Checklist." An applicant should oriip hS*016en credit for those"esents that require the applicant to perform verifiable idMi ht provide insight to the applicant's competence. ;lnstmment and component failures.that are initiated , j after the. major. transient should be careidlly reviiiddBidause.they.sometimes s ight Teach EEshfsli6uld only be counted require little:. action'and provide little"in~ once per applicant; for example',Ipdderri:bange cariSeAUnted as a normal evolution OR as a reactivity m5hidUliii6n7ahd,Ssimilarly' immediately results in a madSitransiefc60r the idsother, 3donk o,% but not component both. fa Furthermore, each scen}anl? f o set giust kalso en&YNieellow the examiner to the applicant's performance on each competency and rating factor germane to the applicant's licehiilevel. .dA Form ES601-6, " Competencies Checklist," to
- Aand event nuimids that are3[nterididid assess each compej Nhs DE i
[A.Rif the'd$N$b facility.licinsee normallp: operates with?and M" 5533ficationsIIfiTsave more'than two ROs'in the' control room lthe chief exam '
"i6i]y uihodzeWpInss*6f~a55tionailsukogaies!togout the[ crews 3In su'ch cases;:iME.careWpliMing'iliescenariostoiensure.thatithe[additionalo' perators M,Cnoi~roMthelexahiinersiability to evaluate ~es& applicant _on;the; requil
.g7 purriberRedand?orijverf;competenWandJ'atirig] acto 6 l
g V W%W Appendim;fD provides detailed instructions for completing Form ES-D-1, the \
" j "Scenafoutline," and Form ES-D-2, the expected " Operator Actions," that O
$ examinars will use to administer the simulator operating tests. In order to l l
minsMie the amount of rework that might be required as a result of changes in ; f$Ianned scenario events, Form ES-D-2 should be completed after the NRC i g ief examiner has had the opportunity to review and comment on the proposed simulator operating test outlines (i.e., Form ES-D-1) in accordance with ES-201.
- e. When the proposed simulator operating test outlines are complete, forward them to the NRC chief examiner so they are received by the date agreed upon with
. NUREG-1021 16 of 27 Revision 8
ES-301
~
the NRC regional office at the time the examination arragements were confirmed; the outlines are normally due at least 60 days before the scheduled eweefr. f.er, review date. Refer to ES-201 for.additionalinstructions regarding the review and submittal of the examination outlines. S The NRC chief examiner shall review the operating test outlines in 45bordance i with ES-201, and forward any comments to the originatorfor resolution.
- f. x kw %%V After the NRC chief examiner approves thehrating tE$thdli$es, prepare th final simulator test materials by revising Form (s) ES-D-fas; requested by th NRC chief examiner and completing a detailed operator actibn fshii(ES-D-2 each event. All substantive operator actions o throttling valves; starting and stopping eiguiin(e.g.c p l entNaising and lowerin(15 vel,
. flow, and pressure; making decisions and gisirsdirections; not acknowledging alarms or verifying automatic actions) shallMd6cumented, and critical tasks shall be identified. Events that do not repire'a(operator to take one or mere substantive actions will not count toward the minirrastEnumber of events required for each operator per Form ES-
- g.'
Review the completed simulator operistiris taaffor quality using Form ES-301-4,
" Simulator Scenario QualidXssurandsIChicidi5tynid make any changes that might be necessary. Thiivewow stiall be performed in conjunction with the associated walk-throughtest (refer'to Sections D.2 and D 3) to minimize duplication.
! j3if f /
, Submit the entimloperatinghpackage'to the designated facility reviewer or the NRC chini examiner, as appropriate, for review and approval in accordance fGith Sectl5It51%
~-
e test mu'iit tEriceived by the NRC chief examiner at least 30 E arrsiijements hiOEbsiiimade. ~ ~ ' E. 3$ifi% QUAUTyA88URANCE REVIEWS N$$$[
- 1. ja i Mari=
w if the operating tMrt was prepared by the facility licensee, i&the: preliminary; outline thiiiproposed;tedshall be independently reviewed and approved by[si knowledgeable
.- supervisorfor!riiinagerf.: , :.;1 died fx2 4 ;;+;; ;rf. ..a p;;;; ^; CO 20') before iHe
!arelsuliinitted to the NRC regional office for review and approvalin'accordance
~ 'E@$6f. The reviewer should evaluate the enemination outline;and test!usin YlheMTn Forms ES-20142AES-301-3, and ES-301-4 and include the signed
. An$h different operating'iest) in the examination package submitted to the NRC in accordance with ES-201.
- 2. NRC Examiner Ravin NUREG-1021 17 of 27 Revision 8 K
i
t ES-301 a. The NRC chief examiner shal1 independently review each operating test for content, wording, operational validity, and level of difficulty. As a minimum, the chief examiner shall check the items listed on Forms ES-301-3 and ES-301-4, as applicable. The examiner should keep in mind that counting the number of scenario quantitative attributes is not always indicative of the scenario's level of difficulty. Although there are no definitive minimum or maximum attnbbte values that can be used to identify scenarios that will not discriminate Meause they are too easy or difficult, scenarios that fall outside the target]inri6es[specified on Form ES-301-4 should be carefully evaluated to ensure thspire appropriate. If the chief examiner wrote the operating test $nother NRC$nmi65r;shall perfomk_ the independent review.
$T a
gf M&% MP b. pQy Av % f f +47 The chief examiner should review the opera;:.l.ing tests as soon as p,s;3 w yo sible after 4 receipt so that supervisory approval can N#6btSined before the finalMiew with the facility licensee, which is normally scheSdlE(a6out two weeks before the 6tyief examiner promptly administration date. It is especially imppnt"tOtM[ofgext{! review tests prepared by a facility licensee becausg required if extensive changes are necessary. The chief examiner shall consolidate the comments from othkregional.reviewirEihd submit one set of comments to the author. - Jf If' written examinations"ar iso preh tesif first because;the Grition'exa.be' advantageous. to~ mmations can be more review the ' operating $1dsIuifimpaIiiing otOIaisminatior easN[ rescheduled requiredt i7 ~ 4Bif 2F A(g$
- c. f the facilit licensee devel hhmW ylincihty.hcEn(E57@'gistHlis0 operating
. resp 55Eible for technical test, n ;m.;r.;d %i accuracy and compliahce dillidhe restricdeins cgcoming the use of examination banks. However, the
%%imaminer Wax (pet:6ed to use his or her best judgment and take r measuressJncluding' selective review of reference materials and past tests, to T' The chietenswh)ine,r will note any changes that need to bei '
tests to il%%nsible supervisor (or a designated attemate other than the chief examinepfor review and comment in accordance with Section E.3 before reviewingihe examinations with the author or facility contact. There are no minirrysis'or maximum limits on the number or scope of changes the chief
^
^,p, ,dr may that direct the author or facility contact to make to the proposed tests, they are necessary to make the i.5is conform scceptance criteria. Refer to ES-201 for additional guidance regarding NRC response to facility-developed examinations that are significantly deficient.
e. Upon supervisory approval, and generally at least 14 days before the operating tests are scheduled to be given, the chief examiner will review the tests with the NUREG-1021 18 of 27 Revision 8 I
q l l l 1 1 ES-301 facility licensee in accordance with ES-201. Tests that were developed by the NRC shall be clean, properly formatted, and
" ready to-give" before they are reviewed with the facility licensee. The region shall not rely on the facility licensee to ensure that the tests are of acceptable
. quality to administer.
, g
- f. After reviewing the tests with the facility licensee, Ek the chief e, $ .
xaminer will ensure j that any comments and recommendations are re, solved snd"ths tests are revised ' as necessary. _ If the facility licensee devel6h0 the tests 3it Nil Usherall dE expected to make whatever changes arescommendeId by the NRCP y be )' /' g. 5 rs( _ f . W'
~_.,::ga&.[
ir After the necessary changes have been m,sda;anh9the chief exam'iner3 satisfied ; with the examination, he or she will sign Form (s)"ES-301-3 and forward the test package to the responsible supervisor for finst appsval. i l
- 3. NRC Supervisorv Review g# 4 ! ;^q
'% ' '" Q a,
jFik[$Ibs DO NRC author or chief examiner) stbiiEMn(eeds$i aper ES-201, th before the facility prereview?The % supIsIrYis6sy[%pprovaIthe not intended to be I another detailed review,g0t rathe including a review of the::changeg'paffbeneral:assosiiment o i check to ensure thaN6f the licable adininistrative requirements have been implemented. .F 47
~
A 9 37 l
. b. responsibhM(g pervisor bre that any significant deficiencies in the f5iriginal ofistairth$3ests subrititiidTy a facility licensee are evaluated in accordanceYllh'ES1201 to determine the appropriate course of action. At a NiihirMan, the 50perili3Eshould ensure that they are addressed in the final 4
'essEsination repoffMp65Edance with ES-501.
Meblsggg NigF '
- c. Foll tEdpity review, the responsible supervisor should again review the tests to ensureft6st the concems expressed by the facility licensee and the chief
,' 'examinerhlieen appropriately addressed. The supervisor shall not sign
& Form (s) ES-301-3 until he or she is satisfied that the examination is acceptable to be adsdnistered.
g ! CHMENTS/ FORMS a FannES403*1, " Administrative Topics outline" Forhists561-2, " Individual Walk-Through Test Outline" Form ES-301-3, " Operating Test Quality Assurance Checklist" Form ES-301-4, " Simulator Scenario Quality Assurance Checklist"
; Form ES-301-5, " Transient and Event Checklist" NUREG-1021 19 of 27 Revision 8
p ES-301 Form ES-301-6, " Competencies Checklist" m ll W y
, e <
p e d ,. t iM +
' 4.,Q.:-
j:.". n- .;[ p_.- 3"- p.m
. g.
g):- c \ :5-(*. E ,g'E -
,gp . ,eps b .:+ >- .
10 Nf':'., yg. ,.
. su.Mss w$: ,.: ,
.y.c-47
, : ,.4 <-
,, 0 h,' #
.:< ( _ n;.
4 .:.:<. A 3 s -:: 4;;3 n % -:#.
;; ^
j<'."(. .J,f-$-@:
> .w-y;
.gd:^. r@ Q:,,
Y g: .f~' a9 %
.N' N y;e e(
..dE.i s Ms>.
nji'iyh s Vi. ] ' .4 y+ dip.$ 4+g(.exx %v- .s p:;..e. : ,
>+.+.
c, vw;- g,.. A w +c.,.e gg. 9a .:. 7 ::4 -: , y, fvy
.:us-j.0" :: :x.o w .c, :.-i..;...a.s a ;-..ye.g ^ .p:.
g v: r - -.g... 3:.:
, p. . xp;:. '.
g. de
'yg xt p4.N ;:.i.g.:.y:;+;:q 5:i:(#r4v
. g ga Ij.c l' j3..s.f%.W1' m-s*%; st
. .s ,
4 $* Ax~
, 9 w v
- y f? ec g :A .
.s
- f. ;g ,p; e - , p+ ' MW
- s. g > a::,;.yx.
#4 3- :[
. ^>m
- i.h.v xt.$ ;x;?,.. ....m p..,- s.g;
- y. .g:p ,f.v ' ,:
.w .m.c p; . -
N*
.. I
'@4 A m. Ngpdh.m.
%p.s.m,. ag c%m. r m.:s--2g.%s o; .
y
.s 3
m_0..
.p4 A 3: .s 4 m.msv,.,
> - . <N:
$. ,, 9, wrm:>4 4E Tff;W, - :. ,p ,tl: .:.;7.f; 3 :.9..;.;4 ;y.3 i' 4 .. .
<p.w. -> ' e m -F/.et
- y '> a/jf .7 3 .)
. :.3:
#9p'
- 9 c ' ::)
w g @ag:.A.. . ; ,r;..ck .,
- . 8 . . . .
- Q:;: :A.
f ; .: 4:xy:.
.a.: -. vt H..w>
';) .
p, ,,y;.*r.v .p+:f
..o+ '[V'S'.<E?h'[
. IU .;9[
, ;t01.F '4 ';"
M?QT MiM 6. Rd r gis t
- p
- s:sdep:.
sin:i.#r s
.R..,"..w'<../'; .-h ,:: #
r y,g. hk".l;<M.w::k.y' :- : -: e
- y. '- i. .:
A
)$. . . .n-.
v3 p
'yv v . is4%xar$.d aug :: :r. < $>i:c?.4:G.fF a.: ;;;::. .:.r? .:M93
- -c;g w-n.au y- .
gagp
%m NUREG-1021 20 of 27 Revision 8
ES-301 Administrative Topics Outhne Form ES-301-1 Facility: Date of Examination: . Examination Level (circle one): RO/SRO Operating Test Number: Administrative Describe method of evaluation: >* Topic / Subject 1. ONE Administrative JPM, OR #' Description 2. TWO Administrative Questio's n 'S A.1 4.;e , r~s ,. j p@y r;> y 2g 7 Mpi <
,glc T.
t:- ,
,: : s . , . .
~ \ i4j:#-
'id?.t'
.w m 4
pg 1
#, \ N' 5(h . . ^fj#
,'g y %, N .;s: / - y.
q'
^
i p, ., , " ; , , %, f c v My %
.1# m %;; 'i>k %cs n . nein F y M % , .sy j '* b;iau.w ha? M Rf A.2 $; *" M 4?
- t. . I. s. p: ;
,A
,, A 4
,. .,. t$f y *n.s g, 4 .k::n .
9 Rz,4 A.3 e* C / M,-.-.,.,&E- %e y#"Nyj$*$nis:r i L.. h. V 4(Vi$i:,> Ec :s
'$Q),,
e ~ '.ll.fl'$'
## ky.;yf x l?
4 :ss> 4
- f. pr 9
1y , n-7 o.& $^.,-g,
, .v gs +
- A'4, + :' d ,e ws a+
v ^ <, < + sy
# y,-;" ,,,h' .
y% i NUREG-1021 21 of 27 l Revision 8
ES-301 Individual Walk-Through Test Outline Form ES-301-2 Facility: Date of Examination: Exam Level (circle one): RO / SRO(l) /.SRO(U) Operating Test No.: System / JPM Title'/ Type Codes
- Safety ' Planned Follow-up Queshons:
Function i K/A/G -Importsnce - D5scription
- 4) CO.:x S
/sh g.
- 1. a. $'$ r i
- b. h AIhkSkN gh
- 2. .y
, m?
- a. b g MP wuW%,.. . > n; q@w r
9*:p
- b. u.YW
- A @m
'n f n'e%. ,s i
- 3. e ug a.jy"Trh$i!N:dh
*$dd!hA RSp MMA.. frastMbo i 4.
h inh $$Q ' hV p h
$g w'*k$. rlE Q p
- 5. t#g ' '3lptw J a.
th$ )h? b.j? v W 6, f, s p ,,d,yl':,9 Jh,m ~ !
^
$$$e$5%
y -- i
^ " l% ?'M h. ..
91""' 7 % Jha:r . . b.
- 7. Myj@43;y j$ 'W y5a.L .;
99:sg,e ,. q p . a.
.geS;. :- '
7 r - N O s&ug8If 'l% .ugg
- S!jjM s
e b.
' %~
m$y$g$$14
- 8. /,m, ..
a. dWT i b. IA .A v ., 10;v';: .m$P" y f 1 mm a. I D.
\
- Type Codes: (D)irect from bank, (M)odified from bank, (N)ew, (A)ltemate path, (C)ontrol i room, (S)imulator, (L)ow Power. (P)lant, (R)CA NUREG-1021 22 of 27 Revision 8 O
L u . ES-301 Operatino Test Quahty Assurance Checklist Form ES-301-3 Facihty: Date of Exammation: Operating Test Number-
- 1. GENERAL CRITERIA a b c a.~
. The operstmg test conforms with the previously approved outhne; changes are consistent with i
samphng requirements (e g.10 CFR 55AS. operationalimportance, safety function disittution). /Fg
- b. .
o <
.s we Repetition from operstmg tests used dunng previous bcensing exammatione is withm acceptohle.
limits (30% for the walk-through) and should not compromise test int 4. T i.4 Nec ' +.He x c. There'is no day-to-day. repetition y.c.m. a. z.. wgy between this and other operating tests to be adminieletedg f* G.c duringthis examinehon. L. : . W % 2:-- _s_.-. W h g
--%2
- @O MCP Wh .
A 9
- d. C.%.
Overlap kmits. with the written exammetson and between operstmg test categories i wswithin W x 4 ' ' ' Y >< acceptable W.. e;U W' w:M e, + + my It appears that the operatmg test will differentiate between competert and inse than competent apphcants at the designated hcense level. rF WNN* n.
- 2. WALN.THROUGH (CATEGORY A & B) CbRIA nN:EAEN~ w@A - - -
- a. Each JPM includes the following, as apphcable: hh3 H WI conditions 'NN#
+
irWhetag cues - ,
.fhh references and tools,includmg associated im:- - i JDfe"'$ g.,e w gg I i
I vahdated time hmits (average tune ellowed for completion) sad deemed to be time critical by the faciNty hoensee 'desi0n8 tan If
- specenc performance creeriether enclude:
V detailed expected actions wNetsenet cribone en#7$ nomenciatutey i
- system response and othereneminor cuss #g .
1
- statements describing impeglant observatsee to be made the applicart !
- crtterie for successful complebon of the led $ .431V 1 i
j*restrichens on theeggesene of %. IfguisableWidentlRceton of crtiscal steps and their enes
- b. mmk7tipercentof
.m y;awan wagr Ishe pseessgesd (Administrative and JPM fotow-up) queshons are :
"psestemineellyopen refereneseed snest the criterte in Appendix C. There are no shrect look-up queellene,*acessary level 9-8===.deact J
.;.;the use of references.
od.
. W.s.wm.tr ww M.gs.r ]
Atleast 2Desseestef4he JPMs enseeseess'on each test are new or .;,a...;;p modined i w MEEdMEfiinanATOIMATEGORY w am C) CRfTERIA - - - ; a. The associated sirredelsrepo;gn.seEng tests (seenetto sets) have been reviewed in screrdance with Form ES-3014 anri a espyisabsched. I' h Pnnled Nome / Signature Date 1
' Reviewer (*)
% i Q??%kkh$$W%V
(*) (*) The tecluty atonature is not apt *=hle for NRC developed toets: two independent NRC reviews are required. ES-301 ' Simulator Scenario Quality Assurance Checklist Form ES-301-4 NUREG-1021 23 of 27 i Revision 8
Facility: Date of Exam: Scenario Numbers: / / Operating Test No. l QUALITATIVE ATTRIBtTTIS Initials a b c 1 ; m ' . . ; ' ::. ' /d- - u ; _n - 1. The initial ecnditions are realistic, in that some equipnent and/or instreentation may be out of service, but it does not cue the operators into exDetted events.
- 2. ,. p The scenarios consist most1v ef related events. #1
- 3. Each event description consists of the point in the scenario when it is to be initiated e. 0%
## Q Si . . .e 4 the malfunction (s) that are entered to initiate;the event : .s #
the syinptoms/ cues that will be visible to the crew 4 d& *% ' ' the expected operator actions (by shift position) lggs. N the event termination point (if acolicable) ""+ 4 y*",i g"
- O w
- 4. 1:. -, w.v No more than one non-mechanistic failure (e.g.. pipe breek) isiincorporated into the @a . c scenario without a credible preceding incident such as a seismic < event.
- 5. The events are valid with regard to physics and theracdy M N E N M b h.
- 6. g. .,. ,,a.w Eequencing and timing of events is reasonable, and al b the examinationNeam to obtain conolete evaluation results cu..~.surate wit &the scenario 'objectnesi
- 7. s:e .. m .. m %..,.,.
If time ccrnpression techniques are used, thepscenariosamary clearly sojndicates. Operators have sufficient time to carry out? expected actilrities withoutwndue time constraints. Cues are given. # D ' MA M B. The simulator modeling is not alteredT m j >
-- - 2 g 42 s ,%MM
- 9. The scenarios have been validated dp #p. yg w ex/
- 10. a gg Everyoperatorwillbeevalua(M$sifing at Jeest one new.or signi.ficantly modified scenario.
E5 301. All other scenarios V' have been. n ekmodified in accordance with Section C.4 of
#W 4 ..a./o. NO
- 11. F2.m.a . D ^ :.M @
A1Hindividual operator.. competencies can.belevalgeted, as verified using Form ES 301 f isutantithe form along with:the simulator' scenarios). v .<,;.m nm w w .mu 12. EachLapiteent will be significantly involved in the minima neber of transients and eventsospecified on Form ES-301-54 submit the form with the simulator scenarios). 13. w a.n n y. mm The levehof1tlifficulty is ap,p,r,a opr ,ciate to support licensing decisions for each crew positio C ' W A # A. WW 4vw - wp . +. ~ v.e..., m... TARGET gw %QtWITITATIVE ATTRittNES'TPER SCENARIO: SEE:3ECTI(N DJ4.d) Actual Attributes -- -- -- tyr, wy 1. et Total malfunctionsd5-8W / / f&, p w 2M Malfunctions afteFTOPe 'ntry (12)
/
A% a
/
135t Abnormal events 42-4) p mr ser
/ /
CO Major trans4ents (1-2) w / / C6P ,s 0Wesepw ntered/ requiring substantive actions (1-2) / / m, . . . u.y 6M ! E0P contingencies reouiring substantive actions (0-2) / /
- 7. Critical tasks (2 3) / /
ES-301 Transient and Event Checklist Form ES-301-5 OPERATING TEST NO.- NUREG-1021 24 of 27 l Revision 8 { 1 1
)
Applicant Evolution Minimum Type -Type Scenario Number Number 1 2 3 4 Reactivity 1 Normal 1 R0
-s
. Instrument 2 + W
< u.mm Component 2 A,h # N Major 8.
gy/' .. ..^ 9 fy 1 < A W %.
~~ g-g.g.g...
in %%+ ?lW Reactivity 1 @ t '4.
%%d > j/
Normal p <~ w, 0 4 ' d6 As R0 Instrument-
, yese m 1 .A W#i %
v Component .g : : 1_. ' Eh Itw Major swgem wp8~ :t v
.Aa?TMT h. v i
SRO-I AI IS4A$5NM}hMI l Reactivitk /[ Ikhh Nora,e w #gr. 1 y s1P f
~
As SRO me i Instrument i N 1.Ay : gk fl$t$$mnent ( 6536if u' cna A. *( frun~3. .p ;jNidor - , 1 a.nnam ,~n ,:,,, )
\
4Nf N Rea$jAs % 93 M.%s%.g,f;.gjp. ctMty# 0
'~
gM (f- NhNormd Np qua m 1 N Winstrument SRO-U I i t Component
/p 1 s, e . ' gh N3jQf }
- h;;:ih. ' '
'u stions: (1) Enter the operating test number and Form ES-D-1 eveat ggpqmm numbers sa "tb (2), Reactivityfor each evolution type. Appendix D. manipulations must be significant as defined in hhWkxaminer: NUREG-1021 25 of 27 Revision 8
ES-301 Competencies CnecKhst Form ES-301-6 Applicant #1 Applicant #2 Applicant #3 RO/SRO-1/SRO U RO/SRO-I/SRO-U R0/SRO-I/SRO-U Competencies ' SCENARIO SCENARIO < SCENARIO 1 2 3 4 1 2 3 % .12 2 3 4 Understand and Interpret Annunciators and Alarms j' g is
*w ~
Diagnose Events y 7.. .1" y? I? .,Y' n F M 7 and Conditions 4 4
.4 Understand Plant *g.A:
%!j; 45 t.
and System Response 7
- s m
e Comply With and , w n
~'
^
Use Procedures (1) ; b k % Operate Control n. ;, ,n
$pPu Boards (2) ,d % ggs E. 4' IT U i fdf
- y -
y Cornnunicate and < q t *a* Interact With the Crew #h 8 Demonstrate Supervisory Ability (3) Aw k 4-? , I $$b ik f. nm a Complygithiand. 4i~li$ M :+ z 5a" Use TedAN$he55IN3) 'DNN hk wap# Notes: w,mm%,>
%$7%p;u
.c - a as. "4$a$@s (1)
R[ !MihdjNN$ )k$%g *[ Ino.g.6 4 1 des'TechnicaHSp<s 9,;.:. mn . . (2) Aptional O for aMSROM: . . hy
.<,,?
.ecification compliance for an R0.
2 (3)4 0nly applicable ,tdc'SR0s.
$$$ hil Isstructions: /F e $p$$$g A dF;;
d klY'Nhappjgent's license type and enter theone"or more[ event numbers that t tsjQfgy eVaibate; every[cometencfifofieVer# appl 1 cant scanc, for cae ;Gn;rie in tM Author: Chief Examiner: NUREG-1021 26 of 27 Revision 8
r 1
, n .9 i ES-302
- ADMINISTERING OPERATING TESTS TO INITIAL LICENSE APPLICANTS
. A.- ' PURPOSE !
This standard describes how to administer operating tests to initial license applicants in accordance with the requirements of 10 CFR 55.45, it includes policies and guidelines for
. administering both the walk-through and the inte' grated plant operations categories 6f the operating tent. It is assumed that the operating test was prepared in acc6rdancinvith ES-301.
B. BACKGROUND j h As noted in ES-201, facility licensees will generally prepa& mek re proposed' operating tesisJn g?" y,
'accordance with ES 301 and submit them to the responsible NRC regional office for rev,iew and
~
approval. Regardless of whether it was prepared by thE Mnsee or the N'RC[eitery
. operating test will be independently administered and grhdiil lan NRC licensing examiner in accordance with the instructions contained herein and in ES-30346gg C. RESPONSIBILITIES - [ R ynnf s i- *
- 1. Facility Licensee . '
4 y 9, L
, m - -
The facility licensee is responsible rthegyfollowir$ ug' y .
- a. ' Make the plant and ation facillity avail , as necessary, for validating and
, administering (integrated plant' Cate@
t ions)#the opera 0$g tests ~(contr'of ro i
, b.; Safeguard rity and he operating tests in accordance with 6 15I i8T
- c. .. pdmini AlmuledierHacility,"@.andjogistics support (e.g., personnel to op
, lebts.in accordance with Section D.
inform t fe
, examinatasA%gional ss before it isoffice complete. in writing if an applicant withdraws from M
24 g a NRC Ramonal @ lice e I office is responsible for the following activities: with the facility contact to coordinate the operating test administration schedule in a manner that maximizes efficiency and maintains security (refer to ES-201). lflnecesserpiihe:: region [mapyela9]tholoperating. tests;for[up.toL30 days hpLB.iapprovalshmerggregionishould heap; OLBjnformed and consider notifyinglthe: facility licensee:inLwriting if the: delay;is'besed on the NUREG-1021' 1 of g t Revision 8 L:1.
ES-302 quayty of the proposed examination.,
- b. Administer the operating tests in accordance with Section D.
' D; TEST ADMINISTRATION INSTRUCTIONS AND POLICIES A a OF
- 1. EtDAIAl SA 3
gWy@N
- a. Before beginning the operating test, an examiner shall brief Napplicant(s) g using Parts A, C, D, and E of Appendix E.fTrisave timeMreconrnended thatf the examiner (s) brief the applicants as a f5iup. ff MQM$g 8
- b. If an applicant requests to withdraw g;any bY part of the durin$$w S examindts.N$$h on process, the examiner shall inform the applicant tha(this.ssill result in automatid, license denial and that he or she may reapply in accordancewith 10 CFR 55.35. The chief examiner will request the facility liceNieEIofdoddment the applicant's withdrawal in a letter to the NRC regionil administratir?$$
em% g$c dQy , W%%*Ab
- c. Each applicant listed on the examination assagnment she.g~ Vet (see E.S-201, rating Attachment 4) shall pr
" Examination Type." be adminisiirid 7 hy%gjg,testMindic d.
lg 3QW For purposes of test ration and continu$ tbe chief examiner should generally schedule snhame edininer to siNn,"inister all three categories to an aj fifcant. HSiiliever, urds certain circumstances, such as when a licensee'is)imulation3scB$hMilocated near the plan
/. quests exiihin5Nons for an'$anususilh large group of applicants, the f$$%p$mponsitssiiishi6051,supervi$8rM authorize the chief exam
%goperating te'st;cetegones and subcategories among different examiners
%sinTuistortoperlising Weis consistmg of multiplefsoonarios u shall not.be' divided EnWexaminers$p%ief examiner will be responsible for ensuring that gets a35mplete operating test and that the tests are thoroughly and accura
/gsiB(@telj,idocumented.N ffathree-pers$h on operating crew consists _entuely of seniorreactor. operator yp upgrade applicants;(who;do;not.have;.to be; evaluated on;the, control; boards)T.the chief exsithinermay[assignLonly.tuso~examinersito' observe the;cremisAlthough M.:ap M,;in.,the reactor. operator land.. balance..of plant positions.may;not..be indivy' lualRevaluated,NthemiNMgraded!and. heid' accountable prennafors t occur.as's: result of their schon(s) orinaction(s).!
- 5. The examiner is expected to administer the planned operating test in accordance with the prepared and approved walk-through test outlines (Forms ES-301-1,
" Administrative Topics Outline," and ES-301-2, " Individual Walk-Through Test Outline") and simulator scenarios (Forms ES-D-1, " Scenario outline," and ES-D-NUREG-1021 2 of 9 Revision 8 i
)
r j l ES-302 2, " Operator Actions"). Examiners shall document every significant aspect of each applicant's performance for later evaluation, but they shall not use the applicant's unplanned actions and statements to displace any part of the planned operating test. Normally, examiners should substitute or replace planned operating hst l materials only if it is determined that an item is' invalid or.. impossible to perform or simulate because of unanticipated access restrictions or'5q6iprhent failures. f. f ;54 % ?;i Exa' miners may administer the same operatin@g tes - to consecutive applicants and crews on ths#same day $fiUt tdsy ~ mudsnsure that the security of the operating test is maintained. Thei same simulathridersrids shall not be repeated during successive dayshg[ 98gjI if previously agreed upon by the 25dN facility licensee;b pxaminers may also administer the same operating test (walk-through and'simulaM$ dividing the test into segments that can be administered to all of the appkcontsjon the same day. This will minimize the amount of effogdduired to d'%)iifferent operating tests but will complicate the scheduling process. Ng7 g. g y*^ ~ Q Wq? g gt The examiner should normally admin test first and attempt to concurrentl/isterCategoiMs,?8 evaluateasimany of the planned and C of th administrative subjecta8CategorfA as poisiiile. The remaining administrative subjects should theridivaluated in acconilince with the approved outline. h. (( Mik JF The examiner.inust take suf5clent notes to facilitate the thorough documentation
- of any and afspplicant defibishciesgaccordance with ES-303. The examiner
)
! %st be siiiiis?cE6ss-refere'Edikiibh comment to a specific JPM, simulator IAnt, or qtIsstish.' % !
- 1. The kuring the administration of operating tests is not
./,
hh The numtM@o$h!!h. ensure thMintegrity v fpersons present during an operating test should be limited to of the test and to minimize distractions to the applicants. liscept for the simulation facility operators, no other member of the I j cility's staff shall be allowed to observe an operating test without the ' ief examiner's permission. Facility management and other personnel 1 E deemed necessary by the facility licensee should generally be allowed
' access to the examination (under security agreements, as appropriate),
provided the simulation facility can accommodate them and there is no t impact on the applicants.
'Although the simulation facility operator will normally assume the role of i
NUREG-1021 3 of g Rev..ision 8 I
i i
;)
ES-302 the other personnel that the applicants direct or notify regarding plant operations, the chief examiner may permit other members of the facility training or operations staff (e.g.,'a shift technical advisor (STA)) to augment the operating shift team if necessary. The chief examiner shall fully brief those individuals regarding their responsibilities, reporting
, requirements, duties, and level of participation befpre the opeifating test begins. The examiners must not restrict the surrogate opeFators' activities to such an extent that the applicants b irdiistuated are required to assume responsibilities bey 5nd the'shopsMtheir .
The surrogate operators will be exp$ctAd s to a I Withran' STAishall be" conducted irUsecordanbEwith'the i$7$h$libehi&i's
~
hormalIcordrol room practice;;an'STA shahiiot be the stati~orsd~ist6e
~
s' imulatodf.theyjare on-call in the'liiErit?$h.. 2 M $$$A If the facility licensee normally operates withiandis required by its technical specifications to have more thar'iTWeiliActor operators (ROs) in j the control room, Obikthe chi Isiaminer:m$[Initho&e the use of
~
additional surrogates tolill 00EWerews.~ In sDisi8ases, examiners must take care that the presNiEEofsdilitioni operators does not dilute the examiners' ability gEialuateh5EMTi6 ring theiroquired number of eventsland;oripery comhitencygNNibg factor. Examiners shall not hesitate totun additional scenarjos, as necessary, to ensure that every applicisDiigivenMie opportsilty to demonstrate his or her competence #Drdy;ord" individual 5pplicant orsurrogate),is. allowed to fill a shift suddivisor orf45sger.p~o~s'i tion donng;the simulatotoperating test. M NMINE UniNdm@oh@ircumetancesisili another applicant be allow
" ~
opei$$ndisilf> Operating tests are not to be used as training vehicles for M. future app'Ncinish n b )
""^Lott!likexam%$f'V iners may observe an operating test as part of their training or "jlsuditbe per'formance of the examiner administering the operating test.
' xaminer may permit other NRC employees, such as resident inspectors f(jy ,ohserve operating an, regional personnel, test. Personnel who areresearchers, not NRC employeesor NRC (e.g., superviso
/gg sentatives from the Institute of Nuclear Power Operations (INPO))
if$ . may observe the operating tests with prior approval from the Chief, OLB. gg The chief examiner will control the observers activities in accordance with guidance provided by OLB. The examiner should also give the apolicant the opportunity to object to the presence of observers.
- k. The chief examiner should confirm with the facility licensee that the simulator instructors station, programmers' tools, and extemal interconnections do not
~ NUREG-1021 4 of 9 Revision 8
p , t !~ ES-302 compromise operating test security while conducting examinations. The primary objective is to ensure that the exam material cannot be read or recorded at other l_ unsecured consoles and thst examination materialis either physically secured or electronically agreement. protected when not in use by individuals listed on the security
- g
! l. ' x ; i The chief examiner should arrange for any NRC examiners who are not familiar L with the facility to obtain a tour before they administer any 6peMting tests. The
- tours shall not be conducted or observed by anfof thefipplicants. In addition,
! the tours should concentrate on areas of thejiant that willladdd during the [_ L examination process, such as the contro[toom, the sir $0lation"facilify?and planned walk-through locations. @& gV NjPV$ggf f
- m. iN@ k The chief examiner will conduct an exit briefing @,with the facility lic nsee after the i
operating tests are complete. The briefing should address any generic weaknesses noted during the operating tesis idAnp[5ther significant issues l (e.g., problems with the reference material, the simulation [ facility, or the plant) l that might be addressed in the examaission, report. Theindivfdual operating test ! results are predecisional until approved btNRC management in accordance with ES-501 and shall not be shared with the facility licensee during the exit briefing.
- 2. Walk-ThrouohlCateoories A an g[
! a. The examiner should date arty JPMs that were not previously validated by the l facility licensee orjijyihe NRdiduring a p%aratory site visit. This is particularly important forpornplex JPMs pth'at require the applicant to implement an l }-
), WR .f%altemative RQQ method directed $@W% pihntpro s
bNkdTo the extenipessitAthe examiner should have the applicant perform the l kbontrolyom howSorshe wou JP@ld accomplish the task.tisinfimulator, ra L
#g$$$$Wil% Y NcblNNOniner is expected to coordinate the administration of the JPMs t maximiziWiis l be administese%cf the simulator. To increase efficiency, different JPMs
! d simultaneously to multiple applicants, but the examiners must w ensure thEt nIutual interference is minimized and test integrity is not comp ed.
/- .
WhenDPMs or discussions are conducted in the control room, the examiners make every effort to accommodate and not interfere with normal shift "perations. The chief examiner should request that the facility training manag notify the shift supervisor when the NRC will be conducting examination activities in the control room. If the number of persons or the noise level in the control room is excessive, the examiner should, if possible, move to a quieter location, modify the sequence of the JPMs and retum when the level of activity in the NUREG-1021_ 5 of 9 Revision 8
ES-302
. control room has abated, or ask the facility training manager to address the issue,
- c. The examiner should encourage the applicant to sketch diagrams, flow paths, or other illustrations to aid in answering the examiner's questions. In all, cases, the examiner shall collect the supporting material because it provides. additional documentation to support a pass or fait decision (refer to ES-303 copying, the applicant's drawings should be restricted idond$$$h)?T of separate sheets of 8.5 inch by 11-inch paper; the backi6f Form ES-303%or its' ,..
attachments shall not be used for this g purp?ssi. 44yMN g Q Af 46 Nn:a?% v .3% 4
- d. hca,, nt to use&&
The examiner forms, schedules, should and procedures if the encourage the app /5idraie .
- e. The examiner should be careful not to inf system and a inistrative knowledge from observations made durinibai$siifyl67^3The examiner should keep in mind that the applicant's proficiehey in eierp administrative topic and each control room and in-plant syyteriGiould be dell 65dialy* evaluated in accordance with the operating test ttiat was.'preparodirfaccordance with ES-301.
fV *
$P
- f. The examiner should ask unplannad follow'upjusstions only as necessary to
-prescripted clarify or confirm question 9s thepl# cant's u6derstanding of a preplanned tas
- 3. Simulator Test (Catqgo
' ~ .8afore a the tes e examiners will validate each scenario on the Mmulator ti5nsure that it will run as intended. Scenarios that were adapted
]sievious NRC}$N$$mations at the facility or from the facility licensee's bank mayMmquire regjalidation. At a minimum, the examiners will" dry run" Athosoever6having Mable inputs and questionable outcomes and discuss the
- Yremai$NM$lis: scenario with the facility's s!mulator instructor to ensure that it 5
y will runw$iM-w In some cases, the scenarios can be validated while the applicants are taking the written eismination.~ However, it may be beneficial to validate the scenarios duringgpreparatory site visit as determined by NRC regional management (refer toe $201).
# examiners will take precautions to prevent the scenarios from being revealed to the applicants before the tests begin. If significat portions of the scenarios are dry run or otherwise reviewed with the simulator instructor (s), the chief examiner shall ask the instructor (s) to sign a security agreement (Form ES-201-3) to protect the integrity of the simulator test.
NUREG-1021 6 of 9 Revision 8
l. 1
- l. 2 i
ES-302 c. The examiners should revise all copies of Forms ES-D-1 and ES-D-2 to reflect any changes made to the scenario events or the expected operator actions as a j result of the scenario validation runs and reviews. These revisions should be neatly written in ink so that the forms can be used in the final write-up of the -> \ simulator testi as discussed in ES-303. w g_ ! d. g 9 The examiners should review the scenarios together and discuss,the required l l ~ procedures, technical specifications, special circumstances 3srd so forth, related to the scenarios. % % ,
- e. n Ah w.w Immediately before beginning the simulator tests, the examiners s' id+Jid reviewsM the scenario events with the simulator oktor and provide him%he@$th[a#
copy of Fomi ES-D 1. This review should familiafize the operator with the* sequence of events to ensure that they Will proceed as planned. Th@is j ' particularly important if the simulator operaldr[during}the test is not the same individual who assisted in validating the
- scenarios; d i f.
yy ~ x l The examiners should identify important_ plant parameters to be monitored during each simulator scenario. The chiefiIdminer<should aIsEtis simulator operator , to record selected parametersfifNsilbid,!%r$he facility $ safety parameter j display system (s). ParamsE readirdihoulddc611scted at meaningful l parameth, the'
' ofintervals, the simulationdepending facility / on%sThe chief examinegs@r:should as reta] 4 backup documentatikNi augrn$5t the notsEtaken by the examiners during the
, simulator test. g7 g$ gf
- g. e examir $li rge of should arrange a suitable
' idemmunidiN65Mem withMiIi6ulator operator so that he or she can be [$tAplanneh@Mtermined time or power level so that th endilidiscility operatoiare aware of the event that is occurring or pending, if
~ '
, L ' gxaminers may use time compression to speed up the respon meters so that the scenario can proceed to the next event i
within a ~ _ le time. Time compression is acceptable as long as it is used judiciousikhrdthe operators are given sufficient time to perform the tasks that
. they compr w$Esion they should inform the applicants
, of that fact duri g e operating
$ test 6riefing (refer to Section D.1.a). The examiners should also mitigate the
""htial for negative training by debriefing the applicants after any scenario in y _
ich time compression was used. h.' Before beginning each scenario set, the examiners should have the simulator l cperator advance'any control room strip chart recorders that may prove useful in i recreating the sequence of events. The charts should be clearty marked with the j ' NUREG-1021 7 of 9 Revision 8 j
\
L
m ES-302 date, time, and examiner's initials so that they can be accurately matched with the correct operating crew,
- i. The chief examiner should ensure that the simulator operator (or examiner) playing the role of other plant personnelis aware of the time scale for. resp?nding to the applicants' requests for information. For example, fast-time could be specified for auxiliary operator checks or lineups to prevent long delays in simulated operations, while maintenance and chemistry daMplelinformation can be provided with normal time delays to present the appficants with the same analysis problems that they wiu face as ogsiors. [,
g @^$(dh
- j. Before the simulator test#ybegins, the exammers $hu shalb[ ca operator to provide only information that ishocifihilly requested 69jhs7 applicaats:and does not compromise the{integrit(of the ex simulator operator is briefing the applicantsTor?comrsunicating with them on the .
telephone, the examiners should monitorM dhindstion to ensure that the information provided is appropriate and oes not* icants. k. dd % w Each examiner should use the,axpected actions and beha&viors listed on Form ES-D-2 as a guide while adnunisieriidisssidiblator tesiIi. If an applicant performs as expected, theisaminerpYQgin the left-hand column of the form the time when t6 expected actionsoccurred. However, if an applicant
~
does not perform as su (or lack of actions)M^ io or beEYwpscted, the expctedthd' actionexamine and follow up $[s"hould with note th appropriate questions after thesimulatorscenario is completed (refer to Section D.3.1). k 4hg$fEach exa ,,$, dete$ best way to document the applicant's actions. Someauaminers record a minuto-by-minute account of all key plant
. %w%eSenkand the*sjiplident's actions as they occur; other exam j
g;cianlexaminatiohlscumentation technique; however, the doc
/ exam mus
$rovidsin adequate basis for a licensing deci ineidsinot$$hust provide sufficient information to allow the examiner to confidentijj%'the applicant's performance on the competencies described in
#[ I.
Append . Exarniners shall limit discussions with the applicants during the scenarios both to mairlIiii'n realism and to avoid distracting the applicants from operating the plant. _ , ' examiners' questions during the scenarios should be limited to those that re necessary to assess the applicants' understanding of plant conditions and the required operator actions. Whenever possible, the examiner shall defer questioning the applicant until a time when the applicant is not operating or closely monitoring the plant (preferably.after the simulator has been placed in
" freeze). The examiner's follow-up questions or concems can generally be NUREG-1021 8 of 9 Revision 8 l
ES-302 3
' addressed during a brief question and answer period after each scenario or during the control room systems and facility walk-through portion of the operating test (i.e., Category B) if it is performed after the simulator test.
mi The examiners who administer the simulator test shall confer immediately after completing the scenario set to compare notes and to verify that each hxaminer observed his or her applicant perform the required number of transients and events in a manner sufficient to justify an evaluation of allMrequired ; competencies. If necessary, the examiners:shall run arisddNional scenario to , ensure that the required evolutions and compntencies are cousredhAll scenario will be planned and documented & Ay in accordance yew 9 w n tM The chief examiner shall ensure that the examiners' observations are co.~. mnsisten and that their findings are mutually suppstisi$if a performance defiilincy ie
" shared" by more than one applicant, it shdul(65)oted by both evaluating examiners. ideally, this cross-check should be E&er@shed as soon as possible after running the scenarios whild still at tinfacilitEsThe cross-check must be accomplished before finalizirig"Ihe examinitionlreAUSs in accordance with ES-303.
fig %g ]sF (- n.
$Y {kWD$i If the applicants did not perform as expected $ tiny'es)aminer shall ask the simulator operator to prodNie copies'6f the idysMirts, and other materials that may be required afterisiying thencility to sysidste and document the applicants' performa$oep The .aiEminer of $iicord shall retain all documentation related to any opeinting test f$ildre untyMe" proposed denial becomes final or a Ah,b license is issued. /$%ff d $
Jg$g$he chiefex ishouldo"ask N 5aitelmateiisl(On5Ehll:a als$5bbV the simulator operator to retain copies of the
%j0ifssted in tiiieTiEmpiU^pplicants are licensed or all appeals are settled, as corporate notification letter shown in ES-201, Attachment 3. % f[f
,r.3
,.m
)
ifthe neces
- facility should become inoperable and cause excessive delay of the operM_ ngtest$?the chief examiner should discuss the situation with the facility licMtsesInd the responsible regional supervisor so that management can make a diEisi$n regarding the conduct of the operating tests. It may be "to reschedule the simulator examinations for a later date.
i 1 I
~ !
NUREG-1021 9 of 9 t Revision 8 1 1
% f
)
ES-303
- DOCUMENTING AND GRADING INITIAL OPERATING TESTS
~ A.' PURPOSE.
7 This standard describes' the procedures for documenting all 6.ategories of the operating tes ,
- collating the data to arrive at a pass or fail recommendation, and reviewing the documentation to ensure quality.
jk.: g# B.~ -BACKGROUND MNY k+9 .N[? % This standard' assumes that the operating test was prepardand adminEMii$r[accord with ES-301'and ES-302, respectively.- In addition, th'e %cedures e' xaminer to evaluate. each applicant's performance on tiidoperatiriftest and Malis i
. Judgement as to whether the applicant's level of knowledi$?andIuhderstandin titk
~
minimum requirements to safely operate the facility for Mdilicense is sought. The examiner should evaluate each noted deficiency in light of tiididiAl breadth of knowledge and ability demonstrated by the applicant in that subject ares 7 M i MWw
- When used to evaluate an applicant's performance $$iljor part of the operating test as discussed herein, the terms " satisfactory" and , , " are defiried as follows:
l S - Satisfactory Working Knowle dU The applicant may have soridMght or wiinor diffi describing system interactions.
- Competence in the operatidkFequipnIdnt associend with the system is very good, i although there may be scIsiIhesitatidfwhile desiEssing or performing some tasks. The 1
- applicant appears to;be NiIniliar wittNequipinint and procedures.
factoryear; .. rking and Un'derstanding 3hid%f$ ! The a h %has diffi' answering questions h; depth and describing the interactions of systeniEADiisiassionsk$iihi@Ior while operating equipment show lack of familiarity ! h Mres. The applicant is unable to answer questions or
.prohliies
~ with Di@incorridMicomplete answers. The applicant i i em, as evidhiiddijiphesitant answers, inabi!ity to locate information, inability to ,
locate procedures. control boardisilications
# or controls, and lack of knowledge of system operating I C RESPONSIBluTIES 1 g nsee r
The licensee's responsibilities are limited to providing the NRC examiners with whatever additional reference materials and information the examiner might require to evaluate the applicants' performance on the operating tests. Such materials might include simulator strip i chart recordings that document plant status during the simulator scenarios, and procedures that
. NUREG 1021 1 of 29 - Revision 8
I ES-303 h _ document the expected operator actions. > 2.- ~ NRC Examiner of Record As soon as possible after administering the test, the examiner of record shall review,, evaluate, and finalize each applicant's operating test documentation in accordance with the; instructions in Section D. . [^ jQ" A A 's
- 3. NRC Chief Examiner #) 'c 'd;':Y',
a. by;uh h
. The chief examiner shall arrange a meeting of the NRC examination team A!r members after the simulator scenarios als'compleid?Such mEsiili examiners to compare notes to ensure 15ifthe d6Eumenta fs on the same operating crew is consistent and mutually, supportive.
- b. 4%f'5 The chief examiner shall work with the other e;xaminers on the team to resolve any technical questions that might arise'$uring th"AhrodiIdiprocess, and communicate any additional referenciE5Esterial requirdrheritiIto the facility c.
Ah IA g The chief examiner or a managementsppsowed'de. e ,lhnee will review the grading of each operating test toIMIrify thathie exaIrhinskiEomments appropriately support his or her recordihendatieffand to aliiur's that the operating test meets the requirements ofM301.; ifWe chief edEniner or designee does not agree ~ with any of the exisiiher's recibmrnendati6Es, he or she shall confer with the . examiner before OI5ftumingNrecopihimdation. Such disagreements are not {
+ t,.ified.
common 'hy;{@ikre, accurate in ihm very$[rtant arui$$timIly and documentation. for examiners arise bidiiEs to be Ma
' $$p?p g'
- d. TOj xaminer;erdest
~
pii6cs6siendation, sii$Me "inee shall make an independent pass-or- fail Final Recommendation" block on Form ES-303-1, "OpAr~ish(gg Examination Report," and forward the package to the responsibiaisupervisor for review in accordance with ES-501. The supervisor must conisiFhifiny recommendation to overtum the examiner's results, and the specific rMsois for this action must be explained on Form ES-303-2, " Operating 3 Test Coriinents."
. f GRADING #fD DOCUMENTATION INSTRUCTIONS w ~. and Cataaorize Rouah Notes and DaruT.eatation
- a. Review the job performance measures (JPMs) and simulator scenarios that were performed and the questions that were asked. Evaluate all rough notes and documentation generated while administering the operating test to determine the NUREG-1021 - 2 of 29 Revision 8 l
E l' l: g ES-303 l areas in which the applicant was deficient. If the applicant generated or used any material (such as figures, drawings, flowcharts, or forms) during the operating test, the material may be used to aid in documenting the applicant's l
- performance, if it contributes to an unsatisfactory performance evaluation, the material shall be appropriately marked and cross-referenced to the ap' deficiency and attached to the examination package for retention.
e y
- b. Verify the validity and technical accuracy of any questioktNet re asked l during the walk-through test but had not beeri prescrip'ted, as Well as any unexpected events or actions that occurred during the simulatoidperatir'g test.?y
! if necessary, work through the chief examitier to obtaifany'addidohil reference material that might be required to resolve any technical questior$5mlI$f
- c. On the rough notes and documentation, label,or
-%W % Y highlight every n, response, '
note, or comment that may constitute a performance deficiency. L d.1 O N;;pAz i Label each deficiency related to the applicant's admmistrative and plant system knowledge and ' abilities with the alphanIdneric code *bf $iA3dMinistrative topic
)
(e.g., A.1) or the control room orylalhIs/Adm (e.g., EE'1Mr B.2.1) to which it
,pp,;,,, y jpm o ,%cm g;g .
[ 8$up- NL ' $$dg L e. Review each simulator opeirating test'performa$nce' deficiency. Using as a the competency and,nating factorjdiscriptiopiri Appendix D and on Form ES-l 303-3 (RO) or Forrp'EF303-4po), codgesch deficiency with the number and letter of the rating 15ctor(s) itsnost accurately reflects (e.g., C.4.a). ' Whenever possible, attempt $iidentifyhIsoot cuidse' of the applicant's deficiencies and l-l p~kcode eachgefioliiscy with noms 6EEMiin two different rating factors. Howe
~
i
$.cne signifi6lidfdeficiency miyEIf5aed with additional Mcited rati 3M sj(
, ;stai$ din'ES-302 essential that the simulator operating test i docuInisie0Enis consistent and mutually supportive for all applicants in an I operatin{%jhrating errors that involved more than one applicant should
- be noted by tech applicant's evaluating examiner. If the examination team '
membersbo rYot have the opportunity to discuss and compare their observations before lai$ ring the site, the chief examiner shall schedule a conference call after ! the exir6iners retum to their respective offices. 2 .
. . ;.2 er categorizing and' coding the rough notes, review, evaluate, and grade each applicant's performance in operating test Category A, " Administrative Topics," Category -
B, " Control Room Systems / Facility Walk-Through," and Category C, " Integrated Plant
. Operations," as follows: .
NUREG-1021 3 of 29 Revision 8
ES-303
- a. Form ES-303-1, Category A Review the identified deficiencies and decide whether the applicant's knowledge and understanding of each administrative topic was satisfactory or unsatisfactory (refer to the discussion in Section B). Document the grade by placing an "S" or "U"in the appropriate block on pzge 2 of Form ES-303-1. Every unsatisfactory grade must be supported with detailed documentation as'; discussed in Section D.3. 4 20 '71fk l-gh %4kh.
After grading all of the topics in Category [Kassess the Mnt's topic gradesf. and deficiencies and assign a single "S" or*U" grade for the'catgorygif thegr applicant has a "U"in only one administrative topic /the examinermay. fail.the' applicant in Category A depending on thiilm'portinbe of the identifie'd deficiency. However, if the applicant has a "U"in twdbrmore of the administratid topics,
^
the examiner must assign a grade of "U" fgC$te0'ory A. Place the assigned ' grade in the " Administrative Topics" blocli of th470perating Test Summary" on page 1 of Form ES-303-1. Enter"N/E" not examined)#this category was waived in accordance with ES-204.ytF(i%, 3/
- b. Form ES-303-1, Category [ SHINS %jif a e names of the systems OnJPMs and page 2 of the examined duringapplicants opersing test Su Form ES-3031[bcategories 0.1, " Contr Systems," and B.2 f Mity WaW-ThrouggAlso enter the safety function number from the appropriate)(nowledge and Abilities Catalog (NUREG-1022 for PWRs and NUREG-1023 foriBWRs)eg ffs Aff VlIWW .
p ETo determine UbeMPMs listes on~ Form ES-303-1 were properly performed, k l M -evaluate eacit deRdlency coded in the rough notes for Category B. !! the I h,$ stowing criterisassgassign a satisfactory grade by placing an "S"in the "JPM Evaluation"*columr:6r that system; otherwise enter a "U":
#[* kidNk 44fME:JPM Ytime-critical, it must be completed within the allotted time.
was
/
cal steps identified for a JPM must be completed correctly.
, if the a cant initis;ly missed a critical step, but later performed it correctly and f aw,eyshed the task standard without degreding the condition of the system or thepisht, the applicant's performance on that JPM should be graded as
, sfactory.
Further evaluate the Category B documentation to determine a grade for the
- prescripted questions associated with each system listed on Form ES-303-1. It is permissible to allot partial credit, when appropriate, to deternha whether or not the applicant's understanding of a particular K/A topic is satisfactory.
NUREG-1021 4 of 29 Revision 8
1 l r l ES-303 , 1 If the applicant's understanding of both K/A topics is satisfactory, enter an "S"in i the " Question Grade" column for that system on page 2 of Form ES-303-1. If the applicant's understanding of only one of the K/A topics is satisfactory, enter j an "S" unless the knowledge or ability that was missed is of such safety significance that an' unsatisfactory system grade is justified; in the latter case enter a "U." The K/A importance value will not be the only considerkdon in arriving at the unsatisfactory evaluation, and the safety siinificandsimust be i explained in the documentation. If the applicant's understandwidof both K/A topics is unsatisfactory, enter a "U" in the "Q0eition GfadsicoliJmn for the system. Every grade of"U" on a JPM or tlAiystem queN$nnrnst be - 8 supported with detailed documentation asMscussed iriiecnon DXh
-Qr je Y
%fi For each Category B system listed on p5de)Aly of Fo'rm ES-3
" System Grade" of "S"if the applicant's performance on both the JPM'and the prescripted questions was satisfactory. If the[spplicant was assigned a "U" for either the JPM or the prescripted questions, then enters ,"U" for the " System i Grade." \ t^
gg?A N e,?'i; I After grading the applicant's pe respectiois[il the Category B systems, determine an overallys$ifkUsikB byNhiculating the percentage of satisfactorypstem gradesQfQapcant has an "S" on at least 80 percent of the systems examined,*the aphlNant passes Category B and ; receives the an systems, the "S" overally' ~ t fails Category B and receives a "l pp * . i r p , Document the a nt's grade b g an "S" or a "U"in block B, " Control 1 gdhRoom Sys.teinsisNi FacilityM Ti y, n6iJgh," in the " Operating Test Sum dy{$ age 1 oftfriniES403-1. Irit fNE"if this category was waived in accordance g g Using . 3 3 or ES-303-4, depending on the' applicant's license level, evaluatini$efi6iencies coded for Category C. Circle the integral rating value
" (1 througli 3)b6Triresponding to the behavioral anchor that most accurately reflects tisi]iap"hiicant's performance. A rating of"1"would be applicar(Enissed a critical task (i.e., by omission or incorrect performance) or 1
comrrutisid multiple errors of lesser significance that have a bearing on the rating facl0FMissino one or more crifie=1 tanks does not nece==arily mean that the [%nt will fail the simulator test. nor dcss aurease on everv cin;cel inek Nevent the examiner from raccmic.eadina a fai!ure if the asslicaat had other deficiencies that in the mooreaste. iustifv the failure based on the camsetency evaluatens. As discussed in ES-301, Competency 5 is optional for SRO upgrade applicants. Multiply each integral rating value by its associated weighting factor to obtain a NUREG-1021 5 of 29 Revision 8
ES-303 numerical measure of the applicant's performance on each rating factor. Then, circle the corresponding numbers on page 3 of the RO or SRO applicant's Form ES-303-1. For each rating factor, place check marks in the columns on page 3 of Form ES-
. 303-1 corresponding to the numbers of the scenarios in which the rat $d behavior was observed.
g f 14f
//'
For each competency on page 3 of Form ES-363-1, sUr6 tNA$ifeled rating factor gradesshould and range enterbetween the resulting 1 and 3.) competenc[gsde in the " Total").olumn. grades f7 M4*%g$p!V
&, gjf %" Ah Using the following evaluation criteria, deiMmine if the applicant's oseralf 2 performance in Category C is satisfactorilofensstisfactory and doc'ument the grade by placing an "S" or a "U"in block Cgljedated Plant Operations," in the
" Operating Test Summary" on page 1 ofform ES;303fi, Enter"N/E"if this category was waived in accordance with ES-204Nps 44 If the " total" grade for a/I cies is 1.8, the applicant's
,}V performance is generally:sati gy ^
If the applicant mads (an) error senlous safety consequences for the plant or the p6blic, theMam(sinerMncommend a failure even if failure can nodsdjustifiedhn the basik of the competency grades. In such circurristEibes, thUegion~ musi obtain written concurrence from the
, Chief, .0LB' 3efore etingJhi'iicensing action.
If grade f f ency 6, " Communications and Crew I" }is less thilifor equal to 1.8 but greaterthan 1.0, and the s individusthhtil"agrades for allother competencies are 2.0 or greater, the lh%krapplicanVE)N$$Varf$imance is satisfactory. q1f 9dgtal" Nahe for Competency 6 is 1.0, orthe " total" grade for any _ tency is 1.8 or less, the applicant's performance is u ty. Note: Com k applica#. However, petencyif5, it is" evaluated, Control Board Operations," it shall be is applicant's factored into the optional for SRO upgra
- final e.
in detail, as discussed in Section D.3, each rating factor that is assigned In. integral rating value of 1, regardless of the " total" grade determined for the associated competency. If the applicant's overall performance in Category C is unsatisfactory, Justify in detall every rating factor assigned an integral rating value of 1 or2 for each competency that has an unsatisfactory score. NUREG-1021 6 of 2g Revision 8 l
ES-303
- 3.
Einalize the Documentation i a. Review and finalize the simulator scenarios that were run for Category C of the I operating test. i A Complete Form ES-D 1, " Scenario Outline," by entering the applicants' names, i i the positions they occupied during the scenario, and the fability's'name on the top of the form. Also enter on Form ES-D-1 any scenarihiesisi6ns made during the test, so that each form accurately showsfallfof the'eventithat actually occurred during each scenario. Change tha!Afent numbens7mAlf0nction numbers, malfunction types, and descripti6$s as necessary id refissOhe "as rdN' [
~
ypind thi conditions. These changes may be mads [Using p scenario, provided that the final form is clear and legible. g q, Update each Form ES-D-2, " Operator Actions," to reflect the "as run" conditions. Discard or mark as "not used" any events that were not.run, and fill out new forms for any events that were run but not originally comments, and additional actions in tfiEspaces betMl planned. Neatly ente dliIllWexpected operator The final Forms ES-D-1 and'ES-D-2 must be a ar record of the actual ever6and acti6hs thaIix:50irn;tegible, and sequential during the simulator operating test. The fornYi sent toihe applicinilshall not contain any rough notes orirrelevant commeNtsjP #7 Ak fj;in AnyevaluatingNA ctions .' events orpplicants fdidg ion as expected or were not useful
$ perform)preQpe!noted on%'the master copy of the scenari pg%Jacenario S
Q b. N h% 3l[ M R$viewthe.appli t ES-3031 and the rough documentation. Justify in
/'nisinifMFAivr! ESd$,'"
ability'$$ikisNiyjhat contributed Operating to the assignment Test Com of a "U" for any administralidiapic in Category A or any system in Category B. Deficiencies j pf' t that contribubodto Category C integral rating factor grades of 1 and 2, as discusse4Jn Section D.2.c, must also be justified in detailif they resulted in an 3 unsatisf4 story score in the associated competency. es that do not contribute to an unsatisfactory grade anywhere on the ggggg;g[cihEating test shall also be documented; however, a brief statement describi yy*gg**%P>the erroris generally sufficient. Examiners should keep in mind that their licensing recommendation and the associated documentation will be subject to review by the chief examiner and regional management. Therefore, the documentation should contain sufficient detail so that the independent reviewer, t responsible supervisor, and licensing official can make a logical decision in NUREG-1021 7 of 2g Revision 8 1
ES-303 support of the examiner's recommendation to deny or issue the license, it is expected that the documentation for an applicant who marginally passed the operating test would be very similar in scope and depth to that for an applicant who marginally failed. A Provide the following specific information, as applicable, for each deficiency that contributed to an unsatisfactory evaluation: , jg. V A 4bmE the question asked or task administered (i.e., describe.the JPM or the simulator scenario and event, as wsit'as the applidant'sp'Aition on the 8, operating crew) T MI T 'utL4yTI I_ : (h& m je77 g% s.... x w r the applicant's incorrect answer or! action and an indication o"whether the action was a JPM critical step E d it
' %d.> :Q%
sh . k$ % the lack of knowledge or ability that the' applicantfemonstrated v g mr .
- m. Ws kIb the consequences of the applicant's incorrect l answer ~or action M,hE h Ebh Y the correct answer or action, vnth an appropriate facility reference (e.g.,
lesson plan, systeridhscripticIri[prodedurienme and number)
., +,n!
// Jhy 't 1:Y the K/A number!and its importance rating *
(as given in NUREG-1122 or NUREG-112kE8d the15sility's learning objective k? the item from 10 CFR h$$ s$1?
'# tat the applicant did not understand or A was:sisitdsito perfo$$$
k 4%dEIh
_ / General statemeS$lsuch as "did not know decay heat removal system") are 5
~
' 7 possible, Nsbstantiate comments with printouts or strip chart Tillustrations,gensisted by the applicant.
.Miderated Suring the simulator operating test and drawings and Retain mentation until the chief examiner and management have
?ggf . ~~
reviewed $he examiner's recommendations and concurred in the results (refer to ES-501)f
. aq.y.
~ ~
y s reference each comment on Form ES-303-2 with the specific task,
,,p'y sub hject, or competency rating factor to which it applies on th ES-303-1. Do this by entering the applicable alphanumeric subject reference from Form ES-303-1 (e.g., A.2, B.1.3, C.4.B) in the left-hand column of Form ES-303-2, and entering the page number on which the comment is found in the appn,p.iste block on Form ES-300-1.
NUREG-1021 - 8 of 2g Revision 8
ES-303 4. Make a Final Recommendation a. After grading and documenting the operating test, make an overall recommendation by checking the " Pass" or " Fail"(or ' Waive"if the entire operating test was waived. in accordance with ES-204) block, and signing and dating the " Examiner Recommendations" section on the applicant's Form ES-303-1. ' Make a " pass" recommendation only if allsummar bloed 6f the operating test contain satisfactory (S) grades the applicant was not examined $p in that Ng[ ?$$g areatg,or the letto b. b; Aj v % sb If the written examination was not waived' arid the writt5n exisminid: not yet been entered on Form ES-303-1,$ute the eNaminatioribah.kage*16'th5 written examination grader (or NRC revindfif,th5' examination waspradid by the facility licensee) for processing in accordance=with ES-403. If the written examination results have already been ent5 red / forward the examination package to the chief examiner for review.7 Midg gj( E. 9 af s,. ATTACHMENTS / FORMS gfQ %. [ Form ES-303-1, AMbe~A W
>*h
" Operator 1.icensing EbMinsiisiERegi6rt" Form ES-303-2, Form ES-303-3, " Operating Test Comments" g %jj@![hgjf Form ES-303-4, "RO Competency'$$ ding W6rksheet forIntegrated Plant Operations"
~
"SRO Competerdy Grading Worksh5Eiior Integrated Plant Operations" g((# 'Y bir "
~ ,
p Y *
<6hi!A
%mmnw yatdh N
k a yy %"+ vMR Ny% ;..
' . p~% eg , 8%=y .
mq . @ A ij
?
4 > 7 wa NUREG-1021 9 of 2g Revision 8
ES-303 Operator Licensing Examination Report Form ES-303-1 U.S. Nuclear Regulatory Commission Operator Licensing Examination Report Applicant's'Name Docket Number 55- , I R Examination Type (Initial or Retake) Facility Namey , Reactor Operator . Hot Senior Reactor Operator (SR0) Instant 4.p6 ,y '7 A' SRO Upgrade
' Cold ct.: t y, s Fa... n 1:
BWR SDe5?ribtiong # SR0 Limited to Fuel Handling [ t PWRik lOE gf' n : g?
- Written Examination Summaryg;qs NRC Author / Reviewer Total)Ixamili$Ef$ihdint5 NRC Grader / Reviewer g$$1[ApplicantkjNtN Date Administered /p+y .c m. san -
~~'
~ Appl {~cantsGrade[v %
Operating Test: Summary. ; y Administered by ,f! f.[f.k f DateAddnistered . A. AdministrativeTopicq.fY /;hk j 6 k B. Cor!t/5I Systeikhihacilityh$Ik@dugh i
% _ s. spyu a C.
Integrate (Plant Operatyonsf(Sjmulator Operating Test) 4
%hd@h %. Examiner Recommendations meemtg wp CbeWBlocks@ogy;w ssE bfass Fail Waive Signature Date Wrij k Examination h 2pa p hI 0peratingTest ff hb) Recomendat{bli WMs$ffWggfig7 License Recommendation n- IINueLicense Signature - Supervisor Date Deny License NUREG-1021 10 of 29 Revision 8
I i ES-303 2 Form ES-303-1
\
Applicant Docket Number: 55-Page of A. Achinistrative Topics Evaluation Comment Page (S or U) , Number
- 1. Conduct of Operations g .n:-r e
- 2. Equipment Control
/d 3 % g
- 3. Radiation Control [F fY'% '
[
- 4. Emergency Plan n .< . ;- -
b w s.a ~,
.m 9. -
B.1 Control Room Systems Safety , %: ~ 4" JPN Grade Function
- 9, <
lKS or U)
,M
\
jf?f ' Question Grade '
,gs.pF+qi:.. ,D f, - (5 or U) x (fI
?
e,[jjk.~ <<v/
A: h .> hyr$j,i-(i
- 2. 4?4Gth
%da we < $$
w Ni$$5V
- 3. N $j![ > d kjih 4 :: N eir v ~..,+<>.1 ,n w%.a a
ve
*gg w%,
- 5. /f .-
f l 6kY ff at #
- ,; sun pw lB:2Sgac1]JthWalk Through
'4, ' R ,
4
-Od$E$;- ?:bY, 5;,:d
\
- 2. l l
- 3. l 1
NUREG-1021 11 of 29 i Revision 8 l l l
ES-303 3.a Form ES-303-1 Applicant Docket Number: 55 Page of C. Reactor Operator Integrated Plant Operations (Simulator Operating Test) Grading Summary Competencies / Rating Factors Weight 3.0 fcenariosh S Comment 2.0 1.0 Total '50bserved Page No.
- 1. Alarms / Annunciators
- a. Notice / Acknowledge g kli UM 3 0.30 0.90 0.60 O30) f ._ ; $_
- b. Interpret / Verify 0.40 1.20 0.80 dIO
~> n.,. v r ,.. $ JK j,, _ __.d'.i
- c. Prioritize 0.30 0.90 0.60 [0;30 _ _, t_; ;
.x Diagnosis w 2.
'([
t 1 2 3"
- a. Recognize 0.40 1.20 0.80 0.,40;..
- b. Use Reference Material
- c. Diagnose 0.20 0.60 0.40 [d520 ;.$ _._ _ _
0.40 1.20 0.800 70,40 z.>
- s. -
- 3. System Response
/ d h Y 2 3
- a. gb0 Locate / Interpret 0.33 0.67 %;53! ^q [.__ _ _ _
- b. System Operation Knowledge 1.00 0;67
- 0. 33.f 0.33j . . ,
, .e w _ ,__ _ _
- c. Effect of Actions ,0!33). 1.00 0.331, 10.67 a p jp 4 Procedures / Tech Specs jp .g
*;,f} 9e sw .u , e+-
1 2 3 a. b. Reference A kmji * !' 0. 20 ;0j40). *0.20 n;. . x.06.0l! Me _ _, Eop y y g mmediate. k tions' O.40 y20l 10;80 0.40
- r. - ~
- c. re Pr,ogdu,n@$epliance 0.60 0.40
..,,~.,x .i.(
3020 7 0.20 _ _ _,
- d. TechSpe(T.mtryk % 'gD.;20? 0.60 0.40 0.20 _ _ _ _
\
- 5. ControlBoahDper'ations, Edf 1 2 3
- a. Locate su b " " '" 44 ', 5 0.25 0.75 0.50 0.25 s - _ _._ _ ,_._
- b. 14anipulate o (,:-aq
- 0.25 0.75 0.50 0.25 _ _,__
c.f, Response [,, , ,;j O.25 0.75 0.50 0.25 _ __,, ._. j$.jfManualControl
/d 0.25 0.75 0.50 0.25 _ _
m _ _ DCommunicationsg[e. 1 2 3 ti a.m!@p$$NNNSIf
-r 65f
. e I~.w.mation r f rmation 5yr.1.00 0.33 0.33 ov.0.67 1.00 0.67 i.d.a..nf6r,ggy 0.33 _ _._ _ _
%gm .g 0.33 _ _
eMtihy#'Out Instructions 0.33 1.00 0.67 0.33 _ _ _ _ i NUREG-1021 12 of 29 Revision 8
_ES-303 3.b Form ES-303-1 Applicant Docket Number: 55-C. Page of Senior Reactor Operator Integrated Plant Operations (Simulator Operating Test) Grading Summary Competencies / Rating Factors Weight Scenarios 3 , Comment 3.0 2.0 1.0 Total s0bserved4 Page No.
- 1. Alarms / Annunciators 4 .
1 %
- a. Prioritize 0.30 0.90 f$h 7 3 0.60
- b. Interpret 0.30}"
0.35 1.05 0.70
- c. Verify 0.35 1.05 0.70 0l35 ,,_- ,__ l ;<__, _j f0/35 , ,_ u & _P
- 2. Diagnosis j;w
- .g 3
- a. Recognize . $~.s ,g 1 2'g3) f
- b. Accuracy 0.25 0.75 0.50 (.0.25;, Jg*[y 1A 7 0.25 0.75 0.50 '0j25]
c. d. Diagnose Crew Response 0.25 0.75 0.50 Oj25j ]*" __ . _ , _ 0.25 0.75 0.50 4 ;0.25' i _ _ _.
- 3. System Response ~Qa (3
- a. Interpret 0.35
,47" ,_ g; 41; <2 3 1.05j 10;70] '
- b. Attentive (0:35 f5' 0.20 0;60j *Di40j 70;20j P
- c. Plant Effects 0.45 j1:35 0.90? $ 452
- o 4.
a. Procedures Reference 0J5% [ 0.75j [ 10.50 hh 0 325 1 2 3
- b. Correct Use c.
1.50I 1.00 0150 Crew Implementation (A50FUi25 Op57 0.50 [0?25 tyg we C. Contro)(Board Operatiodpi. p.
- a. Locatesgg f % 1 2 3
- b. ManipuistC,h g%gu 30,25 40.25 1pg 0 s0:50 0.25 _ .__ _
0.75 0.50 0.25
- c. Response M $!h g@9 1025 _
0.75 0.50 0.25 d 4 Manual'EnI$h$h $5225s 3 0.75 0.50 0.25
~
m.:.
- 6. Communicationkft l ;.Y I*% r7'y
- a. Clarity ~" 1 2 3 T ' M r.~^ 0.45 1.35 0.90 0.45
- b. CrslInformed \',' " ,, 'pJ >0.35 1.05 0.70 0.35 c.,AMeeive InformatiokLd~',' O 20 0.60 0.40 0.20 7(
#,3DirectingOperation((
83.TimelyAction / 1 2 3 0.20 0.60 0.40 0.20 M15afe Directions @/ 0.40 1.20 0.80 0.40 fhr' Oversight,g[/ 0.20 0.60 0.40 0.20 96^3Cre2Teehac 0.20 0.60 0.40 0.20 bNM@d Specifications
- a. Recognize 1 2 3 0.40 1.20 0.80 0.40
- b. Locate 0.20 0.60 0.40 0.20
- c. Compliance 0.40 1.20 0.80 0.40 _ _ _ -
NUREG-1021 13 of 29 Revision B
ES-303 Operating Test Coments Form ES-303-2 ADDlicant Docket Number: 55- Page of Form ES-303-1 Coments Cross Reference A.:. ) s' j
'C'
.e .
5 y: p ~ 4 d? c
.~.
?w
,, ,> v ,....
a..:t., r
,.3 1 t ':
l
<i.,l. ( -.. J h-
> ...<4 Y
,/. .
/,
>^ 9 %
..t.,'
.A. s s
;kiQf :.:.
- 0%.
g. C '" k g?*[ ((
;?: '
kE!!.EM hk *I
).p}.i.) gi;
.-: -.. C $
a$.
';;,sm XCf DM
,($h$ 8 (4.n.h'
- P " #6':: ifj pkb#
3g~
- d. i , /. M. .ss, [:.T.;. 8..:8 si.>;.lf.g.g
.j$.. .. .. .,:j,h'
%, .i:
- ; g.
gf.3' t.
,Q'>, <
s!4 ' :r-/wj47
%< : . ..-;.v:s x s.
v w.wr A
,$7!flJ 64 p(. .: .;h riY{-[d!)g
+cs p.p. *
. . +. v s -s
<> n:- %es s.; , +...., r::.p: #4:.-
% ' ' ~::& fg g, .'<*ij $ [Jjf)) (g ,p.*
,Az: . :.
ga:9- (w~q+qi:. . e pehr:.; 8+. . % %,ggu. . wg.4.
- 9 g,:;fifh.+we+4Q '+"f ,].
49 x:*:: %sa ,...s. v- pgi Nr,iD9 4bD'e- 3/.!/ #..;}:h$'f5!:)p l' _ [JE,m x. a,;,m: j.E.
. i .<s. o,. v.
u . >$., m </ eA,.. Aw A37
- s
d$ .;SiMgM, Ai :.'3-vo.e r.:chy' M.) 9;' 1:T>ji@tJV
;:yg:::: ow +4;.gSs F
,.e NUREG-1021 14 of 29 Revision 8
F i 1
.ES-303 R0 Competency Grading Worksheet
)
I Form ES-303 3 for Integrated Plant Operations
- 1. N UNDERSTAND A'D INTERPRET ANNUNCIATORS AND ALARM SIGNALS DID THE. APPLICANT:
/>
(a) b f
' NOTICE and ACKNOWLEDGE alarms?
gQ jfF 3
*mkdm 2- $$
Consistent andL Minor difficulties f[$g f Failedeto N$$0[@b
#4fM% 8 g
timely noticejandjbg g ?" -j acknowledgement- orlapsesinawareness[@gg or response Wh acknowledgeimportantW$? Mii$rms: distracted $$$k ' i lj[$n6jsancealarms:etc.
-awgp A ff -G. > w.%
W '<p'C303A - -] (b). Correctly INTERPRET and s VERIFY'that'f$annunc14 II tors $g
;and. alarm signals were ,
consistentwithplantandsysteEcondit1disli5d$Idlik[Eheuseofalarm ! responseprocedures(ARPs)..delinecesdfy)? {f$$P 3 2 ^ 1 Consistent and 1nor inaccu Significant inaccur-efficient. .
;M!$nterpret}qq,.ng' actes resulted in 3
interpregig ' %$$hi rifyind"s'1gnals plant degradation: and I verification - - poor use of ARPs [.40 X =
] ,
. I$I (c) ATTEND to ANNUNCIATORS and ALARM SIGNALS in order of importance and l
-severity? ['
3' jf 2 1 Minor inaccuracies Did not prioritize afkhidA l and oversights attention to signals: a11' cases. { inattentive to ' important alarms [.30X = 3 1 i NUREG-1021 15 of 29 Revision 8 e b r
ES-303 2 Form ES-303-3 2. DIAGNOSE EVENTS AND CONDITIONS BASED ON SIGNALS AND READINGS DIO THE APPLICANT: si., (a) RECOGNIZE off-normal trends and status? y /,;' k 4 .d' 3 2 !
- ' ' 1
- v. ',
- - .h , a Quick and accurate Some delays in f.' f ~ Serious + omissions:s recognition recognizing off-s 4f?
normal conditions
//' '
delaysc or '~<, ,s
, ,,gp- ,
; inaccuracies in.:j, 'g
(' , ]',recogn1zingevents'Md> s , .
' ' [' 40 X
+
)
p . > j, h - p W, s *k s
,v >
's '< v,[
(b) Correct' .SE REFERENCE MATERIAL (prints y bookshcharts)'t'o' aid in diagnosing and cla.i.!ying events and condit,1ons? ,/j ,~>' us n,,a ,~~ ,m,- ,o I? y %; s r ,l s f 3 1 5 j j' }} 1 Correctly used MinorMlVorsiny' fy Did not use or references, using'or relying when necessary M incorrectly used
,pon?referencek'<f 4/M'M references to 4 l kk h 6 diagnose events
<'::l '* H,,g', ', <[N:[,]hhh.
'q , s oh 3 ':/s ,< A [.20 X =
)
4 1 * ",
%;% ,~ l,g:p,?,
n 1, : ', ,
~q')y:&G%y
- ,nw
,e \
i ph52$',~5l'fn \'/ I
\
(c) yCorrectly DIA30$Epfant y :::e y?,>> conditions based on control room indications? f , f3 y f:;'g<p! 2 1 l J Djagnoses were // Minor errors or
"# Faulty diagnoses nisidiEste ' difficulties adversely affected E $ in diagnoses plant status
[jjg - . r' [.40 X - 3 NUREG-1021 16 of 29 Revision 8
L ES-303 3 Form ES-303-3
- 3. UNDERSTAND PLANT AND SYSTEM RESPONSE DIO THE APPLICANT:
(a) LOCATE and correctly INTERPRET relevant instruments and other indicators of plant and system response (s)? ','
.,1 3 2 '
/ '1 "I Accurate and j ,./ * ~' , ' , . ' (' .
p Minor errors in e" Serious' omissions ' , efficient location locating and inter- jy and interpretation
/< , orinfccuracies' ' ~
s'
,, ,<e w preting instruments ' '
of instruments and displays # s, in interpreting'c +s
'w
~
' 2 ins'truments
' C '"
.g s .'#
L'33'X =
]
+sq ' ,g
> ~- >
,, n -
a (b) Demonstrate KNOWLEDGE of SYSTEM OPERATION.' including set p'o1nts. s ,
( >
interlocks. and automatic actions?4' hx
'L ,4
,,t
>f u
/fN v a f' 3 ;$ - . ,
.z,,'i P '
,, i, 1
f,?(.- Demonstrated
/$ l thorough under- MinorJnstances of ' / Inadequate knowledge errorscaused'tiyin- .d resulted in plant starding of g dadequateknokl, edge"'(" degradation system operations / t es! '
b , , >, m, <.
+ $', T's 3,"
L:P# f
%:<' c >:,%.
N ,< *, ,: -j's,<y Y [.33 X
;6i', ,,w,
*,'t:,,,,
=
]
N {' id:Mh ( [' ,',3r
. t %g:^[
b,v+^'%<<,:M^3, e**~ '; ' s s (c) Demonstrate an understanding of how his or her ACTIONS (or inaction) AFFECT 4 PLANT and SYSTEM CONDITIONS? (;*f5
$"3 A
2 k"l'!5 1 b' stood the Minor misunder- Appeared to act
% tof acti standing of effect without knowledge on'plantiPa W '// of actions on plant of or regard for sNteinsyfj"' and systems effect on picnt and systems
[.33 X =
]
1 1 NUREG-1021 17 of 29 Revision 8 >
{ ES-303 4 Form ES-303-3 4. COMPLY WITH AND USE PROCEDURES AND TECHNICAL SPECIFICATIONS DID THE APPLICANT: (a) REFER T0 the appropriate procedure in a timely manner?. AV 3 2 [1 [ Guickly. located t .A . i .. $V Minor. difficulties and gPfoblems)j@n)d4a11ures appropriate procedures oversights in referring gfin refertjngjfofh, jf to appropriate p proceduresMng.g.j%., procedures-g im ptant instay es] Q g '; y* k X =
$[
(b) RECOGNIZE E0P ENTRY CONDITIONS and carry out[hhf[oMjsfeymediate actions without the aid of references or other formstof assistance?s
-grA 3 2 d$$% klkh Consistent accurate Minor lapses,or" errors 1d (sAE4'W4 not' accurately and timely but actionsfjeneral}ff%s(iw[a$[ek$f0te actions recognition appropriate g7 M [.N "h;w
,g 40 X -
]
(c) COMPLY WITH procedures Ar$"W h Al f Y includingsrecautions};and limitations) in an accurate and timely manner?
, g /
3Ahfh k 7. <. C %
--I 2kh[g 1
%$ c !:. k.
Accurateand}.70s..hg Veiderrots: corrections Many significant timely compila$jj@M!Ikb< nce 33 ~ . .. m$ddsihif0fficient time to a~Ydid!Idverse effect errors: excessive as.cistance required y, [.20X -
]
-(d) . RECOGNIZE plant f eo itions that are addressed in technical specifications?
.kl 2
3 2 ff 1 Recogd12ed and,4dk Minor assistance Did not recognize lfidinithY$In required to recognize M'COs@i5y35 L conditions and/or conditions and/or comply with LCOs and stat'ements comply with LCOs and action statements action statements [.20 X - 3 NUREG-1021 18 of 29 Revicion 8
ES-303 5 Form ES-303-3
- 5. OPERATE THE CONTROL BOARD 0I0 THE APPLICANT:
- i. .,- (a) LOCATE CONTROLS efficiently and accurately?
l ,.s 3 2 g,/1 ;M.
<.. d5. . _ . s .< '
Promptly located Some minor hes1tancy appropriate controls 4 illnableit6h0cste and difficulty in '
#controlsMitti6udt 8 in all instances locating controls p[i assistanhs'k;g QV din [Nh4
%g#
[
$$h ...[4Ni X =
ti@dQj[ A [4#4q
' f DI 99 (b)
MANIPULATECONTROLSinanaccurateandtimelfmanh5EfA 3 2 @gWn D% ; N g. pen jEl Control manipulations n%
' Minor shortcomingsG:,F fpk Impr%,e:..m 7.J!15>op g manipul-were consistently j accurate and but efficientlyf@e4 J}$ ationsgesulted in j mitigatedanyff" g g s!!
timely resulting c66sequencds % hmajogsystem 9 ff & h'depturbations
&V Adr /f
\
ms f.;i?7 4 brv [.25 X -
]
n.1 (c) ACT appropriately in response to.INSTRUME READINGS?
\
fist 4W 3 i M:shM 2f$ 1
. SE h 'k$fANIkk N ~
Responseswere$jg?'pg appropriatetAnd timely
*QGenerp]yadequate re.sponsep3some Failed to react appropriately to Mjjfg^ error $iendjilapses instrument readings fpg
%y y y .
Nhy without assistance 4 ! x [.25 X -
]
h^
, Take MANUAL CONTRO automatic functions when appropriate?
((d J i l 13 2 1 Cannual c Minor delays and some Depended on automatic ;
. prompting necessary actions; had to be Whil$I MhjphijM @j @
before overriding prompted to take l automatic functions manual control [.25 X - 3 !
. NUREG-1021 19 of 29 Revision 8
/
IS-303 6 Form ES-303 3 6. COMMUNICATE AND INTERACT WITH OTHER CREW MEMBERS.
.DID THE APPLICANT:
'(a) PROVIDE clear and accurate-INFORMATION on' system status to others for6the
' performance of their jobs? [
gR ,g 3 2 At . 4 4N NlF a g i,s s w L QtfWh.x%
,n g ss &: .:. n Provided others' Minor. instances of gj%.f
.. f Failureftoiacc..~urately ge with accurate and needing to be j - -w
. pertinent information prompted for input: $lg% progde'importanj;gg%g informationtoothbrs yr p
some incomplete and
$5D inaccurate information ' j @]yE6pardized plant % F status pqueh
- <p' =
]
Ty33gS(*Dt
+qg (b) Effectively RECEIVE INFORMATION others (including requesting, hfrom;gibh !)As acknowledging, and attending to y anformation)?.',<. - 'A m/9 f/ fra g%. '^Jl p'7 P 3
er,)(2 w y)<W }.aW' 1 Responded and !)'0 fr! f0 M1 torg instances" // Inattentive to reacted appropriately of]f511ure t6 d 4
information
. to informatijn fromothers,N
#f^,ta do'rp,ckhowledge k provided by others ga'a,k N:' ~ t, ^ ~e.
l:eg,r,qb'pondtoQ[ g ! s% h nforestion
<LF ipr.g,3"g y Qp, Q!";s,'g [.33 X =
]
*q 4E!q3;+ YsL&k?h>. %if R:'ln(9 (c) CARRY OUT theiNSTRUCTIONS of the supervisor successfully?
ly! 3 -
'kbV W[q%t' 2 1 lg? /s$
iAb$f;carriedout /?" Minor hesitancy and Failed to promptly iWispervi
. M ues:sory h difficulty in following and accurately follow dise f orders. but ultimately directions: blindly bus 10d*6dWiMi$o'/ ns complied successfully complied with Niehpjd,$Mable d erroneous orders
['.33 X = 3 NUREG-1021 20 of 29 Revision 8 e
ES-303 SR0 Competency Grading Worksneets Form ES-303 4 for Integrated Plant Operations 1. UNDERSTAND AND INTERPRET ANNUNCIATORS AND ALARM SIGNALS DID THE APPLICANT:
,e (a)
N' OTICE and' ATTEND to annunciator and alarm signals s in order of their7 importance and severity? '
/ ,' n 6 s
W . 3 2 [/:7' ' s '1 " Responded accurately Minor difficulties
;;? , f, , , f,.
and efficiently in l^ Failed to atterid to , inattendingtosignals,$'4, or,prioritize 1'porth
/d all instances m Op orprioritiz1ngattentionl:,"hahtalarms: respondeda N 3 slowly: distracted by 4 O nuisance alarms b T/ ,.,'_,
,,,.. [.30,X'J' -
]
p[m , ,s 3 'g (b) Correctly INTERPRET the meaning and(s'ign' tfica'n8e of alarms'an'd annunciators (including the use ofsalarm t'esponse',proceddr'es (ARPs), when necessary)?- ,- / Nj','S 'y
,,? g 3
g2 4+' /nL8 f + 1
,i ~,'W h*
us J1 sl, Understood and Mino(inaccuracies A Quickly determined Misinterpretations. for[ delays ir((Ry [#( delays, or misuse of ff"1'nterpretingalarus ARPs resulted in wereind}catin(p),6what s>,,- +, e u, " r, failurestalarms N* * '* " plant g%dl'Q degradation
,W 4,s *8' 's q yn, ,
%;"u '<!V%% NR sp.
[.35 X -
] '
}lqh'Q*q,, %{; W, >
(c) VERIFJ;'that, annunciator and: alarm signals were consistent with plant and l,4 system condithns7f'Q*Y"
/s" pt jt"jf3 %':Y fde 2 1 Ensured proper r Minor lapses in Failed to correctly '
pf(cationwhen alarm verification, verify signals on gecessary e but no inappropriate j^2 important occasions: actions were taken as $9G
%^ h j,%jp'#;V a result of inadequate did not notice incon-sistencies between verification alarms and plant a
conditions- [.35 X -
]
NUREG-1021- 21 of 29 Revision 8 l
m ES 303 I 2 Form ES-303-4 l i 2.- i DIAGNOSE EVENTS AND CONDITIONS BASED ON SIGNALS AND READINGS OID THE APPLICANT: (a) RECOGNIZE off-normal trends and status? , m A Af 3 2 f .,g g a Quick and accurate. Some delays in dSerious;:omiss$# ions, recognition recognizing off- [ delays,jE$jh,
' normal conditions nr[ inaccuracies? tin! A fr 9
$@ recognizingYMNd$%3,gh" fg ydegt$yf5X - "$% 'F; NI)d@
(b) $% Ensure the collection of CORRECT. ACCURATE. a dlCOMPLETE information and reference material on which to base *4 diagnosest ' Q Mj.}g,k Q-e4p 3 2- gy% g:Thd$,h se as ue Ensured that all Minorinstanced[$E$d:"mY N Serioud Ynstances o relevant indications looking,ov,epslying E$ ^ and references were or, misinterpreting)ndic NII Jh important ofuse susj6g or failing checked ations rd/or references
~
a '".Ninformationordata N%jfjf,j y .[25 X -
]
(c) . Correctly DIAGNOSE Av conditicasibasedien control room indications? 3 h x 2 T!N!A DiagnosesofQilg Minopsertops or 1 Faulty diagnoses conditions werej M % difffisitjesin adversely aff(cted accurate - plant statue diagnoggg' conditions
' [.25 X -
]
r (d) Ensure that CORRECTQ nd TIMELY DIAGNOSTIC ACTIVITIES.were carried out by the
~
CREW?- 3
~
2 1 I5 Minor errors or Faulty diagnostic d14gid5@ilMffvities difficulties in- activities by crew and~ diagnoses by crew diagnosing by crew adversely affected i plant ctatus [.25 X =
]
NUREG-1021 22 of 29 ' Revision 8 a
1
~ES-303 3
Form ES-303-4 3.
. UNDERSTAND PLANT AND SYSTEM RESPONSE DID THE APPLICANT:
-(a) . INTERPRET control room indicators' correctly and efficiently to ascertain and.
verify the status'and operation of plant systems?
?
, JF g g%..#a nc 3 ,
2 g Accurate and efficient interpretation'of-Minor errors in y{g gV 3{:3 Serious
- g j%
interpreting de1[d}s.cmissions, gjF instruments and-
. displays
. instruments and-,
4 y@h ,,acies in interpreting]y"o Nin displays
%ynj'instrumentsandQiiW
;%g 'idisolays 4
FIN hh!kfIh - ) (b)- . .um*. V D@A M Remain ATTENTIVE to control room m indications k @!@w s s' w Regularly scanned
,fp@w waqhW g SporadicaIIyscanned
[y Rarely scanned indi-indications: antic 1- indicatiys:minoV pated changes in plant # ,' -cations; failed to lapsesdn anticipating anticipate predict-conditions due to: predic' table ctianges able changes in plant events in gress
;j{f..
"J~
status 4 [.20 X -
)
(c)
@h ' nd$h, Demonsthte$3hrough trect$W53and actions, a thorough UNDERSTANDING of how the PLANYN$iSTEMS, anfCOMPONENTS OPERATE AND INTERACT set (including po IIli $iidbnd ad^Miiatic t actions)?
w / Deeptstrated thorough -
\
Minor errors because of Inadequate knowledge Mstandingofhow gaps in knowledge of p tams.and componeny/ s how systems'and f of system and com-ponent operation
, and in components operate'
- resulted in serious e . mistakes or plant degradation
[.45X -
)
' NUREG-1021 23 of 29 Revision 8
g. ES 303- 4 Form ES--303-4
- 4. COMPLIANCE WITH AND USE OF PROCEDURES 1
DID THE APPLICANT: (a)- REFER to correct procedures and procedural steps when appropriate? g 3 2 z lS[i+ M& [ Requested or readily das 4!:. . . :' L., Minor lapses in . located all appro- referring to or gpfailedto.!;corgectly p priate_ procedures gp referJojprocequresp jy locating appropriate pg as necessary procedures p& in inportan61gggy d3,.$g$#p instances %g%$ lS$@pI NNF
%g(072.5 X -
]
qu$QfMf%s%
%an%
(b) USEPROCEDURES' CORRECTLY.includingfollowingproce3Ma$$Nhs1ncorrect 3 sequence,abidingbyproceduralcautionsiind!11mitatidhiHSelEctingcorrect
'pathsondecisionsblocks.'andcorredj}[25h$$tioningbiGeenprocedures?
3: qff RQ - Y 2 as$, Ni 1 4f 4 r. I c' Accurately and Minor g,./F rs. b ,7 Significant errors promptly executed madpJ[necessar.
, impeded or slowed procedural steps cogections recovery or degraded tylyfa plant unnecessarily .
'~ '
. gj$g ..
[.50 X -
]
h (c) Ensure !the; $$$..effi$teid 4.s . y-MENTATION of procedures BY THE CREW?
- 3. .
2 1 Kept c/hw informed of , occasionally had Read procedures to procf0dralstatus: gotit " question SRO ackiistledgment from him/herself: failed l# regarding status: to coordinat'e or
.CMwhenreading J- allowed lapses in verify crew's use of ures -- . implementation by crew procedures
[.25 X -
]
NUREG-1021 24 of 29 Revision 8 8
o
'ES-303 5
Form ES-303-4
- 5. OPERATE THE CONTROL BOARDS ,
DID THE APPLICANT: (a) LOCATE CONTROLS efficiently and accurately? , 3 2 fr. L sk. f Promptly located i 4 u, d om.- IEP Some minor hesitancy or jgpUnabletoQo%e g cate 3 appropriate controls difficulty in locating Q+ contro}spwithouty in all instances controls assistance qu
' gg g s Jf p[#
s
^ijeLi@g z :,'~[225 <
X
=
'Mi]$F (b) MANIPULATE CONTROLS in an accurate and timely man >,<ner? -
+
3 ('fs "' ' ,7 " J ,
- 2 N , 1 <,,
,~;, , , , , . , g, ,7 Manipulations were Minor shortcomings.s
', , a, Improper; manipu-consistently accurate but any res.ulting -
and timely , wy,,',',),,.a>lati.ons ~ caused major consequences were A e
, system perturbations readilyskitigated
f 4,};h,2.d' 4: .' #"; + a,:7 [.25X -
]
$>~ Ni' (c) ACT appropriately in res/ 'f
ponse to INSTRUMENT 4EADINGS? kn;; 3 /k- /s'd'p?3 2 l :'"; i,%^MR'f P s/::nu c/A, 4>>;-[?,, GF 1 Responses 'were:v.v,'h NGeterally responsive. Failed to react appropriate' add , E'+q but some'innor timely bl ^ ':,b appropriately to erros and lapses instrument readings [ [4hs,;b,4 Y '
,t without assistante f@f')
d ';f t / [.25 X =
]
h iY[ (dl,J Take MANUAL C0t{m0L of automatic functions when appropriate? b :, f/ [$43 // 2 W1 ual Took l man % pcap $ 1 trol Minor delays: some Depended on automatic as g upJ ate prompting necessary actions: required before overriding prompting to take automatic functions manual control [.25X =
]
NUREG-1021 25 of 29 Revision 8
L 1 l l , ES-303 6 Form ES-303-4
- 6. C0HMUNICATE AND INTERACT WITH TH'E CREW AND OTHER PERSONNEL
'DID:THE' APPLICANT:
')
(a) Comunicate in 'a clear, easily-understood manner? ' y,. . s Af 3 2 ?ik i
}hw W Comunications were- At times, comuni-- g fbo m u[Ni$k fwere- g timely; clear / and. cations were . y' easy;to' hear.and. " confusing, hard to
$[f{if ordifficulbt ill-timedM55dilebh g,.
understand. hear or understand TA t ~ NksNb Nhh%[r$s
%d -
x ] (b) /%dMSA Keep crew members and those outside the control roomjinformed of plant status? g{ N @ i 3- 2 .jgMk . 47 - B&f Provided others with Had to be prompted for % ...p}}d? .Ntoprovide accurate.. pertinent information in somerininor sportant information information throughout instandei[gaveibi5e scenarios inc " ~' e or Niiccurate 1 Tnation
~
.k hk; .
*hw
-(c) ENSE MCE1PT of ea aily-understood communications from crew and
.Others. } k@!@$$$4 3 3 3
h Reque information 'ed to require Failed to request or_ Qarification when acknowledge infor- needed information: necessary: understood mation from others inattentive when. causin1 Cations from y information was j dthiiF5 ? provided: failed to
; &[ph, gy$[ correct serious-f A MJ# misunderstandings t@%;(:h'$F among crew members
[.20 X =
]
NUREG-1021 z 26 of 29 Revision 8
ES-303 7 Form ES-303-4
- 7. DIRECT SHIFT OPERATIONS DID THE APPLICANT:
(a) Take TIMELY and DECISIVE ACTION when problems arose? 3 2 1 Took early remedial' Minor instances of action when necessary
. Failed toitake thndly failure to take action . action:gresulted in within a reasonable ";detertorationfof period of time - fplant conditionsg
- p. g ,, y gs (b)
[.20)? 'Q{ Q. , P Provide TIMELY. WELL THOUGHT OUT DIRECTIONS!$$at faciiStated CREhPIR#U anddemonstratedappropriateCONCERNforthi;SAfET6.Eftheplant.stOf@$d public? ' 3 2 O % g7 1 Directives enabled Minor instances of m safe. integrated Di}eEthAHinhibited crew performance incorrect.trivialhg e i safe'pst formance: or difficult-tokarry @ ' g.. crew hadito explain out orders g " ~ i . *fwhyy . orders could not MP ME Y ^ iib d b0uld not be [ [ %jfollowed
,bhi [ gh? C *0 X ~
3 (c) Stay in a position of 0\fERSIGHT andjprovideAn' APPROPRIATE AMOUNT of DIRECTION and Gul0ANCE? F s 2 W? 'M%&&p 3 2 4 AM? 2 pyp 1 A-Stayed involvA.ds :ibut 799 A7pECr r$@%occasionlikhadto lost the' big picture; MequestMssistance, crew had to repeat-notintrusiveA@,2*L anticipated'cre seph wh(chliintbnfered edly request or ded provide guidance: needs and guidancewhenf prov}jk;jg/h g carryyt' actionswithdheifsbilityto failed to verify that necessap/
*^
k $alh directives were
. p. A ' correctly implemented
([A e e_n [ [.20X -
]
(di$$ oriented SOLICIT and INCORPORATE FEEDBACK from the crew to foster an effe approach to problem solving and decision making? A fM jf 2 1
}}i h h N Sfi F phobljs@}yhig process At times. failed to Made decisions with-involve crew in decision out crew participa-as appropriate. leading making when it would to effective team tion or consultation:
have been appropriate, crew divisiveness was decision making detracting from team- counter productive oriented approach [.20 X - 3 NUREG-1021 27 of 29 Revision 8 1 I 1
ES-303 8 Form ES-303-4
- 8. COMPLY WITH AND USE TECHNICAL SPECIFICATIONS DID THE APPLICANT:
(a)' RECOGNIZE when conditions 'were covered by technical specifications (JS)?
,c ~ >;
h 3 2 ll1 /+:, Recognized TS A /L , ' ' , _ ' Minor errors and I';, Failed to correctly g limiting conditions for misunderstandings j; recognize situations g?- operation and action with respect to jJ covered by T5 and t 'C', ,jf statements without use TS applications r,<;L of references action statements ' W ] 7 jy, '"^t2f ,
'm;,, f' ,.
A?Q s
,, r
- %< T.40 'X
+s, ' ; ,,
]
+a - ,t (b) LOCATE the appropriate TS quickly and efficiently? N' e g ,g,,, \ g <',t ib' 3 2 Mcf ,^>'lik by e
'p1 a
b k . ,/ ~'~'s
< ry,\ v,, ,,A, Located applicable Had difficuit) locating <" c'CouM not locate TS quickly and TS: had, tdsearchg 'p:ipropriateTS accurately through 'ihdex and body 4 of doediliieht fiI !"J l '0'
/'%lq>f' jh '
d
; ':<yg[
d,$f [.20 X -
]
(c) En/,bi % sure correct C0 % 1ANCE*with TS % i LCO action statements?
' g'O T & ,Tx
*i *' t ':, ','% ,%, gyrg gn?W'sld;;%'> %
3 h,og y , y ~; r, o -9 wp 2s, 1
#%D%g + ;bf
. Directivesjwere485ad'tp'f'Qeeded'50:0 assistance Applied incorrect TS oncorr,ect-Onder- %'frlgfhomcrewtoensure to situation: standing of TS * 'c!caiiI allowed crew to e action, statements 2 gV
~ $liance violate TS
#1 jf y
j f~ [.40 X ]
;,;~],,
'y g f<
,:' ' t'? l w&
\il0?'~'?+
' n , ';r*
NN-NUREG-1021 28 of 29 Revision 8
ES-401 PREPARING INITIAL SITE-SPECIFIC WRITTEN EXAMINATIONS
~ A.- PURPOSE.
1 This standard specifies the requirements, procedures, ~and guidelines for preparing site-specific written examinations for the initial licensing of reactor operator (RO) and senior reactor operator af5/ [(SRO)' applicants at power reactor facilities. 1 n B. ' BACKGROUND - [' O"
<p+ :',;:qo,
., ,s, The content of the' written licensing examinations for ROs, SROs is 55.41 and 55.43, respectively. Each examination shallentative: contain a repres,dctated M0 selection questions conceming the knowledge, skills, and abilitielik/As) need$d to peff55Gutiis'st.the
- desired license level. #
wc%,s Sq,7 The written operator licensing examination is administered.in two. sections, including a generic fundamentals examination (GFE) and a site-specific egn$d6N SFE covers those K/As that do not vary significantly among reactors of the same type (i.'e@a;p ES-205 for a
.description and is ofgenerallythe program). administered The instructions i early istandardinapply the license traimng process'(
examination.- ' Except as noted in Section D.1.b, the "Knowledgean . , . og[s] for Nuclear Power Plant Operators: Pressurized [and Boil 0ig) WatfReacton NUREG-1122 and -1123, respectively, provide the basis for developing content-valid @_ censing exa stem statement has been linked tdWapplic5l$e item nuEber in 10 CFR 55.41 and/or 55.43. Preparing the license examinatiNEsing the%ropriatdN/A catalog, in conjunction with the instructions in,this NUREG5 mill ensure that the'exarfunation includes a representative sample
, of theitemi ed in tog' u lat ns. \@]!'
C. RESPONSIBILITIESb':
- I%-
%e, < w
-1. Facility I%nm% %y:y.y's 'EfJ>fe 4
~
gmye s c yh /
, facility lic rMs[hsible for the following activities, as applicable, depending on the examiriiIl6iferrangements confirmed with the NRC regional office (in accordance
'date: withy #1) approximately four months before the scheduled examination
- a. Prep andj,aMthe proposed examination outline (s) in accordance with ubmit the outline (s) to the NRC regional office for review and approval in
? "rdance with ES-201.
3
. Submit the reference materials necessary for the NRC regional office to prepare and/or validate the requested examination (s) (refer to ES-201, Attachment 2).
- c. Prepare the proposed examination (s) in accordance with Sections D.2 through NUREG-1021 1 of 41 Revision 8
r: l ES-401 D.4, review the examination (s) in accordance with Section E, and submit the examination (s) to the NRC regional office in accordance with ES-201,
- d. Meet with the NRC in the regional office or at the facility, when and as necessary, to review the proposed examination (s) and discuss potential changes (refer to ES-201).
gP g.48 li
- e. . Revise the proposed examination outline (s) and exam' hp)f$b agreed upon with the NRC regional office; however, the NR6retainif6Al[ssthority to approve,
^
i the' examination. ~ Ri Q*i$%.; 4. .,:i.8%.,
- 2. NRC Reaional Office
$W'% 1 A
3.,.
% T
% .s.
NfFt7 :. , The NRC regional office is responsible for the folid$$$.hfg sectivities: g 3+%
- a. Ensure that the examinations are prepared in accon$ance with Section D.
N3 o . A
- b. Ensure that the examinations are revninied for quality Wac2x3fdance with Section E-
;pDd w'":Ib%,
, ,< <~
W y
/,',>
f:t /.
- c. Meet with the facility license /',M'e when and as: t n *uappropnate, to prereview the N^
D. examination EXAMINATION PREPARATION pf (s)in j/ accordarice withes-201. f~.F
- 1. Develop the Outhne -
,,d,,,,<a,
'each ation outline in accordance with the following general
- a. ' ! Nil t; approp$@sd5! hA
$c((h
" $$feioped (i.e3)Mination outline model for the licensing ES-403 ES-401-5, " Generic Knowledge and Abilities Outline," applies to all exa
- b. .Syste_. . .,. , select-eddettenel K/As from NUftEG11122.:(for PWRs)pG123 4
(fof BWRi)]to complete each of the three tiers of the examination outline.?iin.
. order:to[ maintain examhation, ccmsistencytthe; facility licensee'sjsite-specific;K/A P. , list:s@lipot beyse(Mylace;(theR6pataloggAttachmentgprovidesl art
.. .;ofgaoceptable;methodologfjotselecting1100NAspitl*1.the; defined structurofmexamination:outlinerothenn=thodoionie.s;.may:be:used provided they3reyeproduciblajand; scrutable andyield an examinationputiine. that ls; free, i DI.. bias landMheresjolthe; applicable)xamination:.modelsThejexamina. tion mutWnayMyequested;to explantoi; thy:NRC[ chief; examiner;the)ystema. tic MNMwa*lusedgdwetop;nwjxa!ninatio!1gtiinese::::', =g :'4^ :
NUREG-1021 2 of 41 Revision 8
y 1: L ES-401 frem v.tkh % g;,ner;% 1^^, en; pebt, rauhlpk cheb; qu;dkn; d Distributed the,K/As among the three tiers as specified for the applicable outline, select topics from as many different systems and evolutions as possible, and distribute the topics among the K/A categories (including the generic K/As, as applicable to the system), with at least one topic from each category in ; veri ikrapplicableLio Tiers 1 and 2 of the outline. ForTierpof thefoutlineteach;categorfahal.l. sample
.a. .t...l.e. .as...t.2. to. p.ic. .s7 Avoid selecting more than two or three.t
. . ~
.,ics
.m from a given j system or evolution unless they are related to a plant- priority. The genericy/&categqryll.egitemsfrompetiokof the_aFM5hbiA::K/A catalog)is ;
- l. i.n. .c. c.lu.d..e. .d.e:in t,he.s; Tier;1l(E/ APES) and Tier:2 [pliint system } ~~
butjonlithosej.topi.cs thatiam. .relevant
.s s m 4s s .w - sv. . .osu~...
s. to'tk,
. . - . . . . ; evolution applicable a
or system' shill s, "s,l
.be.incipdedjn;thelsample for these;tiersi . i 7 9; E 7^/ i w: e Q weA M7 Enter the K/A numbers, a brief description ofloach topic, the topics' importance l ratings for the license level of the exam, arYdhiihblittotals (system, category,
! group, and tier) on the ekaminationjoutlirE F$arti60lar attention to the ! importance of the selected A/As. Those"below 2YishEi3d be justified on the , I basis of plant specific priorities. l1."^D TM:0 "d38idh Ab; nem ! tche,r th; ;;bd;d l'/A; ;;"'g3.j. h;;:n [$: ;nd k;;-th;a-
;;mp:%M ;;;'ien%, J.tdher...;j ;;;;; .7.ht; %dh; j;b b=l t-b;
" " ~ ^ * " - "'-"-
l exambed, ;nd v.tcher th;!Ilp;;'% wwW L _ .
. ,,_, ,_ ._._. m. -. m. ,. ._ _ _ o m _ _ M x v....
Ensure that the ou dd succe e audit and licensing examinations does not becomegatitive ariifbredictabNI. If a facility licensee proposes to use an outline th -.used_-stMther facility, it shall identify the source
; of the~ outline.at :andgexplain was?previou JNs reuse is expected to have on
) skaminatl6fi .
NW iMg l l c. r:systemati .. ing100.K/As forthe examinationithe' examination lice Yith8fTiiay identif%iiflo~ie~~rrahdiiional K/ s$nsed fs~p~e6ifl5Hiklist"ciMhla5t-specitic[hihpricIrity'Nemelopical( '," operatingon the fa$l even@6ffjiiiildems, PRA-identified risk-important systems and operator actions , and recMSNIi$llogical developments) that ;h;;'d b; 2d;d b ;;% appropriate I for;testindIIIl$iIwritten examination. Enter the applicable information (FdA 37;nd lrr.per%n;; TC.ns) ln the ;pprepri% tbr (sen;ik, p%d
- nu.nL57,[d
;j;% n;,.;n erne.,7;;.n;j/;tner a;; pbM :nLiba;) ;f the ;,x;.ril.ndbn
,. M
'Q [; h u? W M:7 Chapter,13 of NUREG-1560," Individual Plant Examination Progra ReactorSafety and Plant Performance," identifies a number of risk important human actions that may be appropriate for examination. in determining important operator actions, do not overlook actions that are relied upon or result in specific events being driven to low risk contribution. This will help identify those human actions, assumed to be very reliable, that
. might otherwise not show up in a list of risk-dominant actions.
NUREG-1021 3 of 41 Revision B
ES-401 lf ; pb i ps:'; %pb des net ';; eny ;f te %%d ;y O.T.. Or ;sh.;b . , s%r lt in ; ibak ; pace provided at the bottom of Tier 2L.Grouplof the examipation outline;(i.et,' the;.systemtepk, tan indication of.which systematically. selected K/A the plant-specific priority, topic.will replace?a.brief explana! ion.for; making the , 'substitutioniand the? proposed numberLoffexamination; points applicable to the l 10RC))I . gIII n; te.:.:y ::ssa'; ;:3 ;ps:... l N> h
,h ._,e om _ ..
..m .... .. .
_.__.m_ ____m____m._ __,_ m m l
, . . . . - . , _- - ..., _ ..... . _ u.m
..o l
-::::::;;. ln erd.; % .T.;:ntin +;-T.h.;;b p 7. ._. .. . , .m..; boxy,0; :" :;::l' "" " 'y?
i
;t.;l ..;; b; :::d n ;!::: ;f 1; a;.r;(Q . 6F "d3:::!:;f l.
%M$b~d*Wdd
%?
l
- d. [ After completing the outline, check the sesN:ted K/As for balance 6f -covenige <
i within and across the three tiers. ;Epsure ihme.at every; applicable WA~ category;is sampled at least once within each of the thiEtIeFand that the outline for an SRO.; examination adequately samples.%d%gfodos.!requiredjbyMCFR 5533(bl@ake.MadNstmentsjhat frught. befecessary by;systematicaly selectmg replacement KIAs;rAlso chsiIdithe overalliialisiiieI6f the entire
~
iicensing examinit[on', inEludin. ki,vek4biough ww and% dynamic simulator test, and make any necessary a ntaa "w a
# . n
- e. Review and submit the ogmpletedjlutline to _
chief examiner for review ) and approvalin accordsInce witt)RS-201. Facility-developed outlines shall be independently revi manageri;n : 2,h: 2 f:ind aggr6ved by^a knowledgeable facility; regional office in:accordance $0h;ES201. Facility licenseesjare respons Aftensuring'thaTEMor-M oullines niesithkguidelines herein. The NRC ad3Enudrec n- - m. IIJO6Es..,i.stlines mg m by'i6infate agreed'upon'when the examination
^ 4narrangements were:een. w ~
firmed (normally at least 60 days before the scheduled
^+ i..x , MM. .%.
qu 1
- f. .g ,.
exa , will review the outline within five working days and
- provid_e~ %W: M+44.
and recommended changes, as appropriate.bThe' chief
..w.....
s
- 2. Seled and Dev Ch==tions k
" a.
M-Preparathe site-specific written operator licensing examination using a g pg con 6iriation of existing, modified, and new questions in accordance with the j khhg )# criteria summarized below.%g*hq @sh approved examina ) i 1 if it becomes necessary to deviate from the previously approved examination ! outline, discuss the proposed deviations with the NRC chief examiner and obtain I 1 concurrence. Se prepared to explain why the original proposal could not be 1 NUREG-1021 4 of 41 Revision 8 1
n ES-401 implemented and wny the proposed replacement is considered an acceptable substitute.
- b. Take care to ensure that each question is technically accurate and free of the following psychometric flaws that could diminish the validity of the examination:
lowlevel of knowledge . gi low operational validity 4 d[@( low discriminatory validity implausible otstractors fp 4([dpgl, confusing or ambiguous lar.guage@47 g*G^g)A gJ+ [4 confusing or inappropriate negatises collection of true/ false statementii %di# [MF C%, backward logic- t[; '#
" wg WM n Appendix B provides a detailed discussion,:and examples of questions containing each of these errors. Appendices A arid B contsismof@etailed instructions and guidelines for preparing and formattsig conton'thalid examinations and should be referred to as necessarj%w$^ . ating tS@,ination.
p
- c. Ensers that:the"qGestionsM3ifferenilets n"c6fnpetentfand less-thans -
competent!appucentshthathstare;a and thadheir' operational"M[TiprtyriiliElifiie}ob.[ isjsulta613Meferjo Appendix Blevel . for being examin a
' dditional: diar =*-FoHhese.pri6biplesQ !
Y ;
. Establish a level culty .discri Medg _.:between; applicants who have:and have X eZskillsland abilitiesHSee:Apperidices l wA and B;for~fsir~tner _..i. onssettEs individual test.. question. level of dimcutty; Og '
. _.be a. ble_Wc. ompla.te:andfeviewgexamination within five
. ~ .
f .% @[ sop 6$tigm37,5fgran%%p%
% J n]u n er a g g g a g ,n g ; g ;r.; 2 9 g ,s p g ,g, ;f s g " ; E ,
-m n s F m _",w%e _v _ _R _ _ _, _ _ - " G = L G e e -- 6 , " ; "
ws ww 7ws wwu st wu gs we swa ,
./'%mmM_r_ . . .
?..g in order 1 examination quality and consistency, et4eest-between:50 and.60 peMc(. I. ..Ef the questions on the examination shall be written at the
~
7 comprehdilision/ analysis level. Tacility;li.censees mayfat:their; discretion, A[jf Q incMhe:* action)f higherleVel.!questionsibut theyRC will notJequire.:more
$$k pq thanskemony The cognitive level of any question drawn directly from a bank willWoounted at its face value.
t j. pFWhen both RO and SRO examinations are to be given at the same time, duplicate no more than 75 percent of the RO examination questions on the SRO examination. The remaining SRO questions shall evaluate the additional knowledge and nbEties required for the higher license levelMper;10 CFR 55.43(b) pre.-:i; ;;dir.s s :t.; ;dd;;;er.;l :ep n t; b; sr.dited f;r :t.; O'10 NUREG-1021 5 of 41 Revision 8
ES-401 examineteen
- e. . All test questions shall be in the multiple choice format described in Appendix B, and each question shall be worth one point.
A
- f. To avoid compromising the integrity and security of the gamination and to enhance consistency, observe the fonowing limits on question repetition and bank use when preparing the examination: 4 d/
Jbh$,h W h@hg Repeat no more than 25 percent of M questions;.ois tisE esamination [ from examinations, quizzes, or teMadminista$iii to'tSElicoide gF applicants during their license training classYfrom the$asitAlidenhng r examinations at the facility. The foollity4teisOquiz limit doesii6ytly to NRC-developed examinations because.1 hose materials are generally not available to NRC examiners. jiiQn[% O kNSEhW Facility-written examinations shall repeat no; questions directly from the applicants' audit examination $examinatio'6MiGI6ase of retake applicants);or similar tea %%yverisiFryr"N"end of's license training classMiess the two examinations are written independently (i.e./pdinterfaditseihiNNnM5kamination authors). In such cases, up toy $e quesd5ns mayhiE%licated, ^ and the facility licensee sha tify tt di0plicate I ercent of1he questions for the examination Take dire no"the more facii@than 50p$ licensee @ written examination questio wit ificant rM# Questions that the facility licensee has otsisinIdi . anotheMiiinjiand deposited in its own bank may be m tre .
" questions provided they have an equal chance of being electe'd sliiish the examination.
at less 40 new questions at the comprehension and analysis level, niaisiiribed iri Appendix B. Questions from another bank may be thmetedginew items if they have not been made available for review and g.f stessygy the license applicants and there is no basis (e.g., historical - Y p th,hIcedent e applicants toor reciprocal predict arrangements their use on the examination. with the other facility lic i Select the remaining questions for the examination from the facility j licensee's bank, but significantly modify each question by changing the ! conditions in the stem and at least one distractor. The intent or objective ! of the _ question rhos not necessarily have to be changed.
- g. C ;;ri ; z :S x m ;;t M ;; f:::d u d r.; ;h; f:1:;j l;nr.;n'; r;f;rna m;;;;i;; l Fin:d:d ir. :::::2 ..s ;M5 CO 20h-A technical referenc;, lr. dad r.; :he ;
NUREC M21 6 of 41 Revision 8 ]
[- . ES-401 l 'ninl ,g djn;;;; ;;;d ln th; opp lleen;;' ti;lnlng pregi; a, and a cross-reference l to the facility licensee's examination question bank, if applicable, shall be noted for every question. )lf the: facility licensee 1has a,leaming objective applicable lto l the._ question 7it..should be referencedfas;wel!K.HoweverAthe_ absence.of a leaming objective'doesLnotLinvalidate the.'questioriprovided it has;an~ app'ropriate K/A and techriical referencet Refer to ES-201 for additional instructidiis regarding the' documentation of the source of questions on facility-written examinations, p ' 'Jd' [%<';X To facilitate the review process, f examination'g> authors should consider providingi
. brief explanation of why the answer is corfiiict, and eadIof ths" distr 58 tors isAi#
plausible but incorrect. This op#onalprEsice incr$iks the ek%%fM examination review process and promotEdie defi5ction and correitibriF problem questions before the examinatioM$$Mrninistered. dh$$% Reference materials such as diagrams,ysketches and.pogions of facility procedures may be used on a selective basis as"[ attachments to the written examination. Ensure that any referedinimaterial udsdMWexamination is easy to read and clearly marked, providesierfiSictive and'$th56tive way for the applicant to demonstrate knouldAiihe'of tisii away the answers to otheri$estionsihitis$~or " t on. con' cept, and does no jf' f
- 3. Review and Submit the Examination P 1
gf$5W
- a. Faview the entire.mzaminati ensure: t the criteria on Form ES-401-6,
" Written Examinaitlin Qualit[MiurandETQA] Checklist," are satisfied.
bk Forward b.'kM% Ah?htheexamihation Vk packaie;hY including all proposed attachments, and the Nik cornpleted QEchecidist to the first reviewer. Section E provides instructions for kooridiicting the'QXO.
~
f
'%$$ %$$P ed$Minations must be reviewed and approved by a jnowled" supervisorfor;manageriCx;;d ';e:. j repin;;;;; ;; before they are'iiiIsiliiijhe NRC regional office in accordance"with'ES-201. Facility authors sliiNE06mit their examinations IofmanagemenUeview'in time to support tlisiir Edelivery to the NRC regional office at least 30 days before the l
x scheduldif'ewommetierHowiew::date.
.y $- N miners shall submit their examinations to the chief examiner for review k ast one week before the scheduled prereview by the facility licensee (refer to
- 4. Assemble the Examinations l_
i
- a. Format the examinations using the one-question-per-page layout specified in NUREG-1021 7 of 41 Revision 8 l
t
l I ES-401 Appendix B or by placing as many complete questions as possible on each page.
- b. Use a cover sheet in the format shown in Form ES-401-7 for all RO and SRO written examinations. Fill out allitems in the upper section of the cover sheet, except the name of the applicant, when preparing the exa,minations7 E. QUALITY ASSURANCE REVIEWS lb. [
f # 9 >< < r - > p When reviewing questions, reviewers should try to put themselves in the positiongfjhe f47 applicants by attempting to answer the questions withouf using reference matenal or referring;to the answer key. Reviewers should ensure that the conilitions andWkuiremordjdsedWiis question are complete and unambiguous, all necessary'M*n is provided, alli 'ssary information is deleted, the intended answer clearly followsfremshat is asked in the question, and the distractors are plausible.
*[8hWIlh
~
- 1. Facihty Management Review if the examination was prepared by the , it shallbe' independently reviewed and approved by a;knowledgInnEle'sti$$$i5r or manAber s c en outheneed fa-:1;j r;pr::::; .:J; (;;';; t; = 5 befoditW5065ilheb the NRC regional office for review and approvarin" accord $nce):withIS;205%1NNviewer should ev examination using the crMrisioiifform E#401-6 aridInclude the signed form in the examination package submftOS'to theNRC.sfacill'tyJicorpsees:are responsible;for ensuring that contractorgl i ated eMMthel guidelines,herein!and.:are encouraged toyerify:the ongingthiiTuostionIUsed to;constructJhe; examination; 21 N iiner TEM a.
~
N n .g. chief ~ Aall independently review all examination questions for
, . ,'prding, operational validity, and level of difficulty. As a minimum, the shall^ctis6k the items listed on Form ES-401-6. If the chief exar5I%$Isikthe exismination, another NRC examiner must perform the indepen .
I b. The chief, examiner should review the examination as soon as possible after receipt glEthat
^
supervisory approval can be obtained before the final review with l the factity licensee, which is normally scheduled about two weeks before the w exsili$dation date. It is especially important that the chief examiner promptly examinations prepared by a facility licensee because of the extra time at may be required if extensive changes are necessary. The chief examiner shall consolidate the comments from other regional reviewers and submit one set of comments to the author,
- c. If the facility licensee developed the examination, the licensee is primarily NUREG-1021 8 of 41 Revision 8
f ES-401 4 responsible for ensuring its technical accuracy and complying with the restrictions on question duplication and examination bank use (i.e., t.eded ltems 1,4,5, and 6 on the QA checklist). However, the chief examiner is expected to use his or her best judgment and take reasonable measures, including the selective review of reference materials, individual questions, and past. examinations, to verify these items when reviewing the examinatior6ixclusive reliance on the facility authors and reviewers initials is r$f adeqdaie. Depending upon the expected quality of the examinatiodsiisiti$itime available before the scheduled review with the facility 4isnsee, $dcNdbr,another examiner shall independently review and vidfikthe technical"ac5$acy of a ,2 sample of the written examination questianii. " " If the NRC prepared the: h examination,' Item
.QA' Nhd$N
$k$applies checklist on' only,to
~~
the last two, licensing exammation'sjatthi}achiQ Moreover/ltemL5;5id the -
~ facility reviewer blocks hColumnfb].are no~t ;for NRC-prepared
.exammations:
(if
- d. ' Before reviewing the examinations amisThe. author ort ; contact, the chief examiner will note anyW#$nd responsible . supervisor (or a Aisjsi forwahMxaminations to the necessari(gnafindih othedthan the chi for review and comment in McordanciYisii;8Ecill6siE3J There are no minimum or maximum limits on thoTumber adicopeTANiEies the chief examiner may direct the author or fag contacfIo make kIE proposed examinations, 1 provided that they a,rspecessary^to makeJhe examinations conform with established acceptisce critedad Refer toiES-201 for additional guidance regarding theelRC}esponsd6 veloped examinations that are gnificantlyTisilcMint. " "
WE%'Q _ supe _e al, generally at least 14 days before the examinations "sbufflicheduled' the chief examiner will review the written
*5cniiffidens witti licensee in accordance with ES-201.
hk Y ExamM5enshlet are written by the NRC shall be clean, properly formatted, and
" ready-t _ re they are reviewed with the facility licensee. The region shall not 1he facility licensee to ensure that the quality of the examination is accept $50e 'r administration. i O
f. f After remewing the examination with the facility licensee, the chief examiner will l
@$ enashithat any comments and recommendations are resolved and the ;
hi - ~ ' nation is revised as necessary. If the facility licensee developed ihe N dExamination, it will generally be expected to make whatever changes are 1 I recommended by the NRC.
- g. After the necessary changes have been made and the chief examiner is satisfied with the examination, he or she will sign the QA checklist and forward the NUREG .1021 9 of 41 Revision 8 1
ES-401 examination package to the responsible supervisor for final approval. If the examination was written by the facility licensee, the chief examiner should include a copy of the original submittal with the examination package. 3 NRC Suoervisorv Review fp A Av
- a. The responsible supervisor (or a designated attemate otherithanfthe NRC author or chief examiner) shall review and approve the examirMaisiQA checklist before the facility prereview per ES-201. phrvis%reviewis not intended,, _
to be another technical review, but rather a general assessment of examinationf/ quality, inciuding a review of the changes %eing reconj$Inndd e by'the' chief // examiner, and a check to ensure that all'ttie applicable administrativeiL" requirements have been implemented. I r j^y f g { *Q f
%M%
- b. The responsible supervisor should ensure that,,anygificant deficiencies in the original examinations submitted by a facilEy licenses;are, evaluated in accordance with ES-201 to determine thi appropfiskE50Fae of action. At a minimum, the supervisor should epatjiIsIstat they arEMid in the final-examination report in accordanosiiltiisFMO.1. jf Following the facility $I]hMkkes review /[heAk resgonkssOsivlsor should a
- c. /t examination to ensure tp the co the NRC have been agpriately,ncems addressedy The supervisor ex'pressInd shall not sign by the facili Form ES-401-6 until heer sheis satisfied that the examination is acceptable to be administered. f f,f'l f/
4. Peer ~ Review '
+:wyk!
Rf
. fgfM fffk, %W+
% Ikfinal of $$b examination's technical accuracy, facility management
* ~ " consid$I$MniNditaring
^ the NRC-approved examination (under security
) to orQin6$ licensed pemonnel who were previously uninvolved
; .the exani$ation. Any comments made and problems identified duririi indministration shall be discussed with the NRC chief examiner and res disiiiiin the examination is administered to the license applicants.3 e
M. . kexaminationZnotjo;changeyeliminate;questionsLthat;are
. validity;of or;indicsEKtraining"deficienM TTACH S/ FORMS Atta _
g.. ; Example SystematicjSampling77tg-t* -Ty ForiiiES401-1, "BWR SRO Examination Outline" Form ES-401-2, "BWR RO Examination Outline" Form ES-401-3,- "PWR SRO Examination Outline" Form ES-401-4, "PWR RO Examination Outline" NUREG-1021 10 of 41 Revision 8 e '
ES-401 Form ES-401-5,' " Generic Knowledge and Abilities Outline" Form ES-401-6, - " Written Examination Quality Assurance Checklist" Form ES-401-7,' " Site-Specific Written Examination Cover Sheet"- A
/. ,<.
//
,n < s ep:/
p , , , ~,,g , x ,
/ >r> %, Q O y<, 'h s,4p j< m. /,,,
-( ;
&eJr -e,o ,, :+,
. <, : y s j;'
g :.
,f,y N,%. < m.p>s>,;
,1 % >~ o,
' ' * +4'p^
., ; , *ny' ' , ,
(
% A,*
s g% sm '; b, ;,
> 4 i+
,y\$' (O,'> \ +!,K*
f"rl ' m q?%, t,rs,..% f:'di,t 'n% [' u '*';$ %:,: /
, fgM,"?'*~t, c'}b,+;% ' c,[g 4
# =< '
/ p e
u%D,*y>'i
))d^N' f '?
e,n v
,~# ,< 's
/:,$
- Av'y l
~ r :+ <,.
Q t' f< A.+mzt':At ' y}< I'i? i'5M'y ( h/l' &, K %.
%'dt";!l$. ' *
t h
, , <ng c:a&y %*,"', \ql,~,;e )n
l e v ; q,Q f
f N ps M V N.Ji f.A
- 7 3aA' gage *?
NUREG-1021 11 of 41 Revision 8 4 h
ES-401m :n .: Example Systematic Sampling Methodology -
Attachment:
1 The following processpwhich uses the;BWB SRO outline;(Form ES-40131),forillustratientcould
' be;used for each groupJn; Tiers;;1;and,2 of the examination outline.
,1:l - Review each group;andioliminate those; items (Emergency / Abnormal Plant _Evol.utions i
. (E/(9Es) forfiery and systems for Tier 2);that clearly do not apply.to the facility for )
which.the examination isl;being written;;be prepared:to; explain the basis:forlthe deletions to the NRC; chief exammeg s[em$. g Sequentially,! number!.thejmmaining;. items.in,.tWgmu&a da d 2l same numberof tokensH1f we assume:that nonei(;p..andjpquentikiy:annotate the 20~E/APEOhTiiE1FGroupl1 4 was delete. dlthere shoul.d be 2.0.. g tokens.l:numberedf.ro.m.
, y.e. h,o!2'Oh,i\
I a; iftheyumbeM#emsymainin@the iri case:20[i.s ri;the required.:numberjof poets. for;thejgroupfph(iQggight ha.nd. col.u examinatierj 6ctlint: in this: case:26 ) j sampled atMone{Mw2TheEs)Rthen"ea66Hom i m"the: group wo ! randomly _selectmg:and removing tokens'(in this; case iG the.20);until the j requiredctotal. number of;poentsiis:reaHTed.!:Updateform ES40131,to; note:the selected.itemsj MM I 4 k
' b: ' if the. number.ofitems":re ing inthe;grou#2 p r.than the:requirednumber M.pointkfor[tkgroupieN!TptMrouk2 Tai 20; items tiut only..requirosh7 points),ithen randomifielect.andjemove,tisrequired number of. tokens. and note;them.onfntm @il Sj After s
numb, selecting theltggto bei_ .. h ..i0roup!asjdescribedjinJtep2[ count.the
) '
o[N&categones:1rOhe; group. g 8 foteackgroup;in[ Der't i.e2K12K2l K3; AEKf3nd G) a~nd%dintiaty;anriiTakthejsame nurnber;of tokens (inthis; case.6)4
. F4rWE/ APE and.@Q{j(arRsystJem selected;in)APE categorygon Fyrm ESM013MXthe;E/ Step % randomly (or; system):Was: 1selectjandJem sampled once omly; select?a second.RA categorynThen.;mptsce all Mgn,%ge[
tholokens Step 1ald}epeat.tKprocessforcevery:selectedjitem;in;eac containerjar}d group. y ~p Sj Xsirriile[ .. Mrandomly selectfrom am uig the:K/Astatements.underjeach seigctedjK/A"catgorKEDescribe eachWA. topic in theippace/provided..cnform ESM013 j tand:enterNWJatingEK/MhavinglimportanceJatings!.less than12.5jcari;be used iOustilio6ased;onlplar!t pdoritiestihe facilityjcontact should be. prepared to thol.ismis to.t.,he'N.RC_chie.f
-- - exa_mme_r:
5 I l for de ,%).Mthejexaminatsor[ outline lgndomlyjselect;K/Aslaojthateach categorthat,WstJMg.fogMs)[ )j NUREG-1021 12 of 41 Revision 8 L l 9 J
ES-401 BWR SRO Examination Outline Form ES-401-1 i
' Facility: Date of Exam: Exam Level:
. . K/A Category Points i Tier. Group Point !
K K K K K K A A ~A .A Total I 1 2 3 4 5- 6 1 2 /3 4s 9,G " Emergency &
- 1. 1' hj$h hk 3
r
/ hhh Ih o y 26
~~
)
Abnormal 2 h-
@@~ $ h8fs j ((h$gg[ lh 17 gf ;
Tier bk. dh $b I' b hh M Evolutions pg gg g 4 g g pp DTkb" qp > hk/c h hh .. .h Ik@ 1i- <;
+
b $Nk kh e 23 ~ l 2'- Plant 2- _ 3 E A< 13 l Systems L 3 Mkh b kA. N[ 4 ; Tier Totals /
+%0 k s h fff!khk [
/ $? GW 2 40 i
a ag a wr
- 3. Generic Knowledge and AbElie's / Cat 1 Sat 2 Cat 3 Cat 4 ;
rh b[ ; f Note: j$3 . .n+;0;k'Et:::;:$ hhJ' cr. ;;;d;;; ::':d Ensure that at JffiNeast
*fgp 2Eand'tw%
s ordM 5.$wn ever/N#jf5stegory l.i.i_ sampled'within seeMeerTiers.t.a. nd. o for. each category: Inter 3.
.s - - -
ypy$ k hit 0sl'pointWisiniist, match ihose specified in the table.
%8ehd4ppics frWMisystems; avoid selecting more than two or three K/A f did fpN pliafibma give'Epom unless they ralate to plant-specific priorities.
/i The sgsiiilies are not applicable to the category / tier.
M.ggpAsMTierslandXshall,gselqcted from;Sectiogl.cf.the F CatalogMgpWs[befelevanty tijefapplicable:evolutiorforisystem;
'i Onithsfollowing'pagesFyntentheMnumbersia;brief description lofpad 4 - topicib topics'Limportance ratingsfotg RO licens@velfand the point I?jp .
_. $jspstemicategorygigroupEand.;tleDMthe putline?iK/As' below 24 . fh $ kp 3 ....ould;beMied.Lon the htsisMpanyspecific pgoritiesl
~
w ' ..is .
- NUREG iu21: 13 of 41 Revision 8 4
1
- s 1
0 4- W o 8 S P n E i o m i S r o p v F m e I R
)
( s c ip o T A
/
K - 1 p u m G
/
1 r e l i a eT t o n- T u is un - t n Oio t io nlu P ioo t v p 1 aE u n it o r 4 mn al a G f o xP 4 El e 1 Om G Rr Som RtA 2 A Md I n a 1 y c A n e g r e 10 m E 2 K 1 K I V H V f I V I. V r e V p I 1 n r I l w io w e m V e o c P V I l l d e I v e P1 n _ t c i V o T e d i o n C I. n X M r L nn X C u A M p io
/
t I i g r e u r 1 aw I I n F n t n ta t a le 1 e f o I / e e i d e e n l r b / to e g ty I I l e V d l o s V l i W e l e nm t a f e s s
/
e v i T l A m n R o o 1 t e h p W v ei sn R o r a o r e e o y 8 o e C e r ig m lo e eU r e dy S L u L r u o e 1 d s r u u e o L s e s r s P iv M n a s s s H T P r Pr o a H 1 s e s e t s e n e t e e n t c A b e n s e l o t n n t oe le t n r M e e a R l e e e r o e o a e e m lp P r e A d r P tl iia a m r V P s s R C m R ic P r P m is s W dc ns R m N r g e S e c g o n n e n : 1 I C o 1 t o c t o c e w r p t n e o o t is A e w t c o i a e r p w o on Cwo t s ia ls a 2 _
# r 1 a a y p e t e R f-g y a iss t
n p L f-t n t o 0 _ E M e e r u tt lp f n r e e o u r o MD f o T 1 _ P lo A R R D S e l o O E e D R r C S A O C - _ a r t y A t i R h w h h d v m o t n h u h h p p h w c a RM h h r o G
/
E r e C ig o ig ig a n c n o ig f e ig ig u ig o e CR ig ig g E P $ H L H H I i C H R H H O H L R SP A H H t e R 1 0 3 0 6 0 7 0 9 0 0 3 4 5 6 7 3 2 4 2 5 2 6 2 7 2 0 3 1 3 7 3e 8 3 0 0 a U 1 1 1 1 1 1 C 4-S 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0 5 0v 5o 0 5 0 0 A N 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9 9b 9 0 / K E 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2A 2 5
a 1
- s 1 ti 0- i r
4- o 8 S P n E o m-r i s o p
. i v
F n I r e R
)
( s c . _ ip . o _ T . A/ m K _ 2 - p _ u o r G
/
m 1 r _ ie : . eT l a n- t o us un T _ dr Oio t . nlu io . ioo P ,_ tv p m naE u 1 it o r 4 mn ela G f o nP El 5 e Om G 1 . Rr m Son Rb 2 - WA A Bnda - y 1 . c A _ n e g - r 3 e K m E 2 K _ 1 K e h t a r o V ig is e r C t
/ H tn n
l l t f l e A'v o d i I V V l V I I e e n r e - it e c m i I I V v e e it o l l c n r o u u r l l I ; W i ir 4 L A s i D C _ u F c w i 1 ; V I A V V V r I I h h P 1 l e n F f o V a C ip r 1
^
_ 1 e W t I n f t t a e ig ig y T l C e g e V H H t e s r D e _ C n lo n r W f o e r o v e _ ru i l t u t n t n t n a o s n f o t a L t a f o f o e o o I t a l o e e e l l i S L e s r _ r e s s lo s r e o m m m V
/ o e r e T p o s o C p p P n e d n o n t t n u o o t
n n in in i
/ :
m o L e a e r L L t w m u m o e a a e e ls _ G Wr n e n o e le t n t n t n a o C l a T l a la e P T o o o S t o N n e v C d s n Ytp 1 ia t o n o b a e e t o t o t u D g e e c C C O T 1 2
# T ib t
c t w T T t n h R w r p y y y t n r M r r e n y r r e S C y p r a r a r a e i 0 E f o u e o r o o t r r u d d r o 1 P o l T R C D l l e f o f o D S n n dvn i F P - ge e o ie v y _ A s 9r m h p h 11 d s s h h oX o oi t n G .
/
E s e l a ig ig e r t r e o s s ig ig cI cV cle a r o o o E P M H i H H n o H e/ ei ev l g - L P P I L L H Sn Se Se P e R 4i o 4 r 1 2 5 8 1 2 8 9 0 2 8 9 5u 6L 0 t 1 n a 0 4-S E g 0 0 5 9 2 0 0 5 9 2 0 0 5 9 2 0 0 5 9 2 1 0 5 9 2 1 0 5 9 2 1 0 5 9 2 1 0 5 9 2 2 0 5 9 2 2 0 5 9 2 2 0 5 9 2 2 0 5 9 2 2 0 5 9 2 ? a2
& 0 5
9 3te 0S 5: 9e 2R 3s 0s 5e 9r 2P 3r 0e 5e 9 2w i 0 0 0 0 6 C A
/
K U N
1 1 t s 0 bo 4- 8 S E P n i o m r iS o p v F m e I R
)
( s c ip o T A
/
K l t a o
' T t
n io P p u o r G G 4 e1 A n t) q tu o 3 Or A n/G o2 it r ae n 2 A 1 iTs 4 m- f as x o Eme 1 A 6 Ots 1 Ry SS 6 t Rnt K Wla BP 5 K l 4 K . 3 K - 2 K - 1 K f r f o t o t u l a h e u r n s S d g e e io e y o l e s i r R o lo d n ta e lp p M e t r e r o e tn mr y r n m t n d e u a u o a o M n r u S r s C N I C w o io t n c C
)
o to i M n o t m u r s n A d n a m e e S T p P s m l e v e W t r s T s m l F j e n 0r e e I t S M e L i D M l a e n g M r e T r T in r e T t o t I m n R ll M a C br t l a C T 1 s y lo I E a P o T e : u a ;c y t 2 t C ( B C lc 1 C T Wr n S h P n o R e L I r a i N u P / r i r a io 0 1 tc L 6 S S c M e _ -. L o o M s P - S r / I c e / s t t S
"w
/
ic R C e C C C S r R S n S R V c c T E G y G I C e H P lo P P u lc u iC kr I a a r o e P L o P D C H R e e G C D g E R R R H l L H S R S A N R A P P R S R R S A E S 1 0 5 2 0 0 0 1 2 0 0 4 5 0 0 0 1 2 1 2 0 2 0 1 0 1 t e a R 4-0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 C U S 0 2 0 3 0 8 0 7 0 9 0 9 0 1 1 2 1 5 1 5 1 6 1 7 1 8 1 3 2 3 2 6 2 9 3 1 4 9 5 1 6 2 6 4 6 0 0 9 A N E 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 I K
1 1 t s 0 4- i n o 8 S P n E o m r i s o p
. i v
F m e R I
) _
( s c ip o T A
/
K l: a to
' T t
n io P p u o r G G 4 e2 A npn ut Oro 3 A n/G to2 i r ae 2 1 iTni A 4 m-as f x O Eme t A 7 Os t 1 Rr SS S tt n K da Vl BP 5 K 4 K 3 K 2 K 1 K e d e o d s M o e g n M i r a i y l lo a r l i x o p u r e e C S t n A n t m l e l d lo a a o lo m o n iJ Wg N o o p t r P g C P P n a. r tr n A
/ g /
s
/
s iu o n e n : s ln u u q C e t a t is ir o V t o ta m c i o r r E e G w D it n H o e R n o o e g g d ) n r C to u T T n e e C l a o io m i A T 1 t s a io C : : i k a n e D icr M t c o 4- t 2 _ y d r t a n l C I C l d s i br e o t n n _ S y w n e F / C t n R e e io 0 H lu o P P a u r A c o t o P 1 S S c U L L H L T o le s i r :v m - D C C M r ic C d t u S M
/
R 1 R l e V n t c a ( S E e t ia d P u u r W , y r G R M S W e W I P B M H H I S ia C fg e l. t . o C R R R R R h S R R R R R F u M M R e P U D f O R a i F r C o s n C o g E 1 1 2 4 6 1 0 0 0 2 I 3 0 0 0 3 0 1 2 0 0 0 0 3 I 0 0 ta e R 0 4-0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 C U S 1 0 1 0 1 0 1 0 2 0 4 0 5 0 4 1 5 5 9 0 3 4 3 9 3 5 9 2 0 3 1 0 2 6 0 0 0 A N E 1 1 1 4 5 6 6 7 7 8 9 0 0 / 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 4 K
1 1 ts 8 0 n t$ n 4- h S io P l o o E P iS i m v r o e F p R I m no
)
s t ( c s ip e o R T
- A
/
K l: : t a o T t C . n ' o i P p e i T. C
- G ;
r f o A4 : e3 - in)s l tq u oro 3 _ t A e nG / N l r
?
o2 o itr R ae 2 ir P d iT ni A e 1 m- c d n 4 a-i N f x 1 c e o E . A t p m O^ S- m 7 Ri o 1 SS t m c K6 l e Rtn i P R
~
Wla BP N 5 2 K T K 4 F
?
7 5 3 . S : K f
?
7
; t 2 s m K f i
i :m 1 S.
! m K R.
T I O M Si e d lp o m T T f t o n p R T m a u 3 e h n t s : c e "g y ) e lo s 0 e M e C e 4 r 1 m b m o e o d e le r t i v r n e m N
/
i D r P e a g S t t e a s t m e : le J""" m l U (
#. d r
o in t s n n In a t t a n : lo e e o l e t o C n_ o a H Ty o h d g s T n r e n e 1, d o I g n c R C o e g t V s e t n M d y tr S R l o d t n e i o m ie l o isr o n a r o s e V r o P W r r r e P " P 1 e t n c w t c y t n v l e e d t n e r o g 6 2 o e u ie e e a e g m 6 0 C T r F M R R l P R e " r lc 1 1 3 0 t t r r 6 - 1 1 0 0 0 2 a 1 p 0 4-0 0 0 0 0 0 0 0 0 0 0 0 0 0 C 5 m S- G S 1 5 3 9 8 8 0 8 0 0 A 5 = tn E E 0 2 1 2 3 2 3 5 2 6 2 8 2 9 / K g a R 2_ 2 E =
- t = 1 ? . e n l
P U N
ES-401 BWR RO Examination Outhne Form ES-401-2 Facility: Date of Exam: Exam Level. i K/A Category Points Tier Group Point I K K .K K K K A A A A A Total i 1 2 3 4 5 6 1 2 43 4a 7 ! i J84- F >' f P*FE-t@ M #n x h?% > < 1, 1 #- GA Qi2 13 i Emergency & >" Abnormal 2 e- w. m fx> &% gt. we %u k 19 .r 4 ; Plant 3- &# *J iiR # 19F E is udF Tier
- Totals kW hh kg gh M $$. i N*
jh ME IIhS k @@b6 , I a*q Wa i e e en. *gL B 28 2* - T :L
- Plant 2 .e s"tif F@ 19 h
h.47 Systems 3 NE $2% Ti 7 4 ! Tier Totals ti [ [* [kf hg t dit [ 51 am y n Eat 2
- 3. Generic Knowledge and AbNilies
- Cat 1 Cet 3 Cat 4 4 AP' At # 1 p
Note: - [Al^.;;;;..,,;;c. a. g om ; nc.,, : K!,' ;;;;;;,;f.;;; n:n; Ensure;that~at ggeveryMYhtegory
- ;;;::egwantwv k ,is. sampled within :.;2. 2;Ti.ers 1'and
!$l.e.,ast W12 and .two,fo.reac on ,li@id.mhs category]rgier).
~ - - -
"JAishiialpoint M 4 Pd - 166165.%:iUbLmatch
*g-;9;eM: MN)$:k those specified in the table.
ele %ct': topics frots manystystems; avoid selecting more than two or three K/A 4S.'c o mnw m:a givenptem unless they relate to plant-specific priorities. Sydsm;fe+o, eisElutions within each group are identified on the associated outline. The sNadedairiiss my}}