ML20216F073
| ML20216F073 | |
| Person / Time | |
|---|---|
| Issue date: | 07/30/1997 |
| From: | Steiner P NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | Reyes L NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| Shared Package | |
| ML20216F066 | List: |
| References | |
| FRN-62FR42426, FRN-64FR19868, RULE-PR-55 62FR42426-00002, 62FR42426-2, AF62-2-027, AF62-2-27, SECY-97-079-C, SECY-97-79-C, NUDOCS 9709110158 | |
| Download: ML20216F073 (3) | |
Text
_ _ _ _ - _ _ _ _ - _ - _. - _ - _ _ _ - _ _ _ -
['OcarpN UNITEo STATE 3 NUCLEAR REGULATORY COMMISSION 8
k REGION 11 g
j ATLANTA FEoERAL CENTER r
61 FORSYTH STREET. SW. SUITE 23TB5 f
ATLANTA. GEORG'A 30303 July 30,1997 MEMORANDUM TO:
Luis A. Reyes, Region 11 Regional Administ tor FROM:
Paul M. Steiner, Reactor Engineer, DRS Gb
SUBJECT:
EXAMINER FEEDBACK TO STAFF REQUIREMENTS -
SECY-97-079 - PROPOSED RULE -INITIAL LICENSED OPERATOR EXAMINATION REQUIREMENTS I am writing in response 17 ee commissioners regrest for information as, outlined in SECY-97-079 regarding the proposed rule change that would require licensees to generate licensed operator initial examinations. For the past year, we have been allowing the facility i
licensees to generate their own initial examinations under pilot guidance. Region 11 examiners have been aggressively enacting the pilot program and leaming valuable lessons from it. Almost all of the Region 11 licensees have already worked with us in generating an initial examination.
Ac with most areas we regulate, we have come across good performers and poor performers.
My oncem is that we have compromised our ability to administer an examination that will indept idently and objectively evaluate operator competency. This would result in an increased risk of an incompetent operator being placed at the controls of a licensee facility.
The purpose of the letter is to be informative. To date, the communications between our headquarters branch of operator licensing and the regional examiners, have been centered on our experiences in implementing the pilot process, not on whether or not we are doing the right thing. I feel strongly opposed to the direction we are taking with regards to the proposed rule change. I have talked with the headquarters branch, and I have volunteered to generate a survey which would poll all NRC examiners about their experiences with the pilot process. The results of the survey will then be discussed at our annual examiner's conference. I also intend to poll the licensee training managers prior to our Region 11 training manager's conference.
In SECY-97-079, the commission requests that the staff provide additional discussion in the Erderal Register notice (FRN) explaining the safety benefits of this proposed rule making, indicating, if appropriate, that the NRC focus on operator performance will improve through more direct (i.e.. NRC staff replacing contractor personnel) involvement in simulator and walkthrough examinations. I do not believe this to be the case. On the contrary, I believe that the simulator and walkthrough examinations have degraded significantly, and are now only at minimal acceptable standards. The depth of simulator scenarios was created by examiners spending months working through EOP's, and AOP's. We developed exam material that ensured an operator could adhere to the facility license under a full range of accident conditions.
9709110158 970811 PDR PR 55 62FR42426 PDR
This no longer occurs The training staff pulls a scenario from their bank, which they may have run previously on the candidates, and adds the NRC required instrument and component failures.
This has two negative effects on exam quality. First, the scope of the exam is narrowed to the procedures the licensee believes are both important, and heavily trained on. This is negative in the sense that we are not able to determine if the candidates are prepared to work with all licensee procedures. Second, the NRC loses that valuable review oflicensee procedures. The number of procedure discrepancies found during a pilot style exam is lower than that of an NRC generated exam, because the facility has already worked over the bank scenario procedures.
Also, the NRC staffloses the knowledge gained from this intense procedure review. This can be a problem when the candidate strays off the correct path. Previously, the examiners were exposed to most of the EOP's and AOP's and could figure out where the candidate was going and whether or not that path was acceptable. The emphasis of the examiner's time has shifted l
from researching licensee material and generating a quality examination, to that of ensuring the i
licensee has met our guidelines for content. We are regressing into an organization that is l
primarily focused on enforcing admimstrative guidance.
/
l We have re'cently been faced with a heensee tram'ng manager who openly stated (not a direct quotation) that he intended to make the examination as easy as possible to ensure all of his candidates passed. He went on to mention that he already thought all of the operators were I
competent and why would he ever generate an exam that would risk failing them. As an example, a question the licensee proposed on the Senior Reactor Operator (SRO) administrative section was,"What does S.T.A.R. stand for?" I suspect that the site janitors know this answer.
It is not even on the scale, relative to the level of knowledge that we require of an SRO candidate. The disturbing part, is that we had to debate with the licensee to have the question removed from "our" examination. It is this type of continual debate over the two month review peri that results in a degraded product.
The root cause of our problem is that we have created a conflict ofinterest with the training managers. They feel we have put them in a no-win situation. If they generate an exam that results in a failure, there management will evaluate them as poor trainers. If they generate an exam that is too simplistic, the NRC will write a negative report. The weak perfomaers have chosen to write the simplistic exam.
The SECY also asks the staff to include in the FRN a discussion of the importance that the NRC places'on ensuring adequate security of examination materials. I believe that the NRC places a very important belief on maintaining security, but we have no way of ensuring it. The licensees have always had an opportunity to compromise an examination. Recently, a licensee encountered a willful compromise of exam material. During a ren-pilot examination, a copy of an NRC wTitten examination was found in the licensee's copier. Ne rely heavily on the licensee's integrity and on a security agreement signature stating they will not compromise the exam. In the past, our primary means of ensuring security was to limit the amount of time the licensee was exposed to examination materials, usually around three weeks. With the examination being on-site for a much longer period, the potential for exposure increases proportionately. Under the new rule, the exam security process will start around a year and half prior to the exam. During this time, the licensee will have to separate the exam wTiter and the
d...
exam material from interaction with the potential candidates. After administering a simulator examination at a Region 11 licensee, a training member staffinformed the examiners that he found a copy of the scenario in the copy machine. This resulted in the examiners switching to a backup scenario, which had to be validated on the spot, resulting in a degraded exam and undue stress placed on the candidates. Clearly the pilot process increases the threat to examination sec<rity.
The last item the SECY asks the staff to provide, is additional legal analysis regarding the basis for determining that the backfit rule does not apply to this miemaking, particularly with regard to modification of or addition to "the procedures or organizations required to design, construct, or operate a facility." The licensees initially proposed that they generate the examinations to improve quality and cost savings. Feedback from the licensees indicates they realize the pilot program and proposed rule change is a mistake. They did not understand the level of difficulty of writing an NRC examination, nor the impact it would have on their staff. A typical licensee generates one initial license class per year. This does not provide them with sufficie.nt opportunities to train their staff on how to generate an NRC style examination. This results in the licensee starting from ground zero every year. The typical NRC examiner participates in 12 or more initial examinations per year and becor-s very skilled and effici'ent at generating material. Although the intent of this program was to save contractor dollars, the licensee is now hiring the NRC contractors, investing an average of seven hundred man-hours per examination, and are still struggling with the NRC over exam quality. The licensee has been advised to create procedures for exam security. Some licensees have created separate rooms, purchased computers, copiers, modems, and software all in support of maintaining examination security.
We have placed an additional financial burden on the licensee, with a potential decrease in safety. I find it difficult to see how the backfit rule does not apply.
With this leamed knowledge of an inherent conflict ofinterest, degradation of examiner knowledge, increased threat to exam security, and a potential backfit issue, we need to take a step back and reevaluate our beliefs about the examination process. Our initial feeling was that having the licensee generate the examination would improve the quality of the exam in both technical and content areas, all while saving contractor dollars. The poor performing licensee's have demonstrated that they are unable / unwilling to produce both a technically accurate and high quality product. I do not believe that the pilot program has demonstrated that the commissions concems as outlined in SECY-97-079 are not warranted.
CC:
J. Jaudons, RIl DRS Division Director C. Casto, RII DRS Deputy Division Director T. Peebles, RII Operator Licensing Branch Chief R. Gallo, Chief Operator Licensing Branch G. Meyer, RI Operator Licensing Branch Chief M. Leach, RIII Operator Licensing Branch Chief J. Pellet, RIV Operator Licensing Branch Chief