ML20235M553
| ML20235M553 | |
| Person / Time | |
|---|---|
| Site: | Prairie Island  |
| Issue date: | 09/08/1987 |
| From: | Sellman M NORTHERN STATES POWER CO. |
| To: | NRC |
| Shared Package | |
| ML20235M506 | List: |
| References | |
| NUDOCS 8710060226 | |
| Download: ML20235M553 (26) | |
Text
r , . . _
q^
..c . % .a .
"{I ' .. .
t
.i September 8, 1987 TO: 'NRC STAFF
SUBJECT:
NRC EXAM PROCESS 1
l In preparation-for the September 10th public meeting on l implementation of 10CFR55,.1 asked Prairie Island's operators'to l- give me comments on their personal experiences with recent NRC license and requal exams. Attached for your consideration are their uncensored comments.
l Since licenses are held by individuals, I felt'it was most.
appropriate to give you the comments of individuals. On behalf of all the operators at Prairie Island, I would'like to thank you for the opportunity to present these comments.
Sincerely, Mike B. Sellman Gen Supt of Plant Operations Prairie Island Nuclear Plant l
l MBS/jm Attachments 8710060226 870922 PDR ADOCK 05000292 ..
P PDR N 1
1 i
N l
______.___________.m_____ _ _ _ _ _ _ _ _ _ . _ . _ . _ _ __ _ _ . . _ _ _ _ _ _ _ _ _ _ ____._.m_ _ _ . . _ _ _ m_ _. _.._ _._______1______ ___________'_d
- ** v COMMENTS ON THE NRC EXAM PROCESS (BOTH INITIAL AND REQUALIFICATION)
Operator 1 A) A major problem is low creditability of examiners. Part of this is lack of experience but a major.part is also conduct and comments examiners make.
Examples:
- 1. Examiner stated that "We don't believe the KSA ratings on electrical are high enough. So we will ask electrical questions anyway."
- 2. Examiner stated that "The industry pass rates are to high. I don't believe anyplace should have 100% pass rates. The average failure rate should be about 33%."
In this particular case, over 50% of the candidates failed. Later, in meetings with the Regional Administrator and the Section Chief of Operator Licensing they both were visibly embarrassed by this comment and apologized for the examiner's comment.
One year later, the same Section Chief of Operator Licensing denied ever having heard about this same comment in a meeting with plant operating staff. .
l About one year later, still this same examiner was l promoted to Section Chief of Operator Licensing in one of the regions.
- 3. After simulator exams, a comment about the Examiner Standards guidelines was made and the fact that this standard recommends very few questions being asked during the simulator portion of the exam. The examiner's response was that "that's a minimum guide only. I can do anything I want over and above that."
- 4. After simulator exams, a question was raised about the number of events each candidate was subjected to.
Also, why did the exam consist of three scenarios when the standards guidelines indicated two was sufficient and on all previous similar exams only two scenarios were run. Again, the response was the guideline was a ,
minimum and the examiners could exceed it by any amount at their discretion.
- 5. After an examination was conducted, a formal complaint was made (unjustly I believe) to a Resident Inspector about the conduct of one examiner. NRC personnel visited the plant site and interviewed many of the people involved, including several licensed operators.
The NRC personnel were very courteous and assured all involved there would be no. action against the plant as a result of the inquiry.
IBM
-- - _ . u
i j
' , Page 2
)
A requal exam had previously been scheduled for 4 'l months later. As with all NRC requal exams this meant 20% of licensed people would be tested. Of the 17 people selected, 1 RO and 3 SRO's were in the Control Room when the examiner investigated made his remarks.
That's four (4) of the six (6) licensed personnel that were there and included both of-the people-the examiner j was directly talking to. So 67% of the people in the j Control Room with the examiner and both licensees the investigators talked to were randomly selected as part )
of the sites' 20% for examination.
i Further three (3) of the four (4) failed the exam.
l Also two of three failed solely on the written test.
This may have been pure coincidence. But, it is a extremely difficult to convince the operators that it-is, when it's such a low probability. Especially since it's quite similar to what happened with the previously administrated NRC requal exam two' years earlier.
- 6. Prior to the last NRC requal examinations, during discussions with the Section Chief for Operator Licensing, it was stated that the people who took the i last NRC requal exam would be exempt from taking this one. Just before the last examination was given, this 3 same comment was made by the individual who was the l Section Chief of Operator Licensing at that time during l
a meeting with interested licensed personnel at the plant.
This comment was then sent in writing to the plant site, as company personnel did not feel (from previous experiences) they should act only on a verbal i
statement. After receiving written verification from the Section Chief, this information was given to licensed personnel at the plant.
About 2 weeks prior to the exam date, in a meeting with licensed personnel at the site, this same Section Chief said he could not honor this commitment. He ,
stated Washington had informed him personnel could only j be exempted for one year. Since more time than this j had elapsed since the last requal exam, no one was j exempt. It did not matter what he had communicated previously, either orally or in writing.
IBM l
l
_________ - - - -- )
7
... .~.. . i Page 3 1
Now, since I deal with these people frequently, I feel l that in all instances except for the second one, none ;
of these instances show truly unprofessional conduct. I But, when you add these together and include at least '
that many more similar instances a*t one facility in.
four years or less, it becomes truly difficult to put much faith in anything involving the operator licensing process. And this is without getting into the real issues of exam content and performance criteria yet.
By the way, all the instances involved NRC examiners, I not contractors. There have been similar problems, though not as many, with some contractors.
i B) Another major issue is the content of NRC examinations, I especially written exams. A major thrust of the utility industry has been a task analysis for all job positions. A flaw in this is that this process does not identify required fundamental knowledge, nor knowledge required to handle "what if" situations. In real analysis results, the required off the tip of your tongue knowledge an operator needs is relatively limited. The rest is available as reference material and should be used as such. Yet, in practice, the utilities have been forced to add additional training materials and objectives so operators can pass !
NRC exams. Some examples: l
- 1. NRC examiners were asking many questions involving i operation of protective relaying for breakers. By operation I don't mean "what an operator has to do with these relays or does as a result of these relays".
Rather, how the relays work, what their setpoint is, and where the schematic diagrams for them are.
As a result, plant training materials were rewritten, special training sessions were conducted on how these various relays worked, how to read relay drawings, and what the various relay numbers meant. Yet, what do operators really need to know about them? They have to know these relays cause breakers to trip, de-energiring components. They also have to know that they cannot i indiscriminately re-energize these components, but must '
first contact station electricians so that they (the-experts in this area) can determine and correct the problem.
IBM
r Page 4 Now, they should also know that under certain conditions these relays prevent them from operating the components. This knowledge can be gained in one of several ways. This facility made a management decision years ago that operators would use* logic drawings for this purpose. The examiners forced candidates into electrical schematics which they were never taught to use, based on the decision about logics. So here are operators, required to display knowledge they will never use on the job in order to get or keep the license that enables them to do their job.
- 2. As a result of NRC simulator examinations, the methods I of operating on the simulator were changed. This incident involved switches that are behind the main control boards in instrument cabinets in the plant.
Under certain conditions these switches are required to i be tripped. ]
In the plant, the person who trips these switches is l
~ 10 or 12 feet from the operator, though he is out of i sight. The switches are all in separate cabinets (4 i cabinets) that are color coded. In some cases, the 1 operators then selves may actuate these switches, in l other cases the Instrumentation and Control Technician i (I&C) will do it for the operator.
l l On the simulator, these switches must be actuated by an instructor in a booth. The instructor just has a computer terminal to do this from, and uses a list of switches students provide.
The examiners felt this process, on the simulator did !
not provide much control. In truth, they were at least 1 partially correct. But it also did not provide real J l
plant conditions. Under existing real plant conditions, ,
l adequate control existed, at least in the view of the l plant management, operations group, and plant training !
staff. )
l Yet, in order to appease NRC license examiners, the {
training department had to develop a special method for j performing this action. This method is only used in '
i 1
the simulator. It is an additional item operators must remember to do when under not only the pressure of operating, but also of being examined. (And lest this is minimized, real '.e their whole career is dependent on keeping their license. So this exam pressure is tremendous.) But this is an item that does not have any applicability to the real job done in the Control Room. )
i l IBM
' ' Page 5 1
1 One point that must be made is the ability of the-simulator. A person must realize that these-switches are related to reactor protection and are important.
If these switches are not positioned correctly, it can affect (though not completely prevent) the ability of automatic reactor trips or it can cause an automatic reactor trip. But, in order to turn.the correct switch a person must only 1) be able to distinguish between one of four colors to get the correct cabinet, 2) be ;
able to read a six (6) or seven (7) character label, ;
and 3) lift the labeled toggle switch. This has not been a problem at the plant.
Since this is physically a very simple process, a decision was made not to spend thousands of dollars to
. build cabinets identical to the plants which the I
operator rarely uses (maybe once or twice in two or three years in the real plant and thats for all ot < erations not each operator) and which are so easy to use when required. Because of this decision, and l
examiners being convinced they've found a real problem i where none exist, another shift in how we do things :
occurs.
l
- 3. One of the things the industry, through 1NPO, has done to try to improve plant safety is start teaching i diagnostic techniques, team training, and group l dynamics to operators. A fundamental part of this concept is teaching supervisors to listen to their subordinates' input in unusual transient and accident conditions. This allows better decisions to be made and results in significantly safer plant operations in i the majority of cases.
At the same time, this stresses to the subordinate that he should give his supervisor recommendations and observations in these conditions. Not to argue with his supervisor, but rather to provide support. In the end, the supervisor decides, by allowing the subordinate to take the action or ordering it taken, whatever works best for that supervisor with the group.
This is all done within the framework of following, where possible, previously approved procedures. In most cases, the procedures will correct the problem and as long as the right procedure is followed, no major complications will result.
Now the exams come. The NRC evaluates the SRO (supervisory level personnel) candidate on his supervisory ability.
Repeatedly the comment comes back that the candidate lacked supervisory skills. The basis for this' is normally that he did not have dynamic control. He didn't order the operators to do this or that right away, etc.
IBM
Page 6 After talking to examiners, it seems that most of the team and group dynamic skills taught to improve safety, result in poorer results on examinations in the area of supervisory skills. When this item is discussed with examiners and administrators, they*say, "Yes, it sounds like a good idea to teach these things. But we have to evaluate the person as a supervisor and this is over criteria."
The result, the poor operator has to lean one skill to operate safely. Then he learns another skill (play acting) in order to impress examiners, most of whom have never really operated (even most of those hired as licensed SRO's from utilities or vendors never operated commercial reactors. In most cases they were instructors or engineers). A commercial reactor doesn't have a realistic idea of what the operator really needs to know to operate.
I could give many, many more examples of how operator training programs are forced to be changed to get operators through exams. It seems that every exam a "new" concern is found that must be fixed. Sometimes it's plant related, sometimes training. But in almost every case, it's based on an examiner's opinion, does not really constitute a failure to comply with any regulation and results in operators getting jerked around in some manner.
At this point, the basis of training programs is so twisted from a pure job analysis basis that it would involve major effort to go back to a pure one. Yet now the NRC plans to base exams solely on these objectives, and the real kicker is utility managers and executives who individually don't dare rock the boat by complaining too vocally. Why? Because experience has convinced them and their staff (down to the operator level) that if they do, the NRC will get even somehow.
These comments are from someone with 17 years experience in the nuclear field. I've been an operator or involved in training operators since 1970. I have an SRO license, and I've taken and passed an NRC SRO requal exam. I personally have no reservations about taking another exam. But that's primarily because I feel the pass / fail result has almost no bearing or relationship to my ,
ability as an operator.
There are many more ideas and examples I could sight, but the entire process as' presently administered seems blatantly unfair. In fact, it has progressed to the point where senior personnel, with years of experience in operations are giving up licenses or thinking about it. If this continues, it cannot help but have adverse effects on reactor safety. The exact opposite of what should be happening.
IBM
l Page 7 I hope this process improves,;but I. haven't seen improvement yet. In fact, my own frustration withethe' examining process is a major factor (though certainly not the only one) in my decision
~
to leave training and go back into operations. At least then 1 only have to deal with the process for myseIf. ]
'Perhaps the most important first step is.a much clearer definition of pass / failure for simulators and orals. Also~a much-clearer definition of acceptable and unacceptable written exam-questions and answers. Once some. common and more stable 1 l
definition of what's required of, operators is reached,- I think most of the problems will disappear. After all, it!s in the~best interests of both the utilities and the operators to pass. ,There 1 certainly is a lot of effort from this end to try to reach that goal, but it's very difficult when just as you've almost got ,
there, the target is moved.five miles away. j o
j i
1 4
l l
1 IBM
1'
\
.t__J
. <1= *
- a. .
l .
i l Operator 2
- 1. NRC written' test too long for the allowed. time limit. 'If the test length.is.to remain the same,. increase the. time limit.
After.all, are we. grading quality or qdantity in the answers?
- 2. No comment on oral except that pa'rtial. oral during simulator:
exam isn't a good approach.
- 3. Simulatorfexam.seemed a. bit long,and I believe unreasonable in the grading and exam technique. I feel.the most important aspect of the simulator exam is whether the health.and safety of the public is address'ed, not whether an operator inadvertently gets into the wrong' procedure before getting to the right procedure. The object is'not being able to receive an " OSCAR" for his/her acting performance. Also,.it isn't'real life for the NRC to use the " BLACK BOX" and rattle-off malfunction. af ter malfunction. In conclusion, it's'not-whether I'm qualified. It's a flip of1the coin with people
). examining me that have no commercial power plant operating l experience, have no common sense, and can't get a job in the-civilian area.
i i
6 :
]
l l
l f
k
' I i
1
8
>. f. .
i
. i Operator 3 !
'l. Time limit-put on-NRC requal. issued exam-is too short for' ;
~
60% exams. I prefer a full. length written exam with full time limit. The. reason is that some. sections may take a longer time-length,:while others may take less time..
Time length:on even their full length exam may.be~too.short of a time.for slower readers and: writers. 1 Final objective - increase written time' limit. -
I i
j l
l t
1 i
s i
Operator 4 The Training. Department is not allowed to see the exam until'it is already being administered. Some NRC questions are not q applicable to this facility or are too poor 1*y worded to be answered. The effect of these' questions.on the exam is detrimental.because they waste valuable time attempting to answer, demoralize the examinee, and predispose him to incorrectly answer' other questions. I understand the NRC's concern for' security and ,
integrity when dealing with the utility but some solution is -]
possible.such as the NRC.and Training Dept. reviewing the exam !
right before it is administered.
l l
l j
i v.
-y
- l. . . . . .
1
. ')
Operator 5
~
In my_ opinion the exam process has many problems. The.NRC examiners seem to be trying to write the mos,t vague and misconstrued questions that they can think of. In essence they are trying to write an' exam that only.a-professional test taker l
can pass. Listed.below are a few' examples:
1
- 1. Asking vague, and misconstrued questions - if you want'an.
answer to a question, ASK A DIRECT QUESTION.
- 2. Asking questions with. double and triple _ negatives.in the question. Confusing the person answering the question..
- 3. Intimidating the person taking the' exam, causing the person to become more confused - Examiners walk into the room and say " Assume the position". q
- 4. Multiple choice answers - giving two correct answers'- must choose between the most correct answer.
- 5. Asking questions on written exams that should be asked on oral exams.
- 6. Asking questions on operating procedures that are not required to be memorized.
i.e. a) Cautions in-procedures b) Notes in procedures c) The basis for procedures d) Questions from B Sections (operations manual system descriptions) e) Asking questions in general on material that is not required to be memorized. We have enough material that is required to be memorized. For l
the remainder, if you want to see if the person )
l knows where to find the material, ask it on the !
oral exam.
Grading of written exams also needs improving. Examples listed below:
A. Not asking for setpoints, taking off points for not ,
including setpoints.
B. Marking right answers wrong because the answer did not ;
contain the EXACT words~that you'want, yet_the answer j contained the information you were looking for.
l l
1 l
l-
Operator 5 Page 2 C. Giving specific point values, for " key buzz words" and
" phrases". If an answer contains the information asked, but does not contain the key buzz words and phrases, full credit should be given.
D. Examiners do not have plant specific knowledge'and are basing their answers or prior knowledge from other plants.
i Simulator Exams:
1 A. Unrealistic scenario's - charging pmp stopping & starting all by itself - no operator action. This happened when the operator was not looking at the pmp. Pumps don't do that. ;
l B. Requiring the person taking the simulator test to draw flow l diagrams during the simulator exam - distracting the person's attention.
l C. Having instruments fail one after another so that you don't have enough time to complete all the required tasks on the .
first failure before the second failure happens, and then ]
fail the person because he did not properly deal with the l first failure. )
)
D. Requiring the person taking the simulator exam to "think out i loud". We are operators not actors. If you want us to be j actors - require us to take acting lessons. If you want to j know what we are thinking of ask us. q l
E. Simulator exams run too long. Sometimes as long as 10-12 hrs. 1 A good examiner should be able to tell if a person is a good j operator or not in 4 hrs. If the examiner bases his decision j on answers received in the last 2 hrs. of a 12 hr. exam, this is not fair to the person taking the exam.
1 i
ORAL EXAMS )
A. What useful purpose do these questions have?
- 1. Why is the water in the SFP blue?
- 2. Whe does it appear that the Far end of the SFP is shallower than the near end? Is refraction that important?
m_______________ __
1 1
, 1 l
Operator 5 -
Page 3 l
1 ORAL EXAMS (cont.) ,
B. Examiners are asking us to find out why a switch will not i operate by using electrical diagrams of that switch. We are not electricians. What we should know is that if the control room switch will not work, that we should send someone to perform the task manually.
COMMENTS IN GENERAL To justify your existence, I feel that you must have a quota to fill, thus failing a certain percentage of people per plant. Look !
at the statistics, they reflect this feeling. By writing ]
l nitpicking questions, you can create an exam that nobody can pass.
l If you have concerns of problems at a specific plant, require that the plant correct these problems. Don't take it out on the people that operate the plant. Would you like it if your i livelihood was taken away from you over a problem that you do not i have direct control over? That is what you are doing to the i people who have an operating license. One last thing. Do not l apply rules on the commercial nuclear industry that you apply to l the Nuclear Navy. We are not in the military service, If you continue this practice, they will be calling us land locked j submarines. !
l
, , j
.l i
l Operator 6 la
- 1. Requal exams are not getting at the real issue, which should be "Is this person a safe operator." This can be ascribed f to: 1
- a. Exam format - 50% of the SRO written exam is !
administrative and procedural detail, which has very )
little to do with the operator's ability to operate -
safely. Further, it can better be checked in an operational or walkthrough test.
- b. Misuse of KSA's by NRC - there are two issues here. One is that whereas a topical area (e.g. Nuclear Instrumentation Detectors) may warrant a large KSA number, selection of a trivial point within that topical area (e.g. " explain the function of the pulse shaper in i N31") is usually the way the NRC writes questions. The 1 l
second is that by writing a question to a single KSA ;
l rather than an integrated plant response, a question of j j a very narrow (rather than broad) nature can result. j i
- c. Objective / legal questions - In order to be sure their tests are (1) easy to grade and (2) can withstand a formal appeal, the NRC has gone to questions which t require short answers, fill in the blank, etc. While j this can be used by an educational expert to get at a j deep understanding, it rarely, if ever, is used in that i way by the NRC. Instead, the NRC exams are often l exercises in trivial pursuit, which have nothing to do j with reactor safety, and are usually inconsistent with j plant directives on how to operate the plant. For example, a plant administrative instruction may. require use of normal procedures "in hand"-performance of steps !
from memory is not allowed. The NRC will go right j ahead and ask " List the interlocks that must be made up to start a reactor coolant pump."
The alternative is to ask "what if questions" which may be hard to grade on a written exam, but are not hard to grade on an operational exam, and they certainly are more closely related to the question "is the reactor i operator safe?" .
I i
- d. Requal exams should test operating ability and experience, which are the true test of whether an operator is safe. Initial license exams need to test how much of a text is crammed into a head because there j is no experience to test. However, by making requal i exams similar or identical to initial license exams, the :
NRC proves that they haven't grasped this obvious l
)
t I
_ _ _ _ _ _ - _ _ 1
~-
4 a r_ .... e i mer s.
j te , ,
H
,0perator 6: .
Page 2 f point. 1s..the guy who. crams a lot of trivial facts'into his head.and passes his license exam'a safer operator, than the guy who has operated.for;,10 years, and deeply understands operations but has' forgotten a lot'of-the
.trivialnfacts? Obviously:not.- So why'is'the'NRC trying to drive that experienced guy.out of the control-room?
- 2. Requal exams aretraising'the stress level of.all operators, which has two very serious consequences: it causes more mistakes in the' control room and:it drives experienced people out of the control room. Both' contribute to the' obvious conclusion - NRC administered requal exams make the industry less safe.. ,
- 3. If the NRC insists on participation in the exan process, they'.
must. restrict themselves to: j
- a. auditing / proctoring utility-written exams. . This is because.the NRC does not. understand Systematic, Approach- 1 to Training.
b.
~
providing simulator exams'only.. The' simulator is the bottom-line test of whether an operator is safe.
- 4. NRC examiners do not have standards related'to operator-safety. Most of the written questions they- ask 'have nothing to do with operator safety. On simulator exams, they pass or fail people on such. things as personality, ability to grope through cumbersome procedures,-ability to find the- ,
For, 1 status of work requests and. process paperwork, etc.
~
example, one very good reactor operator failed a recent operational exam because he skipped a step in a procedure that would have allowed him to terminate safety injection in an expeditious manner. The consequence was that the pressurizer went solid and the simulator,model blewup~. -;
Failure to expeditiously terminate SI~, at'our plantsat least,- l does not place the plant in an unsafe condition.- The examiner, therefore, could not show that-the-RO'sLactions placed;the plant in an unsafe condition or could have led to ..
an unsafe condition. How then,.can the NRC examiner 1 justify-failing the man? The answer:is that the NRC examiners-do ,
not try to assess whether the man'can operate-safely. .
Instead, they test whether he communicates well, whether-he.
demonstrates verbatim compliance to procedures, and whether-he knows the' paper path.in the SS's office. ,
- 5. Examiners.aren't very well qualified to do their. jobs. _The NRC commissioners want to get aJdegree in the control room'to add perspective and technical expertise. Most examiners.
possess no perspective, little technical expertise'and no degree.
(
i
.~.- :. -
I e . .
I Operator 7 1 September 7, 1987 The Commissioners and NRC Staff Nuclear Regulatory Commission Bethesda, Maryland Gentlemen, First, I would like to thank you for the opportunity to express j my views to you on Operator Licensing. It is encouraging to see 1 i
you, willing to listen to the problems associated with obtaining and maintaining an Operator License. i 1
Operator Licensing and requalification is a condition of my j employment. My family, my livelihood, is dependant upon my ]
passing of requalification exams. The difference between passing i and failing can change my income by thousands of dollars. The passing or failing of a requalification exam by my coworkers can change the time that I am required to work by hundreds of hours.
i Consequently, it is part of my job that I take very seriously because of the severe impact it can have, and does have on my life and on my family.
Obtaining and maintaining a license is not an easy. task; I am sure that anyone who has ever taken an NRC license exam will agree. It consumes many hours and causes many anxious moments. The company I work for makes an attempt to compensate for the time and the' !
stress involved with licensing, but the premiums they pay do not fully compensate for the personal sacrifices . involved with )
licensing. In the last year we lost four licensed operators, two ;
of whom had senior licenses, to a garbage-burning plant. Four men who between them had more than fifty years of nuclear plant experience took a cut in pay to be out from under the hassle of licensing. We would have lost more people if more positions would have been available at the RDF plant.
The problems associated with licensing and requalifications come from many different areas. In the area of NRC administered requalification exams the " random selection process" appears at times to be not all that random. During NRC administered exams last March some of the on duty operators asked one of the ,
l examiners about which operators would be selected for an upcoming NRC administered " audit" exam. The examiner jokingly looked at the security badge of the operator and replied, "well here's one name," and looking at another operator's badge said, "here's i another." Of the thirty-two operators that were eligible to take j the June NRC " audit exam" seven out of the eight names' randomly i chosen were in the control room that day. Coincidence? Perhaps, l however a number of us remember back a couple of years ago when we had our first NRC " audit exam". At that time an NRC Region III
a , .
Operator 7 -
Page 2 administrator, now retired, mote or less told us that the names of the people who were to take the " audit exam" were randomly drawn from a hat. He said that names were drawn until they go the names they wanted. In light of the past experiences you can see why there is some question concerning the random selection process.
My last NRC administered exam occurred in March of this year. I recall my simulator exam as being more of an ordeal than an exam testing my ability to safely operate the plant and protect the health and safety of the public. The simulator exam began at 8:15 a.m. and concluded at around 2:30 p.m., during that time we were allowed two five minute breaks. The scenarios used for the exam had events, malfunctions and failures, occurring on the average of one every five minutes. By the time we had recovered, ;
or partially recovered, from one event we had to deal with ;
another problem. All of the scenarios contained multiple !
casualties that were not always interrelated. The scenarios were set up in such a format that if the operator deviated from the scenario the examiners became lost. An example of this occurred when we were given a steam generator tube leak. We were able to respond fast enough to reach a stable plant condition without meeting any auto or manual SI actuation criteria. Our make-up flow via normal charging exceeded break flow, so no SI was needed or required. This action was carried out using the appropriate .
procedures; it, however, was contrary to the examiners' scenario l which called for us to be in an S1 condition. Although all actions taken by us were within the scope of our procedures and past plant practices it resulted in some confusion amongst the examiners present. l At times the NRC exams seem to become not only a test of the license candidate but an examination of the facility's procedures 1 and practices. At times it becomes apparent that any disagreements between the examiner and the facility are reflected in the exams given. I am sure that most facilities have procedures or follow practices that could be improved upon.
However, is the exam process the appropriate forum to air these disagreements? Should a facility modify its practices or its procedures solely for the exam process? License candidates at times get caught in the middle on differences between the examiner and the plant or training section concerning operating philosophies.
At times the NRC exam becomes an overriding concern when discussing procedure changes. Changes that could aid the operator or enhance an operating procedure, are shied away from because of the fear that they may be opening a Pandora's box of exam questions. Simulator training at times becomes based not on how things are done at the plant but instead what looks good for l the purposes of the exam. The paranoia generated by past exam
v: ..
's , ,
Operator 7 -
Page 3 experiences has reached such proportions that the candidates are not sure what they are being tested on. When given a ATWS event on the simulator, as the operator tensely waits for the reactor trip breakers to open, the question that runs through his mind is, am I being tested on my ability to perform the ATWS procedure, or is he checking to see if I know what breakers on the control board feed the rod drive motor generator sets? In the past we have seen this particular event work both ways, and one response is not necessarily correct, especially if the examiner is looking for the other. Different examiners put a different emphasis on what they feel is important. Questions asked are influenced heavily by their backgrounds, one examiner may stress electrical relaying, where as another may stress instrument and control, and i still another reactor theory. It is becoming increasingly difficult for someone going up for a license exam to deal with all the intangibles.
NRC administered requalification exams add to an ever increasing list of requirements imposed on a licensed operator. In addition to existing in-house annual requalification testing, INPO ,
accredited requalification training, 10 CFR time requirements on l' requalification training, SALP inspections and reviews, NRC audited drills, INPO inspections and reviews, and NRC resident inspectors, the question that crosses my mind, is one of, does this added requirement effectively enhance safety? I think not.
Now in addition to maintaining the health and the safety of the public 1 can wonder about when my name will be " randomly drawn" ,
and if I can pass yet another exam. !
Respectfully Submitted, Dennis Herling
. . . I
. j i
l l
Operator 8 The main problem with the examination process, as I see it, is j What I'm this: We don't know what we are going to be tested on.
suggesting is an NRC commitment to use an exam bank if we go to the j trouble of providing one. In the past, the NRC has indicated I they would use one, but we find out that means 0 to 2 questions i out of 75 to 80.
l Use of a validated exam bank has several advantages:
- 1. Self study - it provides the RO and SRO with an excellent study aid. A shift supervisor can use the exam bank to generate weekly homework assignments for each of his licensed crew members to work on during slack time.
- 2. Finite boundaries - At present, the NRC is sent roughly 42 volumes of material from which to write questions. In 1 addition, they are now asking for lesson plans, which will be an additional 5000 pages; 'We have come to expect the NRC to write exams which require rote memorization of the details from that incredible volume of material. The memorization task is hopeless and as a consequence, many RO's and SRO's do not know where to begin when they get time to study. An exam bank would set clear boundaries on what must be learned i i
and eliminate rote memorization of useless detail that now shows up on NRC exams.
- 3. Quality of questions - We have seen the same questions asked I as many as 10 different ways, because the NRC does not want l to repeat the exact wording of questions. Eight of those ten i l ways are ambiguous and so is the answer key. !
l We have heard in the past that the NRC will use an exam bank if we I produce one, but that message doesn't get to the individual examiners. '
l
' , ,,. m ..- '.
/' 1 1
1 Operator 9 .
t From: George Mans, Independent Consultant and Former NRC Examiner o, -
- l. ,..
My basic lancern~with the examination process _in 1987 is that examiners are incompetent. .The whole exam is often a collection
, of "ookies" - obskure , nitpicking points. The real. heart of'the
!, knowledge base that a RO or SRO should have.js not tested. s i
NRC examiners oftqn ask, theoretical questions that are much deeper than the superficial answer key'would imply.- Further,.it's clear to me from reviewing answer keys with NRC examiners, that they often don't understand the issues they're asking questions about.
Another concern I have is with the. continuous escalation of -
material volume. If.the NRC asks a question that is too deep, training must expand it's program to teach it.
There are usually many acceptable alternatives that an operator can take in response to procedural questions. Yet,- because i examiners don't understand all those alternatives, they only allow'one very. specific answer, which they:found on page 17 of .] ,
some procedure. The result is that in~my-opinion there is at i j least 10% error' margin in the grading of.;an exam. 'If I l understand the material fully that/t am/ examined on, I probably.
won't score higher than 90% because'the examiner doesn't have my .]
l ' understanding, or because,the examiner asked ambiguous questions. 1 My viewpoint is that the NRC examiner should restrict himself to writing a job related exam, and every question should be related l
to the operator's ability to protect the health and(safety of the public. Toward that end, the NRC has to enthblish i standard which represents the minimum level of knowl&dde.necessary to
~
protect the health and safety of the public; 4
I would like to give you an example of a q/uestion
, that showed up on two different NRC exams:
d "The point at which the heat transfer coefficient is at its maximum value.is called the departure from nucleate boiling.
TRUE OR FALSE 7" .
One NRC exam answer key says true, the other says false. I have 3 numerous other examples in my files that illustrate my. 1 fundamental point'- NRC examiners are incompetent.
Please feel free to contact me for further discussion. I am sincerely interested'in improving the exam process.
i 1
~
l a
L I a
~
ji j '
~J (..
e Operator 10 ,
September 8, 1987 i t
/
I recently passed an Imc administered Senior Operator regaalificatice, exam i (while holding a two year license). While I am pleased,to have passed it, I do not feel that the exam, particularly the written portion, was a true acasure of try ablities as a safe and efficient operator. I have been in nuclear power operations for nearly 27 years now. ly current imc license is for a PWR, but I previously held a Senior Operator license at a WR and was also a certified Senior Operator at an experimental HIGR .
First I would like to comment en the written exah process. While it still needs same refining, I feel that the Knowledges and Abilities Catalog, NUREG 1122, was a major step toward the goal of producing more standard and more j meaningful examinations. NUREG 1122 has not eliminated trivial and chscure gaestions from being asked on a subject listed as important. Sometimes questions are asked that a job and task analysis shows is not applicable. j Sometimes questions are asked that have not been validated. Questions are '
asked that require answers fram rote memory when the operator on the job does not need to have these answers remarized. Conversely, same questions about evoultions that an operator must have cammitted to memory are not asked. Questians are asked that require detail in excess of what is appropriate for a comprehensive exam. Same questions asked scald be appropriate if they were open book qaestions instead of closed book gaestions. While most questians asked do not have these problems, even a small occurance rate tw-= signifiannt when the passing grade is 80%.
Requalification ext.minations should corcuhate on requal subjects. Mine did not have a single regaal type question on it. I submit that these problems can be helped by the follow four approaches
- 1) Continue to refine NURDG 1122, with industry input. Train examinns in its use to produce valid question related to the operators job en it exists at his/her station.
- 2) Am=_11 ate these validated questions and answers in the question tnnk that is kept at INEL and require that examiners use the gaestion bank as a source for at least 80% of their exam. Theintasbryshouldassistinthis effortbysupplyingquestionsandanswerstothelh.
,l'
1
- . ' ,* +,,*
- f' * ' 3) The ' industry is well on its way to full implementation of a systematic' -!
.. approach' to trainirq. ' Simularly, the NRC needs a systematic aproach to examining. Questions should be directly related to the candidates specific i
position at his/he2. site.
l
- 4) Examiners who have commercial nuclear experience, preferrably including ;]
havin;pheld an Sao license or certification would be most helpful. .
i I strongly support the increased emphasis that the simulator exams are ;
j receiving. I felt that the simulator exam I received was fair and conducted 4 .[- .
T.j} professionally. However I have seen c.u w.nt on others that were not. The major concerns about same recent simulator eams are that many questions were asked of the candidates during the exam and that an inoridinate amount .. j of " malfunctions" vare presented to the candidates and at too high a rate.
l -l I'm sure that the industry and the NRC have the same goals in the area of operator examining. 'Ihat is to assure that plants are operated by people who will protect the health and safety of the public. Secondarily, the ;
industry wants operators who are efficient as well. With the amount of !
resources utilities expend to produce operators of this type, the industry has a right to expect exams from the NRC that will fairly separate safe -
operators from those Wno need improvement. The individual license cardidates who also invest significant resources of thier own deserve a i
1 fair and resonable assessment of their abilities. At times in the past, i l they seem to have been subject to the Whims of individual examiners. A consistant examining process should produce a consistant product. A joint effort between the NRC ard the industry holds the most promise of ' success in addressing all of the issues.
a s
_.____..-_____m__ _m__
e 3 i
.1 i
i Operator 11 l
1 have no constructive comments on the exam process, but I would j like to express to you the feelings I havs after failing the 1 NRC administered SRO requal exam in June 1987.
I passed both the simulator and oral portion of the exam, but failed section 7 of the written exam. It was the first exam I've s failed since the first grade. I have been licensed for 8 years, first as an RO, and for the last 2 years as an SRO. I was selected to be an instructor after 5 years as a reactor operator, because I was a good RO. J First, I am very worried about my family's future. The NRC is saying I'm not qualified in the only trade I know. Like our other experienced people, I will be gone from this job as soon as I can find another one that is roughly comparable in pay.
1 1 don't understand why I am being driven out of the industry and ~
particularly the mechanism being used. The NRC examiners I was exposed to don't know nearly as much-as I do about reactor operations. The exams I've been seeing are not a test of a ;
reactor operator's ability to operate. They are also not a test )
of a reactor operator's understanding.
1
)
_.___________________________J
1
- fLibe.;ldi.LT.diC2.tu I
NUC1. EAR M ANAGEMENT AND RESOURCES COUNCCL 1726 M Sucet. N W.,
Swte 700 Wuhington, DC. 20(06 (202) 872 1260' September 22, 1987 j Mr. Richard W. Starostecki Associate Director for Inspection and Technical Assessment Office of Nuclear Reactor Regulation l U.S. Nuclear Regulatory Commission '
Washington, DC 20555
Dear Mr. Starostecki:
Thank you for providing the opportunity at the public meeting September loth for industry representatives-involved directly in operations and training to comment on their individual experiences and concerns with the operator requalifi-I cation program and implementation of 10CFR55. It seemed evident j at the meeting that the NRC recognizes and shares some of our i concerns regarding the impact on the operators, on the training programs, safety.
on plant operations, and the combined effects on plant ;
-)
i
'We are pleased that the NRC staff is taking some immediate I
steps to improve the situation and we look forward to further j l
interface with you and your staff on how to improve the operator 1 requalification process. We are forming a NUMARC Working Group to continue to work with NRC on this and related issues.
Enclosed for your information is an edited compilation of written comments received by NUMARC from utilities that did not q have the opportunity to speak at the September loth meeting. i We have attempted to group the comments into appropriate categories but we recognize that there is some overlap. These comments l
reflect opinions of individual utilitiqs. t 1
u I
1 l
4
- =.
7 m . w .[ n' .
.'Mr. Starostecki September.22, 1987 i
Enclosed also for your information'are written comments from the Prairie Island Nuclear Plant operators who asked that they be provided in.their entirety. j
_a l If there are~any questions on this information please call Tom Tipton, . Director of. the Operations, Mant gement and Support' Division, or me. We look forward to workinct with you .and your staff on this very important issue. j
- t Sincerely,
- s. n ce F. Colvin xecutive Vice President l & Chief Operating Officer l
JFC:bjr l Enclosure J cc: James Taylor Jack Roe l
l l
I l
1 l
l l
_ _ _ _ _ _ _ _ _ _ - _ _ _-_ _ _ _ _ _ _ - _ _ _ _ _ - _ _ _ _