ML20032B171
| ML20032B171 | |
| Person / Time | |
|---|---|
| Issue date: | 09/25/1981 |
| From: | Rehm T NRC |
| To: | Collins P NRC |
| Shared Package | |
| ML20030B696 | List: |
| References | |
| NUDOCS 8111050029 | |
| Download: ML20032B171 (15) | |
Text
,
._c._,~
_L _.w.2...
~.._- _<
,__.-u i
r t
j
/' "*%p,,
UNITED STATES g
5 NUCLEAR REGULATORY COMMISSION
[
g WASHINGTON, D. C. 20555
/
September 25, 1981 7
NOTE T0:
Paul Collins i
Paul:
s-Revise the memo as marked-up and replace with the t
following paragraph:
We still have big problems in this area.
This response is not good for the long term but it is about all we can do at this time.
For the long term, we have underway a large I
effort to review the entire function of qualifying and requalifying operators, including the contents and methods of examination.
Also delete the enclosures to the OLB Action Plan and retype the plan accordingly.
r i
Tom Rehm l
8111050029 811006 POR COMMs NRCC i
CORRESPONDENCE PDR m.
,,,, [
TO: Name, oNice symbol, room number. -
In Date Ming, Anency/ Post) y
.Q
]4.lI]...<. 1 i
2, I j AL-n
/.,
r J AG M
[ VT/'
k Aiu ail.
~/
a Ac5on File Note and Retum -
Per Conversation
),
ARq tied
~ re m y'
c m,.t.
S**Me t / h6 '
Comment 8*
Signature KL/
_ Mination
(
(]l [
REMARKS w
&WA' CG y rs
. f,4r 5
//'1Pg,7
=O/5'4. & f 4 f f/M car CL c $*t f( Cf.
"N'?
/ADGESS 7A' h 7 4ft % f.?,
Anacai-Ar4 gg3a a rs n,so y cpygggst,,5 f fiv' c
' ~ C,*'
/MM W
NY
,4<y~)ssv gg Cow vnM SA W^'
argi) p.c nrM/w.
,. en DO NOT use this form as a RECORD of approvals, concurrences, disposals, clearances, and similar actions FROM: (Neme, org. sym Agency / Post)
Room No.-Bldg.
Phone No.
a i
.t
~
5041-102 opygONAL FORM 41 (Rev. 7-76) /-- d
- us. cm. inee-ni,saw g=c,rtgigg,
~
e e
.{,
?
E g
y u
DISTRIBUTIO;!:
OLB Rdg RVollmer ED0 Rdg NRC PDR PCollins L PDR SHanauer Courtesy Copy JKramer 8ho /d7Jf HDenton ECase WDircks itEft0RANDUf1 FOR: Chaiman Palladino HThomson DEisenhut FRON:
Willian J. Dircks TMurley Executive Director for Operations'h RMattson
- KCornell
SUBJECT:
OPERATOR LICENSING EXAMIf!ATIONS,/TRehm VStello
/
BSnyders This is in reply to your request to conduct an'in-depth review of corrective actions to inprove the administration of operator exaninations.
/
As directed by the Director, NRR, on July 24, 1981, we have adopted new procedures to assure adequate NRC proctoring of the,sritten exaninations.
/
On August a,1981 the ftiyector and Depdty Directer Division of Hunan Factors Safety net with nenbers 0.f the Operator Licensing Branch and othars cognizant of OLB procedures and practices, to' con 6sct an in-depth review of the admini-stration of examinations,' including preparation, adninistration at the facility, and exanination grading.
Our recent review revealed several areas for improvement of the administration of exaninations, as well as,those aspects dealing wi.n practoring the exeninations.
These areas included prepa, ration of the exaninations, arrangenents with facility nanagement, detailed inst, ructions to the applicants, and QA prograns for review-of the exaninations and,results.
In addition, we reviewed the structure of the,
operator and senior operator written exaninations so that they would be more relevant to the individuals' job and safety responsibilities.
Enclosedisacopy/
of our action plan for improvements in the adninistration of exaninations. Page 3 of the plan addresses adequate proctoring and spacing betweca particiriants. Details are included in Section D and Appendix A of Exaniners' Standard ES-201.
/
Additional ihdepth review of requalification prograns, training of exanir.ers, and content of exaninations will be scheduled in the near future.
Willian J. Dircks Executive Director for Operations
Enclosures:
See next page d/\\d
/
..... /..DH F %i...j.....
Di
/NRR OLB" %3 Di r.
E.D. 0. 4.....
orner>
P..C. o,........ s. /.b..l..
..S..H_a na u.e r[.. E.Ca.......
.H l ton..
WDircks Su a= >
...g.. / 81.......]./g../. 81..
=>g./.81......
J
.7./g../81
/ /81 CCRC FORM 318110 80) NRCM O240
/
OFFICIAL RECORD COPY 4usceO i.8o-329 2,
j
e.
,n. e w,y /
Q W
s t
Enclosures:
(1) OLB Action Plan (pg. 1-10)
(2)
ExaminersStandardES-108(pg.11-13)
(3)
Exminers Standard ES-201 (pg.14-33) including sample facility letter regarding arrangenents
/
(4) tiemo to All Examiners (pg. 34-35) //
(5)
Letter to Licensees regarding simulator exams (pg. 36-39)
/
(6) Examiners Standard ES-109 (pg,. 40-42)
(7) Proposed operator and
/
senior operator license /
writtenexaminations(p'g.43-61)
/
cc: Comm. Gilinsky
-[
Com. Bradford Comm. Ahearne 7 Com. Roberts f SECY
/
OPE
,/
00C
/
H. R. Denton E. Cas9
S. Hanauer J. Kraner D.pVassallo Contact':
S. H./Hanauer 49-29595 t
"'c'>
DATEk NRC FORM 318 t 0 00t NRCM O240 OFFICIAL RECORD COPY
8 -329 2
. 5 ENCLOSURE 1 OPERATOR LICENSING EXAMINATIONS ACTION PLAN A wide ranging discussion of the operator licensing program was conducted on August 4,1981 by the Director, and Deputy Director, Division of Human Factors Safety with members of the Operator Licensing Branch, including consultants, a former member of the branch, and a represe fative from Analysis and Technology who had been the project manager for a s d,r of the operator licensing program (NRC-03-80-117).
The meeting was pr te d by the recent investigation into cheating on examinations that rev aled 11 adequate proctoring of examinationa.
4 The agenda called for a detailed discussion of present pract ices in examination procedures, including development of the exanyination, adminis tration at the facility, grading of the writtens, and orals /and feedbad to licensees. A
/
critical examination of the above was under;taken, followed by a discussion of
/
methods to improve the effectiveness and efficiency of the examination process.
i Subsequent to the meeting, we revised thef fom of the operator and senior operator written examination.
/
I Listed below are the items of concern, conclusions, and the recommended actions.
Actions are addressed as short tenn, nd long term.
Some actions directed toward eliminating cheating have already been implemented; most of the rest are short tenn and can become effective with appeoval of the action plan.
i
=.
J A.
Reparation of Examinations a.' Members of the Branch indicated that, at times, they were having difficulty obtaining material necessary for preparing the examina-tions in the proper fonn and in a timely manner.
We have developed a fann letter with appropriate attachments, addressed to corporate management that sets fo h the utility's responsibility regarding i
material t be furnished, and other administrative items in orde to assure proper preparation of the examinations and dministration thereof, i
{
b.
Although examiners submit e inations for review to the Section Leaders and/or Branch ef, there is no formal QA plan for review of the examinat ns.
4
{
We hau developed an Examiners Sta dard (ES-108) that details a QA program for examina' tion review i
l prior to their administration.
c.
Prearrangements with facility management do not det il space requirements and instructions regarding required condu of candidates during the administration of examinations.
i ;
4 We have prepared a revision to Examiners' Standard, ES-201 that will provide instructions to the examiners regarding pre-crrangements, administration of the examinations at the facility, instructions to the ca idates and proctoring requirements.
Schedul l
l I
to Examiner /' StandardlES)201. -
Ex hmine s' Stalndard,./ /
/
/
^,R a.
Fom let er attache s
l b.
-108 attached.
/
i Stapdad,/ES-201 attached [
c.
R vis Exa iner
[
/
/
B.
Administration of Examinations a.
Written Examinations (1) There is a lack of detailed structions to the examiners regarding proctoring, instructi ns to candidate, reference material provisions, control of pa er, allowing people to leave the roa1, and other administrat ve matters.
We have incorporated the instructions nto Examiners' Standard ES-201.
(2) There is no detailed procedure for facility managem t review of the examination and concurrence in the applicabilityxof the examination questions and answer key.
1 l
s l
l We have incorporated the instructions into Examiners' Standard ES-201.
x
-Schedule a9(4)V Revis~ed' Examiner '. 'fanda'Fd s 201 attached.
g
\\
(2) Revised Examiners'\\ Standard ES-201 attached.
\\
1 B.
Oral Examinations (1) There is a lack of unifonnity in recording the results of the oral test and a lac of quantitative criteria for evaluating the candidate!s perfonnance.
\\
For a short tenn solution, we will request all the examiners to review Exa iners Stand?rd ES-305.
I Instructions for Completing O'r 1 Examination Notes and to submit their recommendations for improvement
\\
to the appropriate Section Leaders. The Branch Chief t
and Section Leaders will develop additional instructions for completing the notes.
1 (2) There ace no Examiners Standards that address the si lator portion of the oral test.
i t
Prepare letter for distribution to all licensees that scopes simulator examinations for both plant specific and non-plant specific simulators.
The guidelines i this letter will be the basis for the first several ets of simulator examinations.
Based on the above ex.ination experience, we will develop an Examiners' tandard that addresses simulator
\\
examinations.
Schedule (1) kemorandum to all examiners attached.! eplies to be R
- C._
twenty days after issuan e/
Examiners' recei be revised fifty da /s after issuance.
Standard (2) Letter attached. Conduc xaminers' War hops for each N
Section, develop new E niners Notes; Janu 5y 15,1982.
N Test and revise Exao ners's Notes; February 28 1982.
Issue Examiners' andari; iarch 31, 1982.
N
\\
i C.
Grading and Final Review 4
a.
OLB has consistently honored requests for examinations, \\
s including those for reexaninations for individuals who
=.
.1 failed their original examination.
In order to accomodate these requests, OLB has deferred grading of other examinations until several sets of eE inations have been administered.
This has on occasion resulted in long delays between 'administra-tion and final action on an application.
We will review the prey ous twelve month work load to determine the nu ber of assignments that might have been avoided t rough better scheduling.
Based 'on the review, we wi i develop a position that will ninimize the number of trips to a facility in a twelve month period, so as to provide time to grade examinations immediately following their administration.
b.
Graded examinations are reviewed by the S ction Leaders and i
Branch Chief prior to issuance of licenses r denial letters i
l However, this is no formal QA procedure that indicates the scope of the review.
We have developed an Examiners' ~ Standard 4
(ES-109) that details a QA program for i
i the review of graded examinations.
N 4
I i.
s Schedule
/
a.
Complete review of previous / schedules: November 30, 1981.
Develop ton paper an'd forward to all facility licensees
/
(
and applicants: a uary 1, 1982.
\\
b.
Examiners' Standa ES R09 attached.
\\
D.
Improve Fonnat and Relevance of the Written Examinations NUREG/CR-1750, Analysis, Conclusions and Recommendations Regarding Operator Licensing, comments from industry, and our own observa-tions have indicated that the written xaminations should be more relevent to the job responsibilities o operators and senior operators. There are several studies un erway that will use job task analyses and other techniques to develop more appropriate
/
examinations.
For the short tenn, we have developed a examination that changes the format and content and educes the time required to complete the written exa ination.
t Schedule Examination proposal enclosed.
E.
Long Term Action Plans a.
Preparation of Examinations (1) OLB attempts to generate an original examination each time a set of examinations is prepared for s,
f i...
a facility.
Of necessity, questions are repeated with, or without slight variations.
Cor siderable time is required to 6 Jt an examination
- ogether, including rewriting previously used questions.
Insufficient time is available to revielw facility I
operating history, LERs, abnormal occulrences and IE Inspection Reports.
Requirements for and development of a totally new examination appears to be needed and LQB of DHFS has the lead with coordipation from OLB.
In the interim, we plan to dev lop a compre-hensive data bank of generic nd plant specific questions and answer keys to facilitate more unifonn and rapid preparatio; of examinations.
1 Examiner preparation time then can be concentrated on review of procedures, ope rating history, LERS, and IE reports.
(2) Many facilities are very similar; i.e., BWR 4s and Westinghouse four loops. However, they are treated as separate, different plants when scheduling examinations end preparing the examina-tions.
This adds to the time requirdd for overall administration of examinations. -
3...
We plan to investigate'the feasibility of identical /
similar written examinations being administered concurrently for similar facilities at the same time to avoid compromising the examinations.
Tchedtrie, (1) Detennine infonnation to be stored and retrieved, and the i
best automated equipment to serve ou /
purpose. January 1, 1982.
S o e material, review for accuracy, develop procedures 1, 1982.
for retrieva nd begin using s stem, May/
(2) Develop listing of s ilar facilities an identical systems within similar plants, ly 30, 1982.
Develop paper for Office Director consJderati of combi ed examinations August 1, 1982.
b.
Administration of Examinations (1)
Examiner Standards exist for adminis ering the oral test. However, there are no specif c criteria that assure a unifonn, quantitative evaluation of candidates.
We plan to develop a revised Examiners Standard that specifies minimum accepta )1e performance and knowledge in each area that is evaluated during the oral test and simulator test that will assure a more unifonn quantitative evaluation of candidates.
r (2) Simulator examinations' administered by OLB have been conducted at plant specific simulators in conjunction with the oral test.
The simulators and plant oral tests are nonnally administered by the same examiner. With the advent of non-plant specific simulator examinations, there will be many occasions when different examiners administer the simulator and plant oral tests for the same candidate.
We plan to develop Examiner Notes for the Administra-tion of the simulator examination and oral test that clearly f ndicate those areas examined, so as to avoid duplication.
9 ched h b.
(1) Contingent upon recommendations from TMI, action plan regarding Job T Analysis.
The short i.enn sche'dule is addressed in Item B, Oral Ex inations, pages 4-5.
/
(2) Revise Examiners Notes ba ed on the results of the administra-tion of the first twelve / sets f non-plant specific simulator
/
examinations: July'1, 1982..
-...m m
.m
. m.
m.
vesmMhar 2
=
1 a
n n
< o
.,. - ++g w ~s, w. w.a r *- 3 * -
)
ca e
i
[
FROM ACTION CONTROL D ATES CONTROL NO j
10728
=
com omomN<
f Oa41F534 bJlldd!hV AMOM_E OG ME N T DATE OF DOCUMENT jNTE I4f M R EPLY
[
TO. j PREPARE FOR SIGNATURE 2
OF -
FINAL HEPLvjj h,-Qyfg g
0 FILE LOC ATl6N!
[] E XE CU 1!% E UlHLC TOR l
/O
$[
ormw ria fg vtMc s t oo p i E orHER SPECI AL INSTRUCTIONS OR REM ARKS d
DESCRIPTION
',pratar liceutng examint1995 See Aly a nwa from rem attache) puE th
~
1
, - / -
4'/ r/
6l F E L> D A T A i ! i rml E
- u. g.ymc o o3;y- __ _ __ _ __ _ z]' _a 4
+
Fh a---
N m _o.n n a 1
,, ni u t <,ls r H Y NIJ
[?$ 'ilIj N h, b iPV
,. - + - - - -
\\h
'd U__$jf_b.m ullp_.. _ _ _ _ _ _ _ -
CJrnel } Ucaton la- 'SIEIU-i NC ffy, e
lL Ofrcks case 3 *~ '
. suo m o m (kt In.ti m
i< ' ' " ' ' ' " '
1-y( - _ ' _
8_
.p.
.,g
- ** Y
^
Stell
?.
2:tsenh1_1.. W 1 "", - - - - - -
==
_g
,x,
+
1
- A{ T.SCM
-y-
, > i i r I C n ' t.{f. n,- - > [. T r {:_ _ _ _ _ _ '.
t ng is, v
/ I~
'") l
- }
_-------7----
w, 1 3
,j, g-PPiNciP AL CORRESPONDENCE C ON TRO!
g I-y-
^J b
3, ni 1
4 m
M i
d
-m i
_m
...