ML20217Q200
| ML20217Q200 | |
| Person / Time | |
|---|---|
| Site: | Hatch |
| Issue date: | 03/03/1998 |
| From: | Peebles T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | Lewis J SOUTHERN NUCLEAR OPERATING CO. |
| References | |
| NUDOCS 9803120044 | |
| Download: ML20217Q200 (21) | |
Text
,
I-t March 3, 1998 Southern Nuclear Operating Company. Inc.
ATTN: Mr. John C. Lewis. Training Manager. Plant Hatch E..I. Hatch Nuclear Plant P. O. Box 439 Baxley. GA 31513 i
SUBJECT:
MEETING SUMMARIES - NOVEMBER 1997 NRC REGION II TRAINING MANAGERS
CONFERENCE AND JANUARY 1998 NRC REGION II EXAMINATION WORKSHOP
Dear Mr. Lewis:
This letter refers to the Training Managers Conference conducted at the j
Atlanta Federal Center on November 12 and 13, 1997 and the Examination i
Workshop conducted at the Richard B. Russell Building on January 27-29, 1998.
Representatives from all utilities in Region II participated in both meetings.
L The agenda for the Training Managers Conference is Enclosure 1 and the list of attendees is Enclosure 2.
We appreciate the participation of you and your l
staff and believe that the goal of providing an open forum for discussion of operator licensing issues was met.
Mr. Gallo. Chief of the Operator Licensing Branch. Office of Nuclear Reactor Regulation (NRR) made a presentation on the present status of operator licensing and his slides are Enclosure 3.
During the meeting, it was decided that a workshop on operator licensing examination writing was needed and would be held at the first of the year.
Also, we have tentatively set the date for the 1998 Training Manager's Conference as November 4 and 5.
'I Additionally. I am enclosing our preliminary schedule for FY 1998 and FY 1999.
l-dated February 18. 1998, as Enclosure 4.
Please review the schedule and 1
i supply comments to my staff or myself.
j w
The Examination Workshop was conducted with participation by everyone.
A list of attendees is Enclosure 5.
A standard Job Performance Measures (JPM) format L
was reviewed and comments collected by the Southeast Training Managers l
g (SSNTA), with a final version expected this summer.
Concerns on the i
b a.-
. examination process were collected and is included as Enclosure 6.
These concerns'were forwarded to NRR for review.
During the workshop, we discussed some of the problems with,the initial M"
examination process as it is being implemented be Revision'8 of NUREG-1021.
A discussion of those issues is' enclosure 7.
//
NC
/
M It is our opinion that this conference was beneficial and provided an f
I gg excellent opportunity for open discussion of various concerns about the F a.>
Operator Licensing process, especially the techniques of writing the licensing examination.
lll$llllk$,N,$h n o non i
as a
^
vujokCW g
7
?
(
,L/
l SN0PC0 2
If you have any questions regarding the content of this letter, please contact me at (404) 562-4638.
i l
Sincerely.
Original signed by
'Ihamas Peebles Thomas A. Peebles. Chief Operator Licensing and Human
{
Performance Branch Division of Reactor Safety Docket Nos.: 50-321 and 50-366 License Nos.:
Enclosures:
1.
Agenda for Training Managers' Conference 2.
List of Attendees for 1997 Training Managers' Conference 3.
Mr. Gallo's Slides 4.
Region II Examination Schedules for FY 97 & 98 5.
List of Attendees for 1998 Examination Workshop l
6.
Concerns Expressed during Workshop 7.
Discussion of Workshop Issues cc w/encls:
)
l P. W. Wells. General Manager, i
Plant Hatch Distribution w/encls:
-PUBLIC 1
B. Michael. DRS I
l l
- FFICs RIIsDRs j
sIomAToms 7g j
l~
uams TysssLas j
DArs 3/ J
/98 3/
/98 3/
/98 3/
/98 3/
/98 3/
/98 3/
/98 corn ins no ras no ras no ras no Yss no Yss no ras no orrterAL macosn copy pocussar mams: As\\nATLra.ac j
h I
T-
\\
SOUTHEAST TRAINING MANAGER'S CONFERENCE
]
U.S. Nuclear Regulatory Commission, Region II Atlanta, Georgia Meeting Agenda November 12-13,1997 Atlanta Federal Center Wednesday. 11/12/97 8:00 a.m.
Conference Registration Conference Cerler Conference Room C 8:20 a.m.
Introduction Thomas A. Peebles, Chief, Operator Licensing & Human i
Performance Branch j
8:30 a.m.
Welcome Johns P. Jaudon, Director Division of Reactor Safety 9:00 a.m.
Welcome Bruce S. Mallett, Acting Deputy Regional Administrator 9:30 a.m.
Overview of Pilot Exam Process Thomas A. Peebles, Chief, Operator Licensing & Human Performance Branch 10:00 a.m.
Break i
10:30 a.m.
Examination Communications Ron Aiello, RII Exam Development & Coordination 11:00 a.m.
Examination Security Issues Paul Steiner, RII 11:45 a.m.
Lunch 1:00 p.m.
Resident Review of Training Paul Harmon, RII 1:30 p.m.
Lessons Learned from Recent Exams Charlie Payne, Ril 2:15 p.m.
Break 2:30 p.m.
Examination Questions and Answers George Hopper, RII Examples of questions 4:00 p.m.
Meet with Principal Examiners All 4:30 p.m.
Adjorn ENCLOSURE 1 1
l-I'*
- 2 i-Tinirsday. 11/13/97-l 8:30 a.m.
Recap Tom Peebles l
l 8:45 a.m.
Reactivity Changes and Other Issues Robert M. Gallo, Chief.
L Operator Licensing Branch, i
NRR I
i l
9:30 a.m.
Medical Exam Issues - Conditions -
Charlie Payne, RII I
i 10:00 a.m.
Break l
10:15 a.m.
~ Open Session - Other Issues -
Training Managers 12:00 p.m.
. Adjorn-l l
i-
[
l l
l l
p ENCLOSURE 1
I
</
\\
j REGION ll TRAINING MANAGERS CONFERENCE NOVEMBER 12-13,1998 i
Timothy L. Norris Onsite Engineering General Manager Brian Haagensen PSHA CP&L..
Larry Dunlap BK Supv. Ops Cont Trng Rick Garner HR Supv Ops Trng Tom Natale RB.
Supt Ops Tmg William Noll BK Ops Trng Supv Max Herrell BK Tmg Mgr Scot Poteet RB Exam Team Leader Crystal River - FPC Jack Springer CR Supv Simulator Tng Tom Taylor CR Dir Nuc Ops Tmg Duke Power Garmon Clements CT Human Perf Mgr Camden Eflin OC Ops Trng Richard P. Bugert Corp Ops Trng Spec Gabriel Washburn OC Req Team Leader Charles Sawyer' Corp Sr Tech Spec Ronnie B. White, Jr MG Tmg Mgr E.T. Beadle CT Init Lic Exam Leader William H. Miller CT Tmg Mgr Al Lindsay MG Ops Trng Mgr Paul Stovall OC Mgr Oper Trng Bentley Jones OC Trng Mgr Paul Mabry OC Ops Line FP&L Maria Lacal TP Trng Mgr Philip G. Finegan TP Ops Tmg Supv Dennis L. Fadden SL Services Mgr Jo Magennis Corp Trng Assessment Spec Kris Metzger SL Ops Trng Supv Southern Nuclear (SNC)
J. M. Donem FA Sr Inst Ops. Tmg John C. Lewis HT Trng & EP Mgr Tom Blindauer FA Sr Pit Inst Joe Powell FA Sr Inst Ops Trna Bill Oldfield FA Nuc Ops Trn Supv Southern Npelear (SNC) (cont'ql oace 21 l
l ENCLOSURE 2 1
2 Southem Nuclear (SNC) (cont'd)
Steve Grantham HT.
Ops Tmg Supv
.. Scott Fulmer FA Mgr Tmg & EP Leon Ray VG,
. Ops Tmg Supv
- Virainia Power Frank Winks._ -
NA Spv Ops Trng H. Ashley Royal' NA Supt Tmg Thomas Toby Sowers SR Supt Trng
- Bob Greenman BF Tmg Mgr Dick Driscoll SQ Trng Mgr
. Walt Hunt SQ Ops Tmg Mgr
- James Proffitt SQ Nuc Eng
' Marvin Meek BF HLT Lead inst Rusty Proffitt SQ
ENCLOSURE 2
w eau mm-m
_ :-- medo m.h 4 * *.e.
..w
_,,w
- m,%
. OPERATOR LICENSING INITIAL EXAMINATION RULE CHANGE Region ll Training Managers Conference November 13,1997 Robert M. Gallo, Chief, Operator Licensing $timeh, NRR p
ENCUDSURE 3
's
- -=eem.
me 4
og--
HISTORY SECY 95-75 (3/95): Proposec' change GL 95-06 (8/95):
So icitec volunteers ROI 95-25 (8/95):
Pilot guidance 10/95 - 4/96:
Original pilot exams 5/~ /96:
C9G.9 briefing a
SECY 96-123 (6/96):
3ilot results SECY 96-206 (9/96?:
Pros and cons o
GL 95-06, Sup.1 (1/97): Voluntary o
continuation of pilot process NUREG-1021, Interim Rev. 8 (2/97) o SECY 97-79 (4/97):
Proposed rule o
62 FR 42426 (8/97):
Proposed rule o
THE PROPOSGD RULE
- 3. A new s 55.40 is added to reac as fo lows-
!i 55.40 mp ementation.
(a)
Power reactor facility licensees shall--
(1)
Prepare the requirec site-specific written examinations and operating tests; (2)
Submit t,e written examinations and operating tests to the Commission for review and approval; and Proctor and grade the NRC-spproved site-specific written exemittetions.
1 THE REST OF THE RULE (b)
In lieu of requiring a specific power reactor "acility licensee to prepare the examinations and tests or to proctor a.nc grade the site-specific written examinations, the Commission may elect to perform taose tasks.
(c) The Commission will prepare and administer the written examinations and operating tests at non-power reactor facilities.
-.g tee $brw h6
=
OTHER CONSIDERATIONS
- T7e NRC wi prepare one exam per i
9egion per :ca enc'ar. year i
- Faciity 'icensees are expectec to use t7e guicance in l\\ UREG-1021 NRC wi approve ceviations NRC wi not compromise statutory responsioilities NRC is committec to maintaining qua ity, leve of ci"ficulty, consistency, anc security
. NRC intends to use its full enforcement authority against persons who willfully compromise an exam in violation of 55.49
BACKGROUND
+ Goal was to improve e"ficiency w1i e maintaining e ectiveness Eliminate re,iance on NRC contractors (except GFE)
Increase facility involvement Maintain examination qua ity and cifficulty Remain consistent with t,e Act and Part 55 Changes should be transparent to license applicants
- InLtial licensing program was not broken
MILESTONE SCHEDULE
0/2' /97:Comnient perioc ended 4/1/98:
Reso,ve comments; revise
{
rule anc' NUREG 021; seek Offine concurrence 4/98:
Brief CBGR and ACRS 5/22/98:
Obtain Office concurrence and c'e'iver to EDO 6/98:
Obtain EDO anc Commission concurrence 7/98:
Publish the "inal rule and Revision 8 12/3' /98: Imp ement ru e anc Revision 8
~
i i'!..
!)
oT T
O p
h h
r F
ee T
S 1 r i
o Pg Y
E 1
r i
x o
/n 9u i i ar l
t 9c 9g o n 1
a t
e a
7e 6h t a 9
m i
s s
9 s
l l
n C
2 5
u g
Y 2
l t
t s
1 e
3 9 9
9 5
7 4
W so 9
9 2
4 7
9 r
1 2
/3 1
4 R
t f
/
/
i
/
t 1
% 5%
8%
5%
t O ao 4
9 3
4 e
p n
2 n
p e
1 e
O a
e 3
5 8
5 p
E 9
8 9
9 9
e x
1 1
0 0
R X
l 3
/9 5
/3 8
r
/
a a
/
5%
8%
5%
aO 1
A m
4 7
4 4
t r
i e
1 n
M g
p p l
1 eu 4
2 8 8
7 4
8 8
9 T
R ns 4 6 3
8 3
2 oR E
9 5
5 d
/
/
/
/
1 t
io 4% 5%
8%
5%
a O S
nn 9
5 4
l 4
g e
U R
2 1
1 L
3 0
3 8
W T
9 9
9 9
9 S
O 6
0 6
6 r
/4
/4
/4
/3 5
R S
i t
a 2%
% 9%
1 1
t O e
n 5
0 4
2 d
0 6
4 n
o 2
1 O
9 n
3 3
8 p
9 89 9
9 9
eS 1
e 5
7 7
/4
/2 6
6 5
. 6 /.m gS 2%
0%
% 9%
raR 1
/
/
1 t O 4
4 1
i 6
n R
9 3
g O
2 1
9 2
9 58 3
8 9
9 9
T S 6'
1 3
0
/ 9 1
i 0
2 oR
/
1
/
/
2% 0%
% 9% %
t 1
a O a
5 4
2 l
7 4
1
- !l l!
e FY99 INITIAL EXAM RESULTS February 20,1998 i
RO SRO-l SRO-U TOTAL Date Plant Chief Pass Pass Pass Pass 0
9/28/98 Sequoyah MEE 4
10/5/98 Harris -
RFA 4
2 3
10/19/98 B. Ferry WFS write DCP 4
4 11/30/98 Oconee &
MEE 6
6 12/14/98 11/30/98 St Lucie &
RSB 15 15 12/14/98 1/25/99 McGuire &
DCP 14 1
15 2/8/99 1/25/99 C. River &
RFA 10-12 2/8/99 3/15/99 Watts Bar &
RSB 7
5 3/29/99 3/29/99 Surry &
RFA 6
2 4
4/12/99 5/17/99 Catawba &
15-18 5/31/99 5/10/99 Farley 2
6 Watts Bar ? 6/99 6
4 8
07/ /99 Robinson?
4 1
1 07/ /99 C. River?
08/ /99 Turkey Pt?
20 9/15/99 Summer?
4 09/ /99 Sequoyah ?
99
'?' disignates tentative No Initial exams scheduled for:
North Anna f
710/18/99 Brunswick-9 candidates I
710/ /99 B. Ferry 4r, 4i, 4u l
710/25/99 Hatch Br?
j 710/ /99 St. Lucie 2 wk 712/13/99 Vogtle-Sr, Si,2u ENCLOSURE 4 l
L
FY 98 INITIAL EXAM RESULTS
[10/1/97 - 9/30/98]
c Fcbru:ry 20,1998 RO SRO-l SRO-U TOTAL Exam PLANT CHIEF PASS PASS PASS PASS Week 10/14/97 St. Lucie &
GTH 6
6 1
1 7
7 10/20 11/14/97 Cr. River RETAKE RFA 1
1 1
1 12/1/97 Summer JFM 8
8 8
8 12/1/97 Catawba &
DCP 2
3 4
5 6
6 14 12/15 3/2/98 Farley RETAKE RFA 1
1 2/23/98 Robinson + 1 op RSB 3
1+1 1
6 retake 4/13/98 Vogtle (Mellen write)
GTH 4
2 6
5/11/98 Brunswick &
DCP 5
3 3
11 5/25/98 w Sequoyah Retake +
LSM 3
3 6/1/98 op RFA RSB 6/29/98 Crystal River MEE 6
6 6/22/98 St. Lucie &
GTH 8
4 8
7/6/98 8/10/98 Turkey Point DCP 8
8 8/17/98 North Anna &
RSB 8
1 6
15 8/31 9/28/98 Sequoyah MEE 4
4 54 28 26 108 RESULTS TO DATE 16 17 5
6 7
7 28 30
'&' designates examinations that will require two weeks to administer No exams scheduled for B. Ferry Oconee Harris Surry Hatch W. Bar McGuire 4
i ENCLOSURE 4
g.
i.
REGION 11 WORKSHOP - OPERATOR LICENSING EXAMINATIONS JANUARY 27 - 29,1998 l
Exam Workshop' Attendees arlie Brooks -
Asst Manager, Ops Trg - INPO s
Frank S. Jaggar Examiner-WD Associates -
l Ken Masker Senior Licensed instructor Rochester Gas & Electric, i
R. E. Ginna NPP Bob Niedzielski Exam Developer-Baltimore Gas & Electric James F. Belzer instructor-CCNPP/BGE Max Bailey Region lli Operator Licensing Examiner CP&L Gregg Lualam LOR - Supervisor-Brunswick William Noll Supt Ops Training - Brunswick Tony Pearson Initial Training - Brunswick R! chard Edens LOR instructor-Brunswick Rick Garner Sup - OTU - Hams Terry Toler Project Tech Spec - Hams Wiley Killette Project Tech Spec - Hams Scott Poteet Exam Team - Robinson Bill Nevins Instruct Tech - Robinson Crvatal River - FPC Alan Kennedy Senior Licensed Instructor Johnie Smith Training Supervisor Jack Springer Training Supervisor Duke Power -
Alan Whitener Ops Instructor Edward A. Shaw Ops Instructor
- Bobby Ayers Ops Instructor-Oconee Steve Helms Training Super Charles Sawyer initial Training - McGuire x
Reggie Kinvay initial Trining Lead E. T. Beadle Nuclear Instructor-CNS
James K. Black -
Nuclearinstructor.QfG Gabriel Washburn ~
Nuclear Instructor-QRS L
Camden Eflin Team Leader - HLP - Qgange b
L (Exam Workshop Attendees cont'd - See page 2) j g
L ENCIOSURE 5 l
I6.
2 (Exam Workshop Attendees cont'd)
EEAL Ivan Wilson-
. Operations Manager Kris Metzger
_ Ops Training Supervisor - St. Lucie 1
- Roger Walker.
Instructor-St. Lucie Tim Bolander Instructor-St. Lucie
- David P. Clark l Instructor - St. Lucie Maria L. Lacal-Training Manager-Turkev Point __
Rich Bretton
' Ops Cert Tmg Sup - Turkev Point Philip G. Finegan Ops Trining Supervisor - Turkev Point Michael E. Croiteau Cont Trng Instructor-Turkev Point Southern Nuclear (SNC)
Joel L. Deavers
. Senior Instructor -Eaday Scott Fulmer Training & Emergency Preparedness Manager - Farley Gerard W. Laska Training Instructor-Farlev Charlie Edmund Plant instructor-Hatch David Gidden ;
. Plant Instructor-Hatch Training Supervisor-Hatch Ed Jones l
Dan Scukanec Ops Trng Supv - Vogtle Fred Howard Plant Instructor-Vogtle l'
~
Viroinia Power Keith Link Requal..... - North Anna Ed Trask Instructor-North Anna Joe Scott Supervisor Operations Training - North Anna Ken Grover
. Senior instructor (NUC)- S.utty Harold McCallum Supervisor Ops Training - Surry Paul K. Orrison Ops Instructor - Surry IVA l
Ray Schorff Instructor-Browns Ferry i
Denny Campbell Instructor - Browns Ferry Bob Greenman Training Manager-Browns Ferry Mcrvin Meek -
Instructor - Browns Ferrv
- A. R. Champion instructor - Browns Ferry Rick King :
. Sr Ops Instructor - Seauovah j
Frank Weller Instructor - Seauovah Phillip H. Gass Sim Instructor - Seouovah Ed Keyser Instructor-Seauovah i
Harold Birch Instructor-Seouovah (Exam Workshop Attendees cont'd - See page 2) i.
i
'd.
3 (Exam Workshop Attendees cont'd)
WA copfd Terry Newman SRO Instructor-Watts Bar Rancy Evans SRO Instructor-Watts Bar
~ Rick O' Rear Sift Manager-Watts Bar V. C. Summer-SCE&G Perry Ramicone Ops instructor Bruce L. Thompson Ops Instructor William R. Quick Ops Instructor l
I i
i l:
t
)
l CONCERNS EXPRESSED DURING THE REGION 11 EXAMINATION WRITING WORKSHOP
' The following is a condensation of the concems received from the attending facilities during the
-January 1998 Workshop on Examination Writing. The workshop attendees and I would appreciate your consideration of the concems during your revision to the Examiner Standards.
i 1)'
Security requirements are too restrictive, considering the limited resources available. Also, more guidance >n minimum security expectations is needed.
(three comments) 2)
The NRC should de velop the sample plan as this would save both utility and NRC resources. (two comments)
.3)
If independent groups generate the audit and licensing exams, some overlap should be allowed. (one comment, also I believe the standards allow this now?)
4)
The K/A catalog contains errors and omissions and should be corrected, or at the least an errata sheet of know errors should be published. (two comments) 5)
If an exam bank item has not been used during the licensing class, the exam item should be considered at ' Tace value" for the licensing exam. (one comment) 6)
The length of time allowed for written exams should be revised to a more l
reasonable period. Does this time also apply to continuing education.
l (one comment, I had commented that the length of time did not apply to requalification exams the utilities conducted.)
7)
The NRC should periodically publish problem areas encountered during the exam process and distribute it to all training managers. (one comment) l 8)
The facilities appreciated the workshop. They want Region il to have another workshop in about six months. The next time they want to concentrate on good and bad examples of written and operating test items and the sample plan. (six l
comments) l i
i i
~
Lf ENCIOSURE 6 l
t
DISCUSSION OF WORKSHOP ISSUES l
During the workshop we discussed some of the problems with the revised o)erator licensing examination process as implemented by Revision 8 of NJREG-1021. The following were three of the principle issues discussed and a summary of the response given by NRC's Region II Operator Licensing staff.
Why has exam development take so many man-hours? Some facilities did
.1.
not fully understand our methodology, concepts and expectations for developing the initial examination such as content validity, plausible distractors and other psychometric issues. The NRC did not recognize the variance across facilities in their depth of understanding. As a l
result, some facilities submitted examinations with the quality lower than expected and these examinations did not meet the standards t
L described in NUREG-1021. The amount of resources required to modify the l
examinations to meet the standards was more than either the facility or the NRC had anticipated. There was general agreement during the workshop that more discussion with the facility examination writers and i
reviewers, such as these workshops, would better align the facilities' original products with the standards of NUREG-1021 and reduce the
. resources required to develop an acceptable examination.
l 2)
Why has the NRC raised the level of difficulty of the examinatio's?
r Many participants felt that the NRC was " raising the bar." We stated that the purpose of the initial operator licensing examination is to test valid knowledges, skills and abilities required to safely carry out duties as a licensed operator at a specific facility.
The examination should be written to a discrimination level not sper' to the quality of the facility's training program, but so that a m u mal competent i
operator, with specific site knowledge and skills, will pass the l
examination.
Therefore, the level of difficulty of the examination l
should not vary significantly from site to site. The concept of i
discrimination validity is that a given test item is written at a level which will discriminate between a competent and less than competent
{
operator.
In some cases, the NRC examination reviews have adjusted the l
discrimination validity (difficulty) in order.to achieve region-wide l
consistency on what is required of a competent operator. We try to L
create an examination such that an applicant who is capable of safely l
operating the plant will achieve a score of 80 percent or greater For facilities that prepare candidates beyond the minimally qualified level, we would ex)ect the average score to be higher.
Historically, nationwide VRC examination scores have averaged approximately 85 percent, which is a reasonable benchmark and expectation for a discriminating criterion-referenced examination.
I explained that I use a mental description of a minimally competent o)erator to decide if the question is one that he/she needs to know and w1 ether the overall exam is targeted for that person.to achieve a score of 80%.
An 80% score on the written examination for a minimal competent candidate does not correlate to an 80% pass rate and we have no goal u
ENCLOSURE 7
a-
-l 2
regarding pass rate. Overall'. we did not intend to change the 'bar' and are reviewing results to ensure our practice meets our intent.
3)
Why have some applicants not been able to complete the examination in the four hours currently allowed? Prior to the current examination revision, we had two actions in the implementation phase.
One was the improvement in the plausibility of distractors and the other was standardizing the percent of comprehension and analyses questions.
In.
the last two years, we have improved our identification of poor distractors.
A question does.not have discrimination validity if the distractors-(i.e. incorrect ans e s in a multiple choice test) can be eliminated by a less than com3etent operator due to psychometric flaws in the question structure. T1ese types of flaws are detailed in Appendix B of NUREG-1021. At the workshop, several examples of these psychometric flaws were illustrated and discussed. Answering questions with incorrect but plausible distractors should not take longer for a candidate who is sure of the answer, but does take longer for the candidate who must eliminate each distractor. Also, in general, comprehension / analyses cuestions require more thought process than memory level questions anc consequently more time. The recuirement for a fifty percent minimum of. higher level questions was basec on a review of the-last two years of examination audits and an effort to standardize the level of examination difficulty.
We stated that the four hour time limit for the written examination-is i
under review by the NRC for possible extension of the limit and that extensions may be granted in accordance with the examiner standards.
I l
l
,