ML17354A821
| ML17354A821 | |
| Person / Time | |
|---|---|
| Site: | Turkey Point, Saint Lucie  |
| Issue date: | 03/03/1998 |
| From: | Peebles T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | Plunkett T FLORIDA POWER & LIGHT CO. |
| References | |
| NUDOCS 9803120131 | |
| Download: ML17354A821 (24) | |
Text
March 3, 1998 Florida Power and Light Company ATTN:
Hr. T.
F. Plunkett President
- Nuclear Division P. 0.
Box 14000 Juno Beach'L 33408-0420
SUBJECT:
MEETING SUMMARIES -
NOVEMBER 1997 NRC REGION II TRAINING MANAGERS'ONFERENCE AND JANUARY 1998 NRC REGION II EXAMINATIONWORKSHOP
Dear Hr. Plunkett:
This letter refers to the Training Managers Conference conducted at the Atlanta Federal Center on November 12 and 13, 1997 and the Examination Workshop conducted at the Richard B. Russell Building on January 27-29.
1998.
Representatives from all utilities in Region II participated in both meetings.
The agenda for the Training Managers Conference is Enclosure 1 and the list of attendees is Enclosure 2.
We appreciate the participation of you and your staff and believe that the goal of providing an open forum for discussion of operator licensing issues was met.
Mr. Gallo, Chief of the Operator Licensing Branch. Office of Nuclear Reactor Regulation (NRR),
made a presentation on the present status of operator licensing and his slides are Enclosure 3.
During the meeting, it was decided that a workshop on operator licensing examination writing was needed and would be held at the first of the year.
Also.
we have tentatively set the date for the 1998 Training Manager's Conference as November 4 and 5.
Additionally. I am enclosing our preliminary schedule for FY 1998 and FY 1999.
dated February 18, 1998.
as Enclosure 4.
Please review the schedule and supply comments to my staff or myself.
The Examination Workshop was conducted with participation by everyone.
A list of attendees is Enclosure 5.
A standard Job Performance Measures (JPH) format was reviewed and comments collected by the Southeast Training Managers (SSNTA)
~ with a final version expected this summer.
Concerns on the examination process were collected and is included as Enclosure 6.
These concerns were forwarded to NRR for review.
During the workshop.
we discussed some of the problems with the initial examination process as it is being implemented be Revision 8 of NUREG-1021.
A discussion of those issues is enclosure 7.
Illllillllillllllllillllllllllilllilllll L/'
It is our opinion that this conference was beneficial and provided an excellent opportunity for open discussion of various concerns about the Operator Licensing prgc~S,,qespecially the techniques of writing the licensing examination.
9803120131 980303 PDR ADOCK 05000250 V
FP&L If you have any questions regarding the content of this letter.
please contact me at (404) 562-4638.
Sincerely, ORIGINAL SIGNED BY THCMAS PEEBLES Thomas A. Peebles.
Chief Operator Licensing and Human Performance Branch Division of Reactor Safety Docket Nos.:
50-335 '0-389.
50-250.
and 50-251 License Nos.:
DPR-67, NPF-16 'PR-31 'nd DPR-41
Enclosures:
l.
2.
3.
4 5.
6.
7.
Agenda for Training Hanagers'onference List of Attendees for 1997 Training Hanagers'onference Hr. Gallo's Slides Region II Examination Schedules for FY 97 8 98 List of Attendees for 1998 Examination Workshop Concerns Expressed during Workshop Discussion of Workshop Issues cc w/encls:
J.
A. Stall, Site Vice President, St. Lucie Nuclear Plant R. J.
Hovey, Site Vice President, Turkey Point Nuclear Plant H. Allen, Training Hanager, St. Lucie Nuclear Plant H. Lacal, Training Hanager, Turkey Point Nuclear Plant PUBLIC B. Hichael.
DRS OPPZCE SZGNATURE NAME DATE RZZ:DRS TPEEBLES 3/
, /98 3/
/98 3/
/98 3/
/98 3/
/98 3/
/98 3/
/98 COPY2 YES NO ES NO OPPZCZAL RECORD COPY YES NO YES NO DOCUMENT NAME:
A:NSLLTR.<AC YES NO YES NO YES NO
SOUTHEAST TRAININGMANAGER'S CONFERENCE U.S. Nuclear Regulatory Commission, Region II Atlanta, Georgia Meeting Agenda November 12-13, 1997 Atlanta Federal Center 8:00 a.m.
Conference Registration 8:20 a.m.
Introduction Conference Center Conference Room C Thomas A. Peebles, Chief, Operator Licensing & Human Performance Branch 8:30 a.m.
Welcome Johns P.
Jaudon, Director Division of Reactor Safety 9:00 a.m.
Welcome Bruce S. Mallett, Acting Deputy Regional Administrator 9:30 a.m.
Overview of Pilot Exam Process 10:00 a.m.
Break Thomas A. Peebles,
- Chief, Operator Licensing & Human Performance Branch 10:30 a.m.
Examination Communications Exam Development & Coordination Ron Aiello, Rll 11:00 a.m.
Examination Security Issues 11:45 a.m.
Lunch Paul Steiner, RII 1:00 p.m.
1:30 p.m.
Resident Review of Training Lessons Learned from Recent Exams Paul Harmon, RII Charlie Payne, RII 2:15 p.m.
Break 2:30 p.m.
Examination Questions and Answers Examples of questions George Hopper, RII 4:00 p.m.
Meet with Principal Examiners All 4:30 p.m.
Adjorn ENCLOSURE 1
8:30 a.m.
Recap 8t45 a.m.
Reactivity Changes and Other Issues Tom Peebles Robert M. Gallo, Chief Operator Licensing Branch, NRR 9:30 a.m.
Medical Exam Issues - Conditions 10:00 a.m.
Break Charlie Payne, RII 10:15 a.m.
Open Session - Other Issues 12:00 p.m.
Adjorn Training Managers ENCLOSURE 1
Timothy L. Norris Brian Haagensen REGION II TRAININGMANAGERS CONFERENCE NOVEMBER 12-13, 1998 Onsite Engineering General Manager PSHA
~CP8
'arryDunlap Rick Gamer Tom Natale William Noll Max Herrell Scot Poteet BK HR RB BK BK RB Supv. Ops Cont Trng Supv Ops Trng Supt Ops Trng Ops Trng Supv Trng Mgr Exam Team Leader a
iver-FP Jack Springer Tom Taylor
~DPower Garmon Clements Camden Eflin Richard P. Bugert Gabriel Washburn Charles Sawyer Ronnie B. White, Jr E.T. Beadle William H. Miller AI Lindsay Paul Stovall Bentley Jones Paul Mabry
~FP&
Maria Lacal Philip G. Finegan Dennis L. Fadden Jo Magennis Kris Metzger CR CR CT Corp OC Corp MG CT CT MG OC OC TP TP SL Corp SL Supv Simulator Tng Dir Nuc Ops Trng Human Perf Mgr OC Ops Trng Ops Trng Spec Req Team Leader Sr Tech Spec Trng Mgr Init Lic Exam Leader Trng Mgr Ops Trng Mgr Mgr Oper Trng OC Trng Mgr Ops Line Trng Mgr Ops Trng Supv Services Mgr Trng Assessment Spec Ops Trng Supv Sou h rn Nuclear J. M. Donem John C. Lewis Tom Blindauer Joe Powell Bill Oldfield FA FA FA Sr Plt Inst Sr Inst Ops Trng Nuc Ops Trn Supv
)
SN FA
. Sr Inst Ops. Trng HT Trng & EP Mgr Souhern Nuclear SNC cont'd a
e ENCLOSURE 2
OPERATOR LICENSING INITIALEXAIVllNATION RVL,K CHANGE Region II Trwlnlny Mlnsgerl Conference Novembir 13, 1997 Robert M. Qallo, Chief, Operator Licenllhe lfehoti, MAR ENCXQSURE 3
HISTORY SECY 95-75 (3/95): Proposed change GL 95-06 (8/95):
Solicited volunteers
~
ROI 95-25 (8/95):
Pilot guidance
~
10/95 4/96:
Original pilot exams
~
5/1/96:
CRGR briefing
~
SECY 96-123 (6/96):
Pilot results
~
SECY 96-206 (9/96):
Pros and cons
~
GL 95-06, Sup.
1 (1/97):
Voluntary continuation of pilot process
~
NUREG-1021, Interim Rev. 8 (2/97)
SICY 97-79 (4/97):
Prepolod rule
~
62 FA 42426 (I/I7): Proposed rule
THE PROPQSED RVLE 3.
A new 5 55.40 is added to read as follows:
(a)
Power reactor facility licensees shall (1)
Prepare the required site-specific written examinations and operating tests; (2) Submit the written eximinations and operating tests to the Commission for review and approval; and (3) ilroctor and grade the NRC-(ipprovod site-specific written oXNrnlrlNIQAI.
THE REST OF THE RULE (b)
In lieu of requiring a specific power reactor facility licensee to prepare the examinations and tests or to proctor and grade the site-specific written examinations, the Commission may elect to perform those tasks.
(c)
The Commission will prepare and administer the written examinytiens and operating tests at non-jowor reactor facilities.
OTHER CONSIDERATIQNS
~ The NRC will prepare one exam per Region per [calendar] year Facility licensees are expected to use the guidance in NUREG-1021 NRC will approve deviations NRC will not compromise statutory responsibilities
~ NRC is committed to maintaining quality, level of difficulty, consistency, and security
~ NRC intends to use its full enforcement authority against persons who willfullycompromise an exam in violation of 56.49
BACKGROUND
~ Goal was to improve efficiency while maintaining effectiveness Eliminate reliance on NRC contractors (except GFEj Increase facility involvement Maintain examination quality and difficulty Remain consistent with the Act and Part 55
~ Changes should be transparent to license applicants
~ Initiillicensing program was not broken
MILESTONE SCHEDULE
~ 10/21/97: Comnient period ended
~ 4/1/98:
Resolve comments; revise rule and NUREG-1021; seek Office concurrence
~ 4/98:
~ 5/22/98:
Obtain Office concurrence and deliver to EDO
~ 6/98:
Obtain EDO and Commission concurrence
~ 7/98:
Publish the final rule and Revision 8
~
'l 2/31/98:Implement rule and Revision 8
0 t7 ~
~ I
~
o
~ 0
~
~
~
~
~
~
~
~
~
a
~
~
~
~
~
~
~
0
~
~ ~
e
~
)
OQO
~
~
4
~
~
~ 0
~
~ 0
~
0 ~
~
~ g
~
0
~
~
4 4
~ ~
~
~
4 0
~
~
~
~
~
~ g
~
~
~
~ 0 Oo
~
0 ~
0 O
~
~
~
~
~
~
4 0
~
~ 0
~
~ g
~
~
g
~
n ~
4
~
F 99 NITIA EXAM R SULTS February 20, 1998 RO SRO-l SRO-U TOTAL Date Plant 9/28/98 Sequoyah 10/5/98 Harris 10/19/98 B.Ferry WES write 11/30/98 Oconee &
12/14/98 11/30/98 St Lucie 8 12/14/98 1/25/99 McGuire 8 2/8/99 1/25/99 C. River &
2/8/99 3/1 5/99 Watts Bar 8 3/29/99 3/29/99 Surry &
4/1 2/99 5/1 7/99 Catawba 8 5/31/99 5/1 0/99 Farley Watts Bar? 6/99 07/ /99 Robinson?
07/ /99 C. River?
08/ /99 Turkey Pt?
9/15/99 Summer?
09/ /99 Sequoyah
?
99 Chief MEE RFA DCP MEE RSB DCP RFA RSB RFA Pass 15 10-12 15-18 14 20 Pass Pass
¹ Pass
'?'esignates tentative No Initial exams scheduled for:
North Anna
?10/18/99 Brunswick-
?10/ /99 B. Ferry
?10/25/99 Hatch
?10/ /99 St. Lucie
?12/13/99 Vogtle-9 candidates 4r, 4i, 4u 8r?
2wk 5r, 5i, 2u ENCLOSURE 4
February 20, 1998 FY 9 INITIA XAM
[10/1/97 - 9/30/98j Exam Week 10/14/97 10/20 11/14/97 12/1/97 12/1/97 12/15 3/2/98 PLANT St. Lucie 8 Cr. River RETAKE Summer Catawba 8 Farley RETAKE CHIEF GTH RFA JFM DCP RFA RO PASS SRO-I PASS SRO-U PASS 5
6 TOTAL PASS 14 2/23/98 4/13/98 5/11/98 Robinson+
1 op retake Vogtie (Mellen write)
Brunswick 8 RSB GTH DCP 1+1 5/25/98 w 6/1/98 op 6/29/98 6/22/98 7/6/98 8/10/98 8/17/98 8/31 9/28/98 Sequoyah Retake +
Crystal River St. Lucie 8 Turkey Point North Anna 8 Sequoyah LSM RFA RSB MEE GTH DCP RSB MEE 54 28 26 15 108 RESULTS TO DATE 16 17 6
7 7
28 30
'8'esignates examinations that will require two weeks to administer No exams scheduled for B. Ferry Oconee Harris Surry Hatch W. Bar McGuire ENCLOSURE 4
REGION II WORKSHOP - OPERATOR LICENSING EXAMINATIONS JANUARY27 - 29, 1998 Exam Workshop Attendees Charlie Brooks Frank S. Jaggar Ken Masker Bob Niedzielski James F. Belzer Max Bailey Asst Manager, Ops Trg - IMP(N Senior Licensed Instructor Rochester Gas 8 Electric, Exam Developer - Bal imor Gas 8 lec ric Region III Operator Licensing Examiner CP8L Gregg Lualam William Noll Tony Pearson Richard Edens Rick Garner Terry Toler Wiley Killette Scott Poteet Bill Nevins LOR - Supervisor-~Brun wi k Supt Ops Training - ~Bru swic Initial Training - ~Bunswtc LOR Instructor - ~Bunswtck Sup - OTU - Harris Project Tech Spec - Harris Project Tech Spec - Harris Exam Team-~Robins n
Instruct Tech - ~obinso C
stal River - FPC Alan Kennedy Johnie Smith Jack Springer Senior Licensed Instructor Training,Supervisor Training Supervisor Duke Power Alan Whitener Edward A. Shaw Bobby Ayers Steve Helms Charles Sawyer Reggie Kinvay E. T. Beadle James K. Black Gabriel Washburn Camden Eflin Ops Instructor Ops Instructor Ops Instructor - Oconee Training Super Initial Training - McGuire Initial Trining Lead Nuclear Instructor - QNS Nuclear Instructor - ONS Nuclear Instructor - ONS Team Leader - HLP - Oconee (Exam Workshop Attendees cont'd - See page 2)
ENCLOSURE 5
(Exam Workshop Attendees cont'd)
~FP8 Ivan Wilson Kris Metzger Roger Walker Tim Bolander David P. Clark Operations Manager Ops Training Supervisor -2 Lucia Instructor - St. Lucie Instructor - ~SLucte Instructor - ~Specie Maria L. Lacal Rich Bretton Philip G. Finegan Michael E. Crolteau Training Manager - ~Turke Poin Ops Cert Trng Sup-~Turk ~oisin Ops Trining Supervisor - ~Turke Poin Cont Trng Instructor - j.gJr~e~~oin Sou hem Nuclear Joel L. Deavers Scott Fulrner Gerard W. Laska Charlie Edmund David Gidden Ed Jones SNC Senior Instructor - ~Frl Training 8 Emergency Preparedness Manager - ~Frle Training Instructor - ~arle Plant Instructor - ~ach Training Supervisor - ~ac Plant Instructor - ~Ha ch Dan Scukanec Fred Howard Ops Trng Supv -~Vo te Plant Instructor - ~Vo LII Keith Link Ed Trask Joe Scott Ken Grover Harold McCallum Paul K. Orrison Requat..... - Norah Anna Instructor - ~No h
nna Supervisor Operations Training - North Anna Senior Instructor (NUC) - Surry Supervisor Ops Training - Surrey Ops Instructor - ~Sur TVA Ray Schorff Denny Campbell Bob Greenman Marvin Meek A. R. Champion Rick King Frank Weller Phillip H. Gass Ed Keyser Harold Birch Sr Ops instructor- ~Se uo ah Instructor-~Se uo ah Sim Instructor - ~e~uo ah Instructor - ~Se
~uo gh Instructor-~Se uo ah (Exam Workshop Attendees cont'd - See page 2)
(Exam Workshop Attendees cont'd)
~TV con 'd Terry Newman Rancy Evans Rick O'Rear SRO Instructor - ~Ws Bar SRO instructor - W~sgar Sift Manager - W~aBar V
C ummer - SCEBG Perry Ramicone Ops Instructor Bruce L. Thompson Ops Instructor William R. Quick Ops Instructor
CONCERNS EXPRESSED DURING THE REGION II EXAMINATIONWRITING WORKSHOP The following is a condensation of the concerns received from the attending facilities during the January 1998 Workshop on'Examination Writing. The workshop attendees and I would appreciate your consideration of the concerns during your revision to the Examiner Standards.
1)
Security requirements are too restrictive, considering the limited resources available.
Also, more guidance on minimum security expectations is needed.
(three comments) 2)
The NRC should develop the sample plan as this would save both utilityand NRC resources.
(two comments) 3)
Ifindependent groups generate the audit and licensing exams, some overlap should be allowed. (one comment, also I believe the standards allow this nowt) 4)
The K/A catalog contains errors and omissions and should be corrected, or at the least an errata sheet of know errors should be published. (two comments) 5)
Ifan exam bank item has not been used during the licensing class, the exam item should be considered at "face value" for the licensing exam. (one comment) 6)
The length of time allowed for written exams should be revised to a more reasonable period.
Does this time also apply to continuing education.
(one comment, I had commented that the length of time did not apply to requaiification exams the utilities conducted.)
7)
The NRC should periodically publish problem areas encountered during the exam process and distribute it to all training managers.
(one comment) 8)
The facilities appreciated the workshop.
They want Region II to have another workshop in about six months.
The next time they want to concentrate on good and bad examples of written and operating test items and the sample plan. (six comments)
DISCUSSION OF WORKSHOP ISSUES During the workshop we discussed some of the problems with the revised operator licensing examination process as implemented by Revision 8 of NUREG-1021.
The following were three of the principle issues discussed and a
summary of the response given by NRC's Region II Operator Licensing staff.
klhy has exam development take so many man-hours?
Some facilities did not fully understand our methodology, concepts and expectations for developing the initial examination such as content validity, plausible distractors and other psychometric issues.
The NRC did not recognize the variance across facilities in their depth of understanding.
As a
results some facilities submitted examinations with the quality lower than expected and these examinations did not meet the standards described in NUREG-1021.
The amount of resources required to modify the examinations to meet the standards was more than either the facility or the NRC had anticipated.
There was general agreement during the workshop that more discussion with the facility examination writers and reviewers, such as these workshops'ould better align the facilities'riginal products with the standards of NUREG-1021 and reduce the resources requi red to develop an acceptable examination.
2)
Vhy has the NRC raised the leve7 of difficultyof the examinations?
Many participants felt that the NRC was -raising the bar."
We stated that the purpose ot the initial operator licensing examination is to test valid knowledges. skills and abilities required to safely carry out duties as a licensed operator at a specific facility.
The examination should be written to a discrimination level not specific to the quality of the facility's training program.
but so that a minimal competent
- operator, with specific site knowledge and skills, will pass the examination.
Therefore, the level of difficultyof the examination should not vary significantly from site to site.
The concept of discrimination validity is that a given test item is written at a level which will discriminate between a competent and less than competent operator.
In some cases, the NRC examination reviews have adjusted the discrimination validity (difficulty) in order to achieve region-wide consistency on what is required of a competent operator.
We try to create an examination such that an applicant who is capable of safely operating the plant will achieve a score of 80 percent or greater.
For facilities that prepare candidates beyond the minimally qualified level, we would expect the average score to be higher.
Historically.
nationwide NRC examination scores have averaged approximately 85
- percent, which is a reasonable benchmark and expectation for a discriminating criterion-referenced examination.
I explained that I use a mental description of a minimally competent operator to decide if the question is one that he/she needs to know and whether the overall exam is targeted for that person to achieve a score of 80K.
An 80K score on the written examination for a minimal competent candidate does not correlate to an 80% pass rate and we have no goal ENCLOSURE 7
regarding pass rate.
- Overall, we did not intend to change the
'bar 'nd,,
are reviewing results to ensure our practice meets our intent.
3)
Vhy have some applicants not been able to complete the examination in the four hours currently allowed?
Prior to the current examination
- revision, we had two actions in the implementation phase.
One was the improvement in the plausibility of distractors and the other was standardizing the percent of comprehension and analyses questions.
In the last two years, we have improved our identification of poor distractors.
A question does.not have discrimination validity if the distractors (i.e. incorrect answers in a multiple choice test) can be eliminated by a less than competent operator due to psychometric flaws in the question structure.
These types of flaws are detailed in Appendix 8 of NUREG-1021.
At the workshop, several examples of these psychometric flaws were illustrated and discussed.
Answering questions with incorrect but plausible distractors should not, take longer for a candidate who is sure of the answer, but does take longer for the candidate who must eliminate each distractor.
Also, in general, comprehension I analyses questions require more thought process than memory level questions and consequently more time.
The requirement for a f'ifty percent minimum of higher level questions was based on a review of the last two years of examination audits and an effort to standardize the level of examination difficulty.
We stated that the four hour time limit for the written examination is under review by the NRC for possible extension of the limit and that extensions may be granted in accordance with the examiner standards.