ML18153A226
| ML18153A226 | |
| Person / Time | |
|---|---|
| Site: | Surry, North Anna  |
| Issue date: | 03/03/1998 |
| From: | Peebles T NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION II) |
| To: | Ohanlon J VIRGINIA POWER (VIRGINIA ELECTRIC & POWER CO.) |
| References | |
| NUDOCS 9803120018 | |
| Download: ML18153A226 (22) | |
Text
.e March 3, 1998 Virginia Electric and Power Company ATTN:
Mr. J. P. O'Hanlon. Senior Vice President - Nuclear Innsbrook Technical Center 5000 Dominion Boulevard Glen Allen. VA 23060
SUBJECT:
MEETING SUMMARIES - NOVEMBER 1997 NRC REGION II TRAINING MANAGERS' CONFERENCE AND JANUARY 1998 NRC REGION II EXAMINATION WORKSHOP
Dear Mr. O'Hanlon:
This letter refers to the Training Managers Conference conducted at the Atlanta Federal Center on November 12 and 13. 1997 and the Examination Workshop conducted at the Richard B. Russell Building on January 27-29. 1998.
Representatives from all utilities in Region II participated in both meetings.
The agenda for the Training Managers Conference is Enclosure 1 and the list of attendees is Enclosure 2.
We appreciate the participation of you and your staff and believe that the goal of providing an open forum for discussion of operator licensing issues was met.
Mr. Gallo. Chief of the Operator Licensing Branch. Office of Nuclear Reactor Regulation (NRR). made a presentation on the present status of operator licensing and his slides are Enclosure 3.
During the meeting. it was decided that a workshop on operator licensing examination writing was needed and would be held at the first of the year.
Also. we have tentatively set the date for the 1998 Training Manager's Conference as November 4 and 5.
Additionally. I am enclosing our preliminary schedule for FY 1998 and FY 1999.
dated February 18. 1998. as Enclosure 4.
Please review the schedule and supply comments to my staff or myself.
The Examination Workshop was conducted with participation by everyone.
A list of attendees is Enclosure 5.
A standard Job Performance Measures (JPM) format was reviewed and comments collected by the Southeast Training Managers (SSNTA). with a final version expected this summer.
Concerns on the examination process were collected and is included as Enclosure 6.
These concerns were forwarded to NRR for review.
During the workshop. we discussed some of the problems with the initial examination process as it is being implemented be Revision 8 of NUREG-1021.
A discussion of those issues is enclosure 7.
It is our opinion that this conference was beneficial and provided an excellent opportunity for open discussion of various concerns about the
/
Operator Licensing process. especially the techniques of writing the licensing'; I examination.
9803120018 980303 PDR ADOCK 05000280 V
\\_ 'f..
I IIIIII IIIII IIIII IIIIII IIIII IIII IIII IIII
!ii
!ii 8
B A
5
.. /-
OFFICE SIGNATURE NAME DATE COPY?
e VEPCO 2
If you have any questions regarding the content of this letter. please contact me at (404) 562-4638.
Sincerely.
Original Signed by Thomas Peebles Thomas A. Peebles. Chief Operator Licensing and Human Performance Branch Division of Reactor Safety Docket Nos.:
50-338. 50-339. 50-280. and 50-281 License Nos. : NPF-4. NPF-7. DPR-32. and DPR-37 Agenda for Training Managers' Conference
Enclosures:
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
List of Attendees for 1997 Training Managers' Conference Mr. Gallo's Slides cc w/encls:
Region II Examination Schedules for FY 97 & 98 List of Attendees for 1998 Examination Workshop Concerns Expressed during Workshop Discussion of Workshop Issues W. R. Matthews. Statiori Manager.
North Anna Power Station D. A. Christian. Station Manager.
Surry Power Station J. H. McCarthy. Manager. Nuclear Licensing and Operations Support Distribution w/encls:
PUBLIC B. Michael. DRS A
--?H 3/.J
/98 3/
/98 3/
/98 3/
/98 3/
/98 3/
YES NO YES NO YES NO YES NO YES NO YES OFFICIAL RECORD COPY DOCUMENT NAME:
A:\\NSLTR.JC
/98 NO 3/
YES
/98 NO
e SOUTHEAST TRAINING MANAGER'S CONFERENCE U.S. Nuclear Regulatory Commission, Region II Atlanta, Georgia Meeting Agenda November 12-13, 1997 Atlanta Federal Center Wednesday, 11/12/97 8:00 a.m.
8:20 a.m.
8:30 a.m.
9:00 a.m.
9:30 a.m.
Conference Registration Introduction Welcome Welcome Overview of Pilot Exam Process 10:00 a.m.
Break 10:30 a.m.
11:00 a.m.
11:45 a.m.
1:00 p.m.
1:30 p.m.
Examination Communications Exam Development & Coordination Examination Security Issues Lunch Resident Review of Training Lessons Learned from Recent Exams 2: 15 p.m.
Break 2:30 p.m.
4:00 p.m.
4:30 p.m.
Examination Questions and Answers Examples of questions Meet with Principal Examiners Adjom Conference Center Conference Room C Thomas A. Peebles, Chief, Operator Licensing & Human Performance Branch Johns P. Jaudon, Director Division of Reactor Safety Bruce S. Mallett, Acting Deputy Regional Administrator Thomas A. Peebles, Chief, Operator Licensing & Human Performance Branch Ron Aiello, RII Paul Steiner, RII Paul Harmon, RII Charlie Payne, RII George Hopper, RII All ENCLOSURE 1
e 2
Thursday, 11/13/97 8:30 a.m.
8:45 a.m.
9:30 a.m.
10:00 a.m.
10:15 a.m.
12:00 p.m.
Recap Reactivity Changes and Other Issues Medical Exam Issues - Conditions Break Open Session - Other Issues Adjorn Tom Peebles Robert M. Gallo, Chief Operator Licensing Branch, NRR Charlie Payne, RII Training Managers ENCLOSURE 1
.e REGION 11 TRAINING MANAGERS CONFERENCE NOVEMBER 12-13, 1998 Timothy L. Norris Onsite Engineering General Manager Brian Haagensen PSHA CP&L Larry Dunlap BK Rick Garner HR Tom Natale RB.
William Noll BK Max Herrell BK Scot Poteet RB Crystal River - FPC Jack Springer CR Tom Taylor CR Duke Power Garmon Clements CT Camden Eflin Richard P. Bugert Corp Gabriel Washburn oc Charles Sawyer Corp Ronnie B. White, Jr MG E.T. Beadle CT William H. Miller CT Al Lindsay MG Paul Stovall oc Bentley Jones Paul Mabry oc FP&L Maria Lacal TP Philip G. Finegan
J. M. Donem FA John C. Lewis Tom Blindauer FA Joe Powell FA Bill Oldfield FA Supv. Ops Cont Trng Supv Ops Trng Supt Ops Trng Ops Trng Supv Trng Mgr Exam Team Leader Supv Simulator Tng Dir Nuc Ops Trng Human Perf Mgr oc Ops Trng Ops Trng Spec Req Team Leader Sr Tech Spec Trng Mgr lnit Lie Exam Leader Trng Mgr Ops Trng Mgr Mgr Oper Trng oc Trng Mgr Ops Line Trng Mgr Ops Trng Supv Services Mgr Trng Assessment Spec Ops Trng Supv Sr Inst Ops. Trng HT Trng & EP Mgr Sr Pit Inst Sr Inst Ops Trng Nuc Ops Tm Supv Southern Nuclear (SNC) (cont'd page 2)
ENCLOSURE 2
(
Southern Nuclear (SNC} (cont'd}
Steve Grantham Scott Fulmer Leon Ray HT Ops Trng Supv FA Mgr Trng & EP VG Ops Trng Supv Virginia Power Frank Winks NA Spv Ops Trng H. Ashley Royal NA Supt Trng Thomas Toby Sowers SR Supt Trng TVA Bob Greenman BF Trng Mgr Dick Driscoll SQ Trng Mgr Walt Hunt SQ Ops Trng Mgr James Proffitt SQ Nuc Eng Marvin Meek BF HL T Lead Inst Rusty Proffitt SQ V. C. Summer - SCE&G Terry Matlosz SM MgrTrng Al Koon SM Ops Trng Supv 2
ENCLOSURE 2
OPERATOR LICENSING INITIAL EXAMINATION RULE CHANGE Region II Training Manager, Conference November 13, 1997 Robert M. Gallo, Chief, Operator Licenalnt ltit\\oh, NRR ENCI.DSURE 3
HISTORY SECY 95-75 {3/95): Proposed change GL 95-06 {8/95): Solicited volunteers ROI 95-25 {8/95): Pilot guidance 10/95 - 4/96: Original pilot exams 5/1 /96: CRGR briefing SECY 96-1 23 (6/96): Pilot re.suits SECY 96-206 (9/96): Pros and oe>ns GL 95-06, Sup. 1 ( 1 /97): Voluntary continuation of pilot process NUREG-1021, Interim Rev. 8 (2/97)
SICY ~1-1i (4/97): Propo1ed rule fJ2 FR 4242tj (~/~1): ~toµus,ee rute
e e
THE PROPOSED RULE
- 3. A new § 55.40 is added to read as follows:
§ 55.40 Implementation.
(a) Power reactor facility licensees shall--
( 1) Prepare the required site-specific written examinations and operating tests; (2) Submit the written examinations and operating tests to the Commission for review and approval; and (3) '**fo.e.ter and grade the NRC-a,>-oroved 1ite*specif ic written tjl(iminations.
~-.--"'""'"*.-*....
e e
THE REST OF THE RULE *
(b) In lieu of requiring a specific power reactor facility licensee to prepare the examinations and tests or to proctor and grade the site-specific written examinations, the Commission may elect to perform those tasks.
(c) The Commission will pr@psre and administer the written examin8.tiens and operating tests at non~power reactor facilities.
e OTHER CONSIDERATIONS
- The NRC will prep3re one exam per Region per [calendar] year
- Facility licensees are expected to use the guidance in NUREG-1021 NRC will approve deviations NRC will not compromise statutory r@sponsibilities
- NRC is committed to maintaining quality, level of difficulty, consistency, and security
- NRC intends to use its full enforcement authority against pereons who willfully compromise ein
@xam in violotion of 66.49
BACKGROUND
- Goal was to improve efficiency while maintaining effectiveness Eliminate reliance on NRC contractors (except GFE)
Increase facility involvement Maintain examination quality and difficulty
- Remain consistent with the Act and Part 55
- Changes should be transparent to license applicants
- lnithtl licensing program was not broken
MILESTONE SCHEDULE
- 10/21 /97: Cornn ient period ended
- 4/1 /98:
Resolve comments; revise rule and NUREG-1021; seek Office concurrence
- 4/98:
- 5/22/98: Obtain Office concurrence and deliver to EDO
- 6/98:
- 7/98:
Obtain EDO and Commission concurrence Publish the final rule and Revision 8
- 12/31 /98: Implement rulo and Revision 8
E.XAM RE.SUL rs
- e.
' l Exams RO RO RO S;R:Q SRO s,m,o i
Written* Operating Total Written Operating T0t'='I l
l
- -1 F.=Y 1; 995 94o/o 98%
92%
95%
95%
92*01*
i I 1)
I Or.igt.nal 22 91 o/o 9301o 83%
93o/o 96%
9,0'%
p;.~ots 49/54 50/54 45/54 86/92 87 /'91 83/92 I
I
- Through CY 93%
95%
88%
94%
96%
91 %
1996 77/83 80/84 75/85 1 36/144 137/143 131 /144 I
I Since 92o/o 89%
83%
- 94%
92%
8 19*%
1 /97 54/59 51 /57 49/59 100/106 98/106 915/107 Total 92°/o 93°/o 86o/o 94%
94%
9:0%
131 /142 131/141 124/144 236/250 235/249 2216/251 The resudts 0 1f oin1e exa1.m p1liu;s oin1e RO and one SRO
- operating test appea1I are p,en,di,n1g..
} /. *:~:'* -
-** ~ !*
,~:*:..
... :¢:
e e
FY99 INITIAL EXAM RESULTS February 20, 1998 RO Date Plant Chief Pass 9/28/98 Sequoyah MEE 10/5/98 Harris RFA 4
10/19/98 B.Ferry WFS write DCP 4
11/30/98 Oconee &
MEE 6
12/14/98 11 /30/98 St Lucie &
RSB 15 12/14/98 1/25/99 McGuire &
DCP 14 2/8/99 1/25/99 C. River &
RFA 10-12 2/8/99 3/15/99 Watts Bar &
RSB 7
3/29/99 3/29/99 Surry &
RFA 6
4/12/99 5/17/99 Catawba &
15-18 5/31/99 5/10/99 Farley 2
Watts Bar? 6/99 6
07/ /99 Robinson?
4 07/ /99 C. River?
08/ /99 Turkey Pt?
20 9/15/99 Summer?
4 09/ /99 Sequoyah?
99
'?' designates tentative No Initial exams scheduled for:
?10/18/99 Brunswick-
?10/ /99 B. Ferry
?10/25/99 Hatch
?10/ /99 St. Lucie
?12/13 /99 Vogtle-9 candidates 4r, 4i, 4u Sr?
2wk 5r, 5i, 2u SRO-I Pass 2
4 1
2 6
4 1
North Anna SRO-U TOTAL I
Pass Pass 0
I I
I 4
I I
I 3
I I
I I
I 6
15 15 5
4 8
1 ENCLOSURE 4
e February 20, 1998 Exam PLANT Week 10/14/97 St. Lucie &
10/20 11/14/97 Cr. River RETAKE 12/1/97 Summer 12/1/97 Catawba &
12/15 3/2/98 Farley RETAKE 2/23/98 Robinson + 1 op retake 4/13/98 Vogtle (Mellen write) 5/11/98 Brunswick &
5/25/98 w Sequoyah Retake +
6/1/98 op 6/29/98 Crystal River 6/22/98 St. Lucie &
7/6/98 8/10/98 Turkey Point 8/17/98 North Anna &
8/31 9/28/98 Sequoyah I
I RESULTS TO DATE I FY 98 INITIAL EXAM RESULT,,
[10/1 /97 - 9/30/98)
RO SRO-I CHIEF PASS PASS GTH 6
6 1
1 RFA JFM 8
8 DCP 2
3 4
5 RFA 1
RSB 3
1+1 GTH 4
DCP 5
3 LSM RFA RSB MEE 6
GTH 8
4 DCP 8
RSB 8
1 MEE I
54 28 I
I 16 17 5
6
'&' designates examinations that will require two weeks to administer No exams scheduled for B. Ferry Harris Hatch McGuire Oconee Surry W. Bar SRO-U TOTAL PASS PASS 7
7 1
1 1
1 8
8 6
6 14 1
1 6
2 6
3 11 3
3 6
8 8
6 15 4
4 26 108 7
7 28 30 ENCLOSURE 4
REGION II WORKSHOP-OPERATOR LICENSING EXAMINATIONS JANUARY 27 - 29, 1998 Exam Workshop Attendees Charlie Brooks Frank S. Jaggar Asst Manager, Ops Trg - INPO Examiner - WO Associates Ken Masker Senior Licensed Instructor Rochester Gas & Electric, Bob Niedzielski James F. Belzer Max Bailey CP&L Gregg Lualam William Noll Tony Pearson
- Richard Edens Rick Garner Terry Toler Wiley Killette Scott Poteet Bill Nevins Crystal River - FPC Alan Kennedy Johnie Smith Jack Springer Duke Power Alan Whitener Edward A. Shaw Bobby Ayers Steve Helms Charles Sawyer Reggie Kinvay E. T. Beadle James K. Black Gabriel Washburn Camden Eflin R. E. Ginna NPP Exam Developer - Baltimore Gas & Electric Instructor - CCNPP/BGE Region Ill Operator Licensing Examiner LOR - Supervisor - Brunswick Supt Ops Training - Brunswick Initial Training - Brunswick LOR Instructor - Brunswick Sup - OTU - Harris Project Tech Spec - Harris Project Tech Spec - Harris Exam Team - Robinson Instruct Tech - Robinson Senior Licensed Instructor Training Supervisor Training Supervisor Ops Instructor Ops Instructor Ops Instructor - Oconee Training Super Initial Training - McGuire Initial Trining Lead Nuclear Instructor - CNS Nuclear Instructor - ONS Nuclear Instructor - ONS Team Leader - HLP - Oconee (Exam Workshop Attendees cont'd - See page 2)
ENCLOSURE 5
2 (Exam Workshop Attendees cont'd)
FP&L Ivan Wilson Kris Metzger Roger Walker Tim Bolander David P. Clark Maria L. Lacal Rich Bretton Philip G. Finegan Michael E. Crolteau Operations Manager Ops Training Supervisor - St. Lucie Instructor - St. Lucie Instructor - St. Lucie Instructor - St. Lucie Training Manager - Turkey Point Ops Cert Trng Sup - Turkey Point Ops Trining Supervisor - Turkey Point Cont Trng Instructor - Turkey Point Southern Nuclear (SNC)
Joel L. Deavers Senior Instructor - Farley Scott Fulmer Training & Emergency Preparedness Manager - Farley Gerard W. Laska Training Instructor - Farley Charlie Edmund David Gidden Ed Jones Dan Scukanec Fred Howard Virginia Power Keith Link Ed Trask Joe Scott Ken Grover Harold Mccallum Paul K. Orrison TVA Ray Scharff Denny Campbell Bob Greenman Marvin Meek A. R. Champion Rick King Frank Weller Phillip H. Gass Ed Keyser Harold Birch Plant Instructor - Hatch Training Supervisor - Hatch Plant Instructor - Hatch Ops Trng Supv - Vogtle Plant Instructor - Vogtle Requa I..... - North Anna Instructor - North Anna Supervisor Operations Training - North Anna Senior Instructor (NUC) - Surry Supervisor Ops Training - Surry Ops Instructor - Surry Instructor - Browns Ferry Instructor - Browns Ferry Training Manager - Browns Ferry Instructor - Browns Ferry Instructor - Browns Ferry Sr Ops Instructor - Sequoyah Instructor - Sequoyah Sim Instructor - Sequoyah Instructor - Sequoyah Instructor - Sequoyah (Exam Workshop Attendees cont'd - See page 2)
3 (Exam Workshop Attendees cont'd)
TVA cont'd Terry Newman Raney Evans Rick O'Rear SRO Instructor - Watts Bar SRO Instructor - Watts Bar Sift Manager - Watts Bar V. C. Summer - SCE&G Perry Ramicone Ops Instructor Bruce L. Thompson Ops Instructor William R. Quick Ops Instructor
CONCERNS EXPRESSED DURING THE REGION II EXAMINATION WRITING WORKSHOP The following is a condensation of the concerns received from the attending facilities during the January 1998 Workshop on Examination Writing. The workshop attendees and I would appreciate your consideration of the concerns during your revision to the Examiner Standards.
- 1)
- 2)
- 3)
- 4)
- 5)
- 6)
- 7)
- 8)
Security requirements are too restrictive, considering the limited resources available. Also, more guidance on minimum security expectations is needed.
(three comments)
The NRC should develop the sample plan as this would save both utility and NRC resources. (two comments)
If independent groups generate the audit and licensing exams, some overlap should be allowed. (one comment, also I believe the standards allow this now?)
The KIA catalog contains errors and omissions and should be corrected, or at the least an errata sheet of know errors should be published. (two comments)
If an exam bank item has not been used during the licensing class, the exam item should be considered at "face value" for the licensing exam. (one comment)
The length of time allowed for written exams should be revised to a more reasonable period. Does this time also apply to continuing education.
(one comment, I had commented that the length of time did not apply to requalification exams the utilities conducted.)
The NRC should periodically publish problem areas encountered during the exam process and distribute it to all training managers. (one comment)
The facilities appreciated the workshop. They want Region II to have another workshop in about six months. The next time they want to concentrate on good and bad examples of written and operating test items and the sample plan. (six comments)
ENCLOSURE 6
DISCUSSION OF WORKSHOP ISSUES During the workshop we discussed some of the problems with the revised operator licensing examination process as implemented by Revision 8 of NUREG-1021. The following were three of the principle issues discussed and a summary of the response given by NRC's Region II Operator Licensing staff.
I.
Why has exam development take so many man-hours? Some facilities did not fully understand our methodology, concepts and expectations for developing the initial examination such as content validity, plausible distractors and other psychometric issues.
The NRC did not recognize the variance across facilities in their depth of understanding.
As a result. some facilities submitted examinations with the quality lower than expected and these examinations did not meet the standards described in NUREG-1021.
The amount of resources required to modify the examinations to meet the standards was more than either the facility or the NRC had anticipated. There was general agreement during the workshop that more discussion with the facility examination writers and reviewers. such as these workshops. would better align the facilities*
original products with the standards of NUREG-1Q21 and reduc~ the resources required to develop an acceptable examination.
- 2)
Why has the NRC rajsed the level of djffjculty of the examjnatjons?
Many participants felt that the NRC was "raising the bar."
We stated that the purpose of the initial operator licensing examination is to test valid knowledges. skills and abilities required to safely carry out duties as a licensed operator at a specific facility.
The examination should be written to a discrimination level not specific to the quality of the facility's training program. but so that a minimal competent operator. with specific site knowledge and skills. will pass the examination. Therefore. the level of difficulty of the examination should not vary significantly from site to site. The concept of discrimination validity is that a given test item is written at a level which will discriminate between a competent and less than competent operator.
In some cases. the NRC examination reviews have adjusted the discrimination validity (difficulty) in order to achieve region-wide consistency on what is required of a competent operator. We try to create an examination such that an applicant who is capable of safely operating the plant will achieve a score of 80 percent or greater.
For facilities that prepare candidates beyond the minimally qualified level.
we would expect the average score to be higher.
Historically, nationwide NRC examination scores have averaged approximately 85 percent. which is a reasonable benchmark and expectation for a discriminating criterion-referenced examination.
I explained that I use a mental description of a minimally competent operator to decide if the question is one that he/she needs to know and whether the overall exam is targeted for that person to achieve a score of 80%.
An 80% score on the written examination for a minimal competent candidate does not correlate to an 80% pass rate and we have no goal ENCLOSURE 7
e 2
regarding pass rate. Overall. we did not intend to change the 'bar' and are reviewing results to ensure our practice meets our intent.
- 3)
Why have some applicants not been able to complete the examination in the four hours currently allowed? Prior to the current examination revision. we had two actions in the implementation phase.
One was the improvement in the plausibility of distractors and the other was standardizing the percent of comprehension and analyses questions.
In the last two years. we have improved our identification of poor distractors. A question does.not have discrimination validity if the distractors (i.e. incorrect answers in a multiple choice test) can be eliminated by a less than competent operator due to psychometric flaws in the question structure. These types of flaws are detailed in Appendix B of NUREG-1021.
At the workshop. several examples of these psychometric flaws were illustrated and discussed. Answering questions with incorrect but plausible distractors should not take longer for a candidate who is sure of the answer. but does take longer for the candidate who must eliminate each distractor. Also. in general.
comprehension/ analyses questions require more thought process than memory level questions and consequently more time.
The requirement for a fifty percent minimum of higher level questions was based on a review of the last two years of examination audits and an effort to standardize the level of examination difficulty.
We stated that the four hour time limit for the written examination is under review by the NRC for possible extension of the limit and that extensions may be granted in accordance with the examiner standards.