ML20035B412

Forwards Summary of Contract Examiner Evaluations & Operating Test Audits from First Quarter of FY93,for Review

ML20035B412
Person / Time
Issue date:	03/24/1993
From:	Lange D Office of Nuclear Reactor Regulation
To:	Morgan M Battelle Memorial Institute
References
NUDOCS 9304010316
Download: ML20035B412 (10)

Text

,

s e

MAR 2 41993 Mr. Mark Morgan Battelle Northwest K6-47 Battelle Boulevard Sigma 3 Building Richland, Washington 99352

Dear Mr. Morgan:

Enclosed for your review is the summary of the Contract Examiner Evaluations (Form ES-501-4) and Operating Test Audits (Form ES-501-5) from the first quarter of Fiscal Year 1993.

If you have any questions, please call Larry Vick at (301) 504-3181.

Sincerely, f

Original sigred by:

7 David J. Lange, Acting Chief Operator Licensing Branch l

Division of Reactor Controls i

and Human Factors i

Office of Nuclear Reactor Regulation l

I

Enclosure:

As stated DISTRIBUTION-

[Centralfilesl i

OLB R/F i

Public Document Room i

LVick DLange i

AMendiola i

JCaldwell i

RGallo

}

DMcCain.

BBoger 0FC OLB:DRCH OLB:DRCH 0LB:DRCH g )

I'

~

NAME LVick:rc [ D AMbkd DLange N '

f iDATE 3 ' //E793 NI3/93 3 /,Z(/ /93 f

0FFICIAL RECORD COPY Document Name:

PNLQTRFY.EV1

{

i "Y

Ano munion emiut nus 93040 ggt 69g'j24 PDR PDR I,

_r

MAR 2 41993 Mr. Mark Morgan Battelle Northwest K6-47 Battelle Boulevard Sigma 3 Building Richland, Washington 99352 l

Dear Mr. Morgan:

Enclosed for your review is the summary of the Contract Examiner Evaluations (Form ES-501-4) and Operating Test Audits (Form ES-501-5) from the first quarter of Fiscal Year 1993.

If you have any questions, please call Larry Vick at (301) 504-3181.

Sincerely, Original signed by:

David J. Lange, Acting Chief Operator Licensing Branch Division of Reactor Controls and Human Factors Office of Nuclear Reactor Regulation

Enclosure:

As stated DISTRIBUTION:

Central Files OLB R/F Public Document Room LVick DLange AMendiola JCaldwell RGallo DMcCain BBoger

)

b

~~

0FC OLB:DRCH OLB:DRCH OLB:DRCH 3 t

NAME LVick:rc [ D AMhkd DLange N '

DATE 3 //d93 Sk3/93 3 /c24 /93 0FFICIAL RECORD COPY Document Name: PNLQTRFY.EV1 e

i 9

ENCLOSURE

.I i

L OVARTERLY

SUMMARY

OF CONTRACTOR EXAMINER PERFORMANCE BY PNL

{

The following is a summary of the Contract Examiner Evaluations and Operating e

Test Audits received during the first quarter of Fiscal Year 1993.

INITIAL EXAMINATIONS Write /Develoo written examinations. simulator scenario sets and JPM sets.

One initial examination (Sequoyah) JPH sets submitted by the contractor was i

reported to be of poor quelity. For example, several JPMs had questions that were not associated with the system tested by the JPM.

ES-301, G.2.c, requires that the questions be associated with the subject area. The region had to replace several contractor submitted JPM questions in order to comply with the Examiner Standards. Other significant problems regarding the quality of the JPM sets submitted by contractor were:

(1) no questions were included for one JPM, (2) simulated JPMs were missing required cues, (3) a JPM i

associated with cold leg accumulators had cues for level in percent when the meters read out in gallons as had been noted during prep week, (4) a JPM that was supposed to test an alternate response path did not contain the required r

malfunction on the simulator operator's setup sheet causing the alternate path not to be tested, (5) two questions had wrong answers on the key, and (6) i there was a question on a JPM for a nonexistent system (Upper head Injection).

The region was forced to correct these problem areas. A written response is requested explaining how this situation has been corrected.

One initial written examination (Braidwood) submitted to the region did not meet NUREG-BR-0122, " Examiners Handbook for Developing Operator Licensing i

Examinations." K/A sample requirements in that there was no K2, K3 or Al plant systems section for the SRO exam and no Emergency / Abnormal, K2 and K3 sections for the R0 exam. The region had to add additional K/As to meet the requirements.

Five initial examinations (Summer, McGuire, Farley, Maine Yankee, and University of Michigan), in which the contractor activity included writing examinations and on site preparation of simulator scenarios and JPMs/walkthroughs, received positive comments from the NRC chief examiners,-

indicating that the performance level was satisfactory and in accordance with the Examiner Standards, t

All other aspects of the contractor performance were conducted in accordance with the Examiner Standards.

Administer written and operatina examinations and JPMs.

One initial examination (Sequoyah) in which the contractor activity included writing examinations and on site preparation of simulator scenarios and j

JPMs/walkthroughs received negative comments from the chief examiner, indicating that the performance level was less than satisfactory and not in l

accordance with the Examiner Standards. Two specific areas of Examiner Standard non-compliance was reported.

First, contrary ES-402.0, while performing the duties of proctor during the written examination, the contract examiner read material (studying material from an engineering course) which provided the opportunity to divert his attention from the applicants. This situation was immediately corrected after it was pointed out to the contractor. Secondly, a potential exam compromise situation occurred when the contractor inadvertently left one of the JPM initial conditions handout sheets in the control room during the walkthrough exam. The Region ensured that a compromise did not occur by removing the test material before the same JPM set was to be administered to a second set of candidates later in the day.

Positive feedback from the NRC regional examiners indicate that an otherwise excellent job was done in administering written and operating examinations at all other facilities.

Grade / Document written and operatina examinations and JPMs.

All efforts were in compliance with the appropriate sections of the Examiner Standards.

Review /0A the cradina of written and operatina examinations and JPMs.

As pointed out and discussed in the examples above, the management oversight in the review process of a contract examiners product prior to submittal to the NRC warrants improvement.

k REQUALIFICATION EXAMINATIONS AND OPERATIONAL EVALUATIONS Review. write or modify: written examinations. simulator scenario sets and JPM sets.

All efforts were in compliance with the appropriate sections of the Examiner Standards. No additional comments were made.

Administer written and simulator examinations and JPMs.

Several operating examinations (Nine Mile Point 2, WNP-2, Crystal River, and Davis-Besse) resulted in the chief examiner commenting on the high quality performance of the contractors during the examination _ process. The excellent technical support provided by the contractors was noted by the chief examiners.

Grade / Document written and operatino examinations and JPMs.

j All contractor examiners met or exceeded the requirements of the Examiner

[

Standards.

Review /0A the aradino of written and operatino examinations and JPMs.

All efforts were in compliance with the appropriate sections of the Examiner Standards.

NRC Audit (s) of PNL Contractor Examiners l

The following PNL contractor examiner Initial Operating Test Audits are enclosed for your review and information-i 1.

John Bumgardner, Braidwood 10/26-29/92 2.

Larry Sherfey, Braidwood 10/27-29/92 3.

Bob Orton, University of Michigar.

12/16/92 i

i r

9 i

r I

b

c t

i f

Listed below are-the examinations for which contractor evaluations were received during this quarter (1st Quarter FY 1993):

(Task Item No.1/ Facility)

Exam Tvoe Exam Dates t

P-213 Sequoyah Initial 7/20/92 P-217 Maine Yankee Initial 10/01/92 s

P-220 Farley Initial 10/05/92 P-221 Crystal River Requal 11/06/92 P-223 Braidwood Initial 10/26/92 j

P-227 McGuire Initial 12/07/92 r

P-228 Summer Initial 11/30/92 P-229 NMP-2 Requal 11/16/92 P-230 WNP-2 Requal 11/03/92 i

P-231 Davis-Besse Requal 11/30/92 l

P-240 University of Michigan Initial-Nonpower 12/14/92 i

t I

[

t I

s 3

s' HOLB-MC-170,11/01/91, ATTACHMENT 2 OPERATING TEST AUDIT Examiner: Robert D. Orton (PNL) INorePower Cert;fication!

Feeltity: Ford Nucteer Reactor (University of Michipsn)

Ch'ef Examiner: Lawrence Vick (NRC)

Date of Operating Test: December 16,1992 f Author: RDO (Date of test audit - n/a)

Circle Test Type /, Level: Initisi / Requal

//

Snggi/jjjfpq l Auditor's Name. Lewronce Vick (NRR, DRCH, HOLB)

Signature Mg f f.33 ES Comments Rating Factore Conformance (Cross reference to the applicable esting factor inleft column.)

l 1. Test Scope and Content (ES 301/304)

Yes No Bob administered initialintegrated plant operations examinatione

a. Ad nin Topics (A) s and oral /ws!k through p! ant examinations to one RO and one

1. Scope (License Leve!)*

X

2. Depth (License Level)*

x Bob fully complied with att arses of the Examiner Stenderds during the operating tests.

l

b. Systems We?k-through (E)

, [

y

(

1. Safety Function Distribution

• X ttem 2 e 7 - The demeanor of the examiner during the r.orepower l

2. Subject Aree C.sestions*

X exam process was professione! et a!! tirnes. Bob's {>erformance on this cross-certif; cation examination was excellent. The quehty of

3. License Level' X

Bob's work was very hlgh. The level of effort was in accordance with the task order. As the exerrsner of record,I bighly c.

Integrated Plant Operations (C) recommend that Mr. Orton be certified es a Non Power Reactor

1. Trans;ents/ Events X

2. Competencies

• X d.

Other (Describe in Comments)*

2. Test Administration (ES 302.ES 304)

~

a.

General

1. Applicant Sriefings X

• - Responsibility for these items is shared by the author, the Chief Examiner, and the test administrator, if there is e problem,

2. Adhered to Exam Outline X

the aud; tor should ensure that afl responsible indMdua!s are l

3. Use of Simutator X

4. Knowledge of Facihty X

S. Integrated Administrative Topies X

6. Asked Follow-up as Necessary X

7. Domeanor / Professionali m Comment Reg'd 8F

b. Waik-through v

4

1. Admin. Topics Integrated X

2. Exem Techrsque X

c. Integrated Piant Operations

1. Admin. Topics integrated X

2. Exam Technique X

d.

Other (Describe in Comments)

3. Documentet;on / Grading (ES-303)

s. UNS AT Grades Justified X

b. Report Cornpleto X

c. Other (Describe in Comments)

O s.

EV E 06/01/99 E5 5015 U. 5. WAI/R EGJLATCRt' CCWSSM raw xs W GN N O U & $

M Read,sced M ~~O,s (h ~ 6,(9NO tw 780 Ro nAi/

L-oNach-

" h t

9 eo/O-29/9' DCE

• N85C NM w4e rm '

m.>c rm m

m

i. m.Do r.wwo m:ma m

m A GDi. TCR:$ $301/L1)

D. DCureCD PUFC CPOW3Ps $3C1/Q t c:>nt:t axw sysTes prt/t2)

2. ocaat a ran/prns/mxcuts

t. Hte nom.

X A sese rnn/rrns commr X X

s scam mxcuts pas =

X

. Azurr X

1 an c o m emm xmc ur x

oc,w:m o n e. =.

X ti.e y

mm.ms m -

t m o.

crun m. rem pri/w M c ons,mm = cm, we. ><

t. En;ocr/ww. Psexcuts X c.ftsctrec m m twosw w X

<m.ms:roecmo w re b

• So ph J a cwfM aHia w) d Ge. A m.a.ka d;L s ppu hch.

h,Y J Gr.s & overbo<r

% Io y l 4 4f ccd, LA suo cea y w ex on-v,:..

om,.za > ka,1,., pn.,,a.u. -c, s., a n, a e; a b.1; H wee wn w

/m '

m e

L cs. x m erx es is$c '

m t saker [Aucos Aac;uay 3JMy c A rnuv 3 Jr7c

c. m.ct e m,.,o m X

L c3&Dt"5 k @ (AHackJ.

1 d

(1) For and mbstd erw chad the ap;rusdsts hkmA.

(!) M e rathy res)hs e ear rest Usa add:ensf pays as asmsenery

Y OPERATING TEST AUDIT COMMENTS 1.

Good use of follow up questions during JPM administration.

2.

Good cuing of required plant parameters during JPM administration when needed cues were not specifically written into the JPM.

{

3.

Stay involved during validation of examination material to ensure you co=pletely understand candidates required actions including l

what/where/how actions are to be completed.

l

~

4.

Follow candidates to back panels to observe verification of instruments and or operation of equipment.

5.

Get in habit of picking up all notes / logs or drawings generated by 1

the candidate during all phases of the examination.

(Also get i

candidate to initial and date as a common practice).

6.

When repeating back candidates response on questions asked, ensure you repeat back verbatim. Do not paraphrase or change answer.

7.

Don't be predictable (i.e. don't repeat back candidates answer or f

ask if that was the candidates answer only on questions that the candidate did not provide the expected response for). Vary times i

and reasons you repeat back response or do.it all the time.

l l

8.

Keep your notes out of candidates view while writing. If you need l

~

i to get caught up with note taking (i.e. during JPM performance) l i

then inform the candidate so you can. Do not try to get caught up while the candidate is performing actions of the JPM as this can distract your attention from the candidate and you could fail to observe an incorrectly performed action. Try to make any lengthy notes regarding candidate performance after the JPM is terminated.

i l.

9.

Good job of working in administrative areas into JPM performance (i.e. key control, P&ID usage).

{

i 10.

Ask radiation control / limits, and security related questions while walking thru plant during JPM performance whenever possible.

I 11.

Ensure you double check that you have covered' all the required i

areas prior to releasing the candidate.

l 12.

Good job at keeping the candidate relaxed but don't go overboard.

Maintain a regulatory perspective.

13.

Overall good job. All of the items identified here were considered.

to be minor and usually occurred only once. These should take care of themselves with more experience.

}

i t

l l

o l

9 6

n : oc/ot/sa N

m r.ps 33 opt ATING H AUDIT 7

M Rs;Lood

"- Larr a 5ke,\\es (9t/D

""" Tc. Gt,,,mm M

<= = snocr3 -c,u,

l.1,-19l9t-DG cr Mmsc 10/,4 1

we L,a

,a nen A m TDPc5 pr/L1) a gia%c Pusc PDtc.6 p3m/L3)

Ltoots.sem M m o m/t2)

2. ocuoct a rnrrtsm/prea::xm X

A scre rury/nm oca. cat X

t. Kr om.

X t se,= m-s -

x zmm ue X

& - m..ao.

x x

t 2 m m a m.o,o:D X

==

c. rnrr -- nura) X

c. aow=== eucmure X

t. ELENDcr/Oc54. PICCI:1FE3 hlisD M m Ts N

4. Dues :resa (cumm:wr:0 krrga me a - sa a. refa.wd a4 peo)~jen,a) at+;L,k J:d.

co a les s h e w v4 a

-1 u.,

e.

e yg5 g

L DMWth It00C M IF:53 yt3 g

L LEPf EAE*D6 Kf2ATJ I.PPCFD A. 3&D"3.L57D C

c. m.m - : m -

x L C3&a"3 l- 00NJhl0) elled of Mu&

bo-

11'M o.L,l,4,;Qa*4 A s ' 14-emi,,./.4, 3. w p e vol u h !.,

e A c~dM d,h' />.+ 1ik Je it o<

s u J-p & w :s r ee,$,r ~L' em!rt osa hag, ana w as a.

s.a u.- a s a,, -u a,-,9 % co tro a

s. m a w c u-w$40 wou pokhal.

n

$d' l

war g.,e sn un {.,e

-to ck em%s a.alc.;) *1lk* 9 *es%s 0)

Ws:n2.,l m a pii.-:p ay.

c) ~ ~ m -

cr) m e ne, e.

c um.uts==r p.,.. m Exc"'ks Sle'd:rf.s

.