ML032720611

Exam Process

ML032720611
Person / Time
Site:	Hatch, Browns Ferry, Oconee, Mcguire, Catawba, Harris, Saint Lucie, Watts Bar, Sequoyah, Summer, Brunswick, Surry, North Anna, Turkey Point, Crystal River, Vogtle, Farley, Robinson, McGuire
Issue date:	09/22/2003
From:	Ernstes M NRC/RGN-II
To:
References
Download: ML032720611 (14)

Text

Examination Process Mike Ernstes

Examination Process Examiners Principal Examiner - point of contact with facility for routine questions.

Chief Examiner - Lead examiner responsible for coordinating a particular examination.

Examination Process Principal Examiners Gerry Laska: Catawba, McGuire, Oconee Steve Rose: Farley, Hatch, Vogtle Ron Aiello: Crystal R., St. Lucie, Turkey Point Kathleen ODonohue: Brunswick, Harris, Robinson Lee Miller: North Anna, Surry, Summer Tim Kolb: Browns Ferry, Sequoyah, Watts Bar

Examination Process Chief Examiners 12/03 Turkey Point - Ron Aiello (Tim Kolb UI) 2/04 Harris - Kathleen ODonohue (Gerry Laska UI) 2/04 Surry - Edwin Lea (Steve Rose UI) 4/04 Browns Ferry - Ron Aiello 4/04 Summer - Rick Baldwin 6/04 Sequoyah - George Hopper

Examination Process Chief Examiners 6/04 Oconee - Kathleen ODonohue 6/04 North Anna - Lee Miller 7/04 Watts Bar - Edwin Lea 8/04 St. Lucie - Rick Baldwin 8/04 Robinson - Kathleen ODonohue

Examination Process Planning Guide P The Chief Examiners Planning Guide breaks the examination process into 77 tasks.

P The Chief Examiner consults with the assigned examiners and facility staff to establish a time line for completion of the various tasks.

Examination Process Planning Guide P The Planning Guide notes who is responsible and targets a completion date for each task.

P A key aspect of this process for all stakeholders is to ensure that resources are available to meet the agreed upon schedule.

Examination Process Written Examination Results by Author Author Average Score FailureRate NRC 87.1% 5.9%

Facility 89.6% 4.4%

Examination Process Lessons Learned P Licensee did not have enough time to review the written examination.

- NRC did not deliver the examination materials early enough for a thorough review.

- Facility did not have adequate resources available during the scheduled review time.

- Initial review was not thorough resulting in many issues being identified just before the examination.

Examination Process Lessons Learned P Major overhaul of Operating Exams occurred during prep week

< Licensee did not have an understanding of the criteria for acceptable malfunctions.

< Licensee had not reviewed the exam materials prior to the prep week.

< NRC examiners had not reviewed the exam materials prior to the prep week.

Examination Process Lessons Learned P Licensee had only one individual to review the examination.

< Unable to work on written and operating test concurrently.

< Licensee had no one in reserve for prep week when the principal become unavailable.

Examination Process Lessons Learned P NRC was not specific in material request resulting in sending more than we needed.

P Adequate indexes were not provided with exam materials.

P No point of contact was available on security agreement.

25 Point Written??

Calulated the binomial probabilities of scoring less than 80% for various test sizes and applicant knowledge.

Assumed three different types of applicants, ones that could correctly answer 90%, 85%

and 80% of the questions correctly.

25 Point Written??

Failure Rates

% Knowledge 100 pts. 75 pts. 25 pts.

90 .08% .32% 3.5%

85 6.4 % 8.9 % 17.1 %

80 45.1 % 43.1 % 38.3 %