NLS2002027, Follow-up Response to the Regulatory Conference Held February 1, 2002 in Regard to an Apparent Violation at Cooper Nuclear Station

Follow-up Response to the Regulatory Conference Held February 1, 2002 in Regard to an Apparent Violation at Cooper Nuclear Station

ML020650463
Person / Time
Site:	Cooper
Issue date:	02/15/2002
From:	Denise Wilson Nebraska Public Power District (NPPD)
To:	Document Control Desk, Office of Nuclear Reactor Regulation, NRC Region 4
References
NLS2002027
Download: ML020650463 (8)

Text

N Nebraska Public Power District Nebraska's Energy Leader NLS2002027 February 15, 2002 U.S. Nuclear Regulatory Commission Attention: Document Control Desk Washington, D.C. 20555-0001

Subject:

Follow-up Response to the Regulatory Conference Held February 1, 2002 in Regard to an Apparent Violation at Cooper Nuclear Station.

Cooper Nuclear Station, NRC Docket 50-298, DPR-46 A Regulatory Conference was held February 1, 2002 in Arlington, Texas to discuss an apparent violation of 10 CFR 50.49 that occurred at Cooper Nuclear Station in June and July of 2000. The purpose of this Regulatory Conference was for Nebraska Public Power District to provide additional information and discuss its evaluation as to safety significance of the issue. As an outcome of the conference, there was one issue regarding the method of regrading the exam that was unresolved and which required followup and closure. Attachment 1 to this letter contains additional information related to the potential exam compromise and regrading of the exam.

Attachment 2 contains examination regrade results.

If you have any questions please contact Mr. David Kunsemiller, Risk and Regulatory Affairs Manager at (402) 825-5236.

Sincerely,

• David L. Wilsn Vice President of Nuclear Energy

/dwv cc: Regional Administrator w/ attachments USNRC - Region IV Senior Project Manager w/ attachments USNRC - NRR Project Directorate IV-1 Senior Resident Inspector w/ attachments USNRC NPG Distribution w/o attachments Records w/ attachments Cooper Nuclear Station .- i P.O. Box 98/ Brownville, NE 68321-0098 Telephone: (402) 825-3811 / Fax: (402) 825-5211 http://www.nppd.com

NLS2002027 Page 1 of 3 Attachment 1 Response to Request for Additional Information Related to Exam Regrading

NLS2002027 Page 2 of 3 Request:

The Nuclear Regulatory Commission (NRC) requested that Nebraska Public Power District (NPPD) provide information regarding the results of an exam regrading exercise in which only those contested questions answered correctly would be thrown out.

Response

NPPD is providing a response to this request below, however, NPPD does not agree that the selective disqualification of exam answers on only those similar questions that were answered correctly is a fair and equitable grading process. Since it is not possible to know which operators took which portion of the exam and therefore which operator derived potential advantage, it is not appropriate to eliminate only a portion of the answers from the exam bank and tests. Based on equity and consistency, the appropriate method is to remove the unacceptable question from all exams and then regrade. The results of such a regrade were provided in NPPD's letter to the NRC dated January 25, 2002. A fair and equitable regrading would result in no additional operators failing the exam.

The central issues that would drive a regrading of the exam are: (1) whether the exam question fails to meet the regulatory guidance for similarity and (2) whether the validation process resulted in exposure of the operator to exam questions in a manner that would violate regulatory guidance.

NPPD provides the following discussion regarding the preceding two issues:

I. NUREG 1021 Appendix A provides guidance regarding similarity of questions seen in training and actual exam conditions. This guidance states, "The most effective tests of knowledge include questions and test items that measure applications of knowledge directly related to the job." In addition, it states, "Test items should attempt to assess similar knowledge applications in different contexts, thereby assessing the problem solving skills of students in new and different applications." Similar questions are not prohibited by regulation and are actually encouraged.

2. The NRC has provided guidance in NUREG 1021 ES-401, which clarifies what to do to limit exposure between questions seen on previous exams. This guidance provides specific criteria as to when a question would no longer be considered a duplicate (i.e., new or significantly modified). A significantly modified question is one where "at least one pertinent condition in the stem and at least one distractor" are changed. All similar questions between successive requalification exams in 2000 were significantly modified, and none could be answered solely based upon exposure to a potentially similar question. Even though the questions were potentially similar, all were significantly modified and the individual had to apply the knowledge to obtain the correct answer.

NLS2002027 Page 3 of 3

3. NUREG 1021 ES-401, Supplement 1 also provides guidance to ensure those practice exams given immediately preceding NRC exams have controls to prevent compromise of examination integrity. These guidelines allow practice exams up to the day before provided there are no duplicate questions between them and the NRC examination. As was stated above, the exams administered in 2000 did not have duplicate questions.

4. The maximum number of questions on the exams that were potentially similar is substantially less than 20% of the 210 total questions. In addition, the exams still met the criteria of 10CFR 55.41 and 55.43 for written examinations. NPPD firmly concludes that while the validation process was not equitable and while it did provide some exposure to similar exam questions that level of exposure is permissible by NUREG 1021.

5. Request for information regarding regrading at the Regulatory Conference is provided in the attached table. The table demonstrates that a bias is introduced by such a regrading in that an operator can only do worse than their original score, and does not represent a homogeneous picture of actual results. Clearly, if enough questions are thrown out using these guidelines, everyone would eventually fail the exam. This is an inconsistent and inequitable grading method that unduly punishes those individuals who scored near 80%.

NLS2002027 Page 1 of 3 Attachment 2 Exam Regrade Data

NLS2002027 Page 2 of 3 Regrade results for 5 NRC identified question sets and 4 NPPD identified question sets. This regrade does not remove a question if it was missed. The bold items are the only test scores that are different from the results provided to the NRC in the letter submitted January 25, 2002.

Original Score NRC Questions NRC and CNS Removed Questions Removed Test 1 Grade Regrade Regrade SRO 91.4 90.9 90.9 SRO 82.8 82.3 82.3 SRO 91.4 90.9 90.9 RO 88.5 88.2 88.2 RO 82.8 82.3 82.3 Test 2 SRO 80.0 79.4 79.4 SRO 88.6 87.8 87.8 SRO 88.6 87.8 87.8 SRO 91.4 90.9 90.9 SRO 91.4 90.9 90.9 SRO 82.8 81.8 81.8 SRO 94.3 93.9 93.9 RO 94.3 93.9 93.9 RO 85.7 84.8 84.8 RO 88.6 87.8 87.8

NLS2002027 Page 3 of 3 Original Score NRC Questions NRC and CNS Removed Questions Removed Test 3 SRO 85.7 85.7 85.3 SRO 85.7 85.7 85.3 RO 77.1 77.1 77.1 RO 80.0 80.0 80.0 RO 82.8 82.8 82.8 Test 4 SRO 88.6 87.9 87.5 SRO 91.4 90.6 90.3 SRO 85.7 84.4 83.9 RO 85.7 84.4 84.4 RO 71.4 68.7 68.7 Test 5 SRO 71.4 70.6 70.6 SRO 80.0 79.4 79.4 SRO 85.7 84.8 84.8 SRO 88.6 87.8 87.8 RO 91.4 91.4 91.4 RO 82.9 82.9 82.9

IATTACHMENT 3 LIST OF REGULATORY COMMITMENTS Correspondence Number: NLS2002027 The following table identifies those actions committed to by the District in this document.

Any other actions discussed in the submittal represent intended or planned actions by the District. They are described for information only and are not regulatory commitments.

Please notify the NL&S Manager at Cooper Nuclear Station of any questions regarding this document or any associated regulatory commitments.

COMMITTED DATE COMMITMENT OR OUTAGE None.

J.

I I-I-

4-I _________________________________________________________________

I PROCEDURE 0.42 1 REVISION 10 1 PAGE 12 OF 14