ML18086A774

LOR Program Biennial Inspection 71111.11B Exam Overlap Issues

ML18086A774
Person / Time
Site:	Millstone, Oconee, Columbia, Seabrook, Surry
Issue date:	02/21/2018
From:	Diane Jackson NRC Region 1
To:
Cushing E, NRR/DIRS, 415-1064
References
Download: ML18086A774 (13)

Text

LOR Program Biennial Inspection 71111.11B Exam Overlap Issues Donald Jackson Chief- Operations USNRC Region I February 21, 2018

10 CFR 55.49 Integrity of Examinations and Tests

• Applicants, licensees, and facility licensees shall not engage in any activity that compromises the integrity of any application, test or examination required by 10 CFR 55

• The integrity of a test or examination is considered compromised if any activity, regardless of intent, affected, or, but for detection, would have affected the equitable and consistent administration of the test or examination 2

Examination Security Section 3.06 of 71111.11

• Observe licensee implementation of examination security during implementation of the annual examination

• Review development, validation, and other activities to ensure no one was pre-exposed to examination information

• Check for excessive test item repetition between exams administered during different weeks of a training cycle 3

71111.11 Appendix E

• Proper sequestering of examinees to ensure elements of the examination cannot be transmitted between examinees

• Physical protection of examination materials and facilities are maintained

• Review of previous exam compromises during cycle

• No examination item duplication for those involved with development, validation, or administration 4

71111.11 Appendix E

• Biennial written exam administered during the training cycle must repeat </= 50% of examination questions from those administered previously in the training cycle

• Annual operating tests administered during the training cycle must repeat </= 50% of JPMs that have been administered previously in the training cycle

• Annual operating tests administered during the training cycle must repeat </= 50% of scenario events that have been administered previously in the training cycle 5

Scenario Events

• The examination overlap review is conducted at the event level

• Events that appear and are mitigated using the same indications, controls, and procedures between scenarios are considered the same event

• Not all LOCAs, or Controller malfunctions are the same, and MAY be considered different events.

• A Large Break LOCA, and a Small Break LOCA are different events.

They present differently and are mitigated by significantly different procedural actions

• Pressurizer master pressure controller malfunction vs spray valve controller failure, MAY be considered different events. They present differently and require different actions to mitigate.

• A controller failure low vs high vs intermediate MAY be considered different events IF they present differently and have different actions to mitigate 6

Millstone 2016

• Green NRC Identified NCV of 10 CFR 55.49, Integrity of Examinations and Tests

• 19 Unit 2 operators had received walkthrough exams where >= 50% had been repeated from previous exams in same cycle

• Specifically, 6 operators had 5 of 5 JPMs previously administered, 4 operators had 4 of 5 JPMs previously administered, and 9 operators had 3 of 5 JPMs previously administered, all during the same requalification exam cycle

• Development of the walkthrough exam was NOT in accordance with Dominion procedures 7

Seabrook 2017

• Green NRC Identified NCV of 10 CFR 55.49, Integrity of Examinations and Tests

• NRC requalification inspection determined that the biennial written examinations that had been developed would repeat >= 50% of the questions for the last 2 of 6 total exams that would be administered during the cycle

• Although these last 2 examinations had not been administered, a performance deficiency and violation existed due to the if not but for detection portion of the regulation

• There was confusion by the licensee concerning the NextEra fleet procedure which stated, each biennial written examination version shall consist of at least 50% new, modified, or different test items compared to all previously administered versions of the same exam.

• The licensee compared exam to exam versus each exam as compared to all administered or exposed questions 8

Oconee 2017

• Green NRC Identified NCV of 10 CFR 55.49, Integrity of Examinations and Tests

• For the 2016 and 2017 operating exam simulator scenario sets, the licensee consistently and predictably re-used one scenario from the previous weeks tests that were administered as a part of the same annual requalification exam

• This repetition represents >= 50% repetition of scenario events previously administered

• Licensee procedures for exam security measures were not implemented 9

Columbia 2016

• Severity Level IV NRC Identified NCV (Green) of 10 CFR 55.49, Integrity of Examinations and Tests

• 2 individual examinees had both of their 2015 annual exam scenarios that were 100% repeated as administered to a previous crew

• 3 individual examinees had both of their 2015 scenarios that were 67% repeated as administered to previous crews

• The LOR exam development procedure did not track exam development and administration down to the individual operator level, and could easily be misinterpreted leading to potential exam compromise 10

Surry 2017

• Green NRC Identified NCV of 10 CFR 55.49, Integrity of Examinations and Tests

• For the 2016 operating exam cycle, the licensee developed a retake set of simulator scenarios for a crew operating exam failure. The same two scenarios were used in totality for the next scheduled group of crews being examined during the same examination cycle.

• This repetition represents >= 50% repetition of scenario events previously administered (Actually, 100%)

• Licensee procedures for exam security measures were not correctly implemented 11

Thoughts

• Each individuals examination must have all elements that are at least 50% different from all other exam elements previously given

• The process must have integrity and can not even have the appearance or potential of exam compromise

• Honest mistakes happen. The recovery needs to involve a complete excision of the potential compromise to recover examination integrity

• Call your Regional Branch Chief when problems occur. It may still be a licensee identified violation, but in the end, appropriate corrective actions can be taken

• Consider how a member of the public would assess our process. It is unusual for the facility licensee to develop and administer a licensing process. The only way that it works is if the regulatory agency can ensure to the public, that the process has integrity that is beyond reproach.

• If the action does not feel or sound right, follow your instincts that it is probably not right. Our process covers most situations, but not ALL situations. Take actions that ensure that there is no potential for or appearance of compromise.

12

Questions???

(.and be gentle, please!!!)

13