ML17264A555
| ML17264A555 | |
| Person / Time | |
|---|---|
| Site: | Ginna  |
| Issue date: | 04/29/1996 |
| From: | Mecredy R ROCHESTER GAS & ELECTRIC CORP. |
| To: | Meyer G NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| Shared Package | |
| ML17264A554 | List: |
| References | |
| NUDOCS 9607180107 | |
| Download: ML17264A555 (8) | |
Text
AND ROCHfSTER GAS ANDElECTRIC CORPORAVON
~, 89 EAS1 AVfhlUE,ROCHESTER, N.Y. /46d9-0001 AREA CODE 716 546.2200 ROBERT C. MECREDY Vice President Nuclear Operations April 29, 1996 Mr. Glenn Meyer'hief, PWR S
BWR Section U.S. Nuclear Regulatory 'Commission Region 1
475 Allendale Rd.
King of Prussia, PA 19406
Reference:
EXAMINATIONREPORT NO. 50-244/96-02 (OL)
~
Dear Mr. Meyer:
As a result of the two written examination failures during the referenced examination, Rochester Gas E Electric Corporation has
'-conducted a root cause analysis to identify, the reason(s) for the failures.
A copy of this analysis is being sent to you per your request at the exit meeting on February 15, 1996.
Rochester Gas E Electric Corporation' management is currently reviewing the attached root cause recommendations for inclusion in the initial license program. If there are further questions regarding this report, please contact Mr. Gary Meier (Department Manager-Nuclear Training) at (716) 771-6622.
S'ely Robert C. Mecr dy cc G. Meier T. White 9607180107 960711 PDR ADOCK 05000244 U
Executive Summary Two operators failed a
written examination prepared by Rochester Gas E Electric Corporation.
Following an in depth
- review, Rochester Gas
& Electric Corporation submitted comments on the written which identified potential technical problems.
An analysis was conducted to determine the root cause.
Three primary causal factors were identified:
1.
Inadequate written examination validation 2.
Inadequate identification of objectives and KMs cross referencing 3.
Question Bank availability The Root Cause is the validation process.
Training guidelines will be updated to incorporate the lessons learned from this event.
Recommendations to prevent future occurrences are:
In the future, written examinations prepared by Rochester Gas S Electric Corporation and approved by the NRC should be ready at least two weeks before the scheduled examination date.
Validation should be by a full crew of operators and all comments should be used to improve the quality of the examination.
2.
Rochester Gas R Electric Corporation should review the NRC KEAs catalog to identify Ginna specific KMs. Initial license class objectives should be referenced to the K&As.
3.
Initial question bank should be closed to can-didates.
Root Cause
1995 Initial License Class Root Cause Report:
Event:
Time line:
Two initial license candidates failed a written examin-ation prepared and administered by Rochester Gas Electric Corporation.
November 1994 February 1995 September 1995 October 1995 November 1995 December 1995 License Class
- Begins, question bank available for their use.
I Volunteered,to be a pilot plant for new examination process provided man power available to support examination.
Final decision made to participate as a
pilot plant.
Review of Pilot Process materials indicates that new questions should be generated due to inability to account for candidates'eview of the question bank.
Examination Team identified. Examination construction begins.
Sample Plan mailed.to NRC.
Audit of
- class, performed by a
team from a
s imilar nuclear
- plant, detected no significant concerns.
January 1996 February 5,
1996 February 8,
1996 February 13, 1996 Draft written examination questions mailed to NRC for comment.
Comments incorporated as they are received.
Final package mailed out to NRC as scheduled.
Outside Contractor audit of class detected no candidate weaknesses.
License review board
- meets, reviews
- records, and certifies candidates are ready to be examined.
All comments from NRC resolved.
NRC completes check against Audit examination for duplicate questions.
Examination is ready.
Time validation is completed.
Written examination administered and graded.
Root Cause
Event Causal Factors:
Inadequate examination validation process:
The only validation of the written portion by non-exam team members was done by a single operator.
This was done four days prior to written administration.
Conditions:
a)
License class program schedule was laid out based on the NRC developing the examination.
Switching to the pilot program's required time line created delays in sending material for the NRC's final review.
b)
Examination compromise and security breach were significant concerns on the part of the NRC.
Heavy emphasis was placed on minimizing the number of people that were allowed to sign onto the 'examination security agreement.
As a result, the examination team decided not to let anyone other than team members review questions until the final product was approved.
BARRIER FAILURE c)
Final NRC approval was not received until February 5.,
1996. This was one week past the planned date.
Operators scheduled to take the written for time validation were no longer available due to other plant commitments.
Only one operator was available on Friday, February 9, 1996 to validate the written.-BARRIER FAILURE d)
Supervisor-Operations Training went to an unplanned MANTG Managers meeting on February 4
and 5," 1996.
He did not participate in or provide clear direction regarding the expected validation process.
BARRIER FAILURE e)
The Lead RGSE examiner had an operator validate the written for time only.
He did not think that there was enough time to incorporate validation comments before the written was administered.
BARRIER FAILURE Critical Human Actions:
Questions should have been reviewed for clarity and technical content by non-exam team members.
Root Cause
Critical Human Actions Profile:
Experience with new examination questions has shown that preview by two or more operators has increased question
- validity, clarity, and.
technical content.
Preview by operators who were not involved with the development process should have been used.
The preview should have taken place as soon as the draft questions were sent back by the HRC. This preview could have been conducted in the training center with all operators signing onto the security agreement.
New comments could then have been submitted to the NRC for further. review.
RE OMMENDATION:
For future NRC examinations, the final product should be ready at least two weeks prior to administration.
The final product should be administered to an entire crew of operators for time,
- content,
.and technical validity.
The validation should be given in exam like conditions,
- graded, and include a
complete item analysis.
- Results, problems, and suggested improvements should be communicated to the NRC Lead Examiner for resolution within two days.
2.
Inadequate identification of objectives and cross referencing:
Many Initial license program objectives do not reflect the actual NRC examination conditions rather they reflect the conditions under which operators will perform and be tested at Ginna.
The cross referencing between these objectives and the Knowledge and Abil'ity items (KKdks) identified in NUREG 1121 (Knowledge and Abilities Catalog for Nuclear Power Plant Operators:
Pressurized Water Reactors) is not done in a consistent manner.
Conditions:
a)
All licensed operator objectives, initial and continuing, are approved by the Operations Cur-ricuulum Committee with the implicit knowledge of the conditions found in the job environ'ment.
The objective approval process is consistent with a Systematic Approach to Training.
Root Cause
b)
No direct cross reference between objectives and K&As exist.
This has never been a
problem with previous NRC initial examinations as the NRC Lead Examiners were interested in questions not objec-tives.
c) d)
The Ginna Question Bank is cross referenced to the K&As but was not used due to the development of new questions.
Candidates used the objectives to study for the NRC examination.
If the objective gave the condition "Using the procedure" they did not spend time reviewing the material as they expected to be given a copy of the procedure.
Critical Human Actions:
Curriculum Committee should review and approve objectives that are specifically designed for conditions utilized on the NRC Initial License Exam.
Critical Human Actions Profile Material developers should develop initial license class lesson objectives that have conditions similar to those contained in the K&As.
This should be done du ing the development stage and approved as written by the Operations Curriculum Committee.
RECOMMENDATION Training and Operations should review the K&As for applicability to Ginna Station.
There should be a
documented basis for any that do not apply.
Once the applicable K&As are identified, missing initial license objectives, for both ROs and
3.
Question Bank Availability.
Allowing the question bank to be open (available to students at all "times), limited the number of questions available for use on the NRC. exam. This availability may also have led to artificially elevated test scores.
Root Cause
Conditions:
a) b)
c) d)
The question bank was available to the candidates on the first day 'of the class.
Original release of the question bank to Operations personnel took place when the bank size had increased to over 1000 questions.
Previous initial license candidates had successfully used the question bank.
Criteria for using the question bank for the pilot examination were much more restrictive than those used by NRC examiners.
Whereas previous NRC writtens contained varying numbers of Ginna questions, thi's written contained all new questions.
Candidates were typically the first to complete the progress writtens administered throughout the program.
Scores on these writtens were consistently 95% or better with no identified areas of weakness.
These results led to a lack of focus for final study for the NRC exam.
dl A minor comment during both audit exarch exits was that the operator candidates seemed to have trouble with new questions but would do well
'overall.
Critical Human Actions:
a) b)
Training and Operations Management should have closed the question bank.
The License Review Board should evaluate each comment from the audit
- exam, despite overall
- results, to determine if further training or evaluation is required.
Critical Human Actions Profile:
Improper use of the question bank as a study guide led to a false sense of security.
High grades on all progress writtens in the class room phase.of the program should have been evaluated by Training more thoroughly.
RECOMMENDATION:
The initial question bank should be closed to all Operations personnel.
Root Cause
==
Conclusion:==
The root cause is an inadequate written validation. Although the other causal factors had a significant impact on the outcome of the written, the examination post mortem revealed that the written instrument had more problems than the team realized.
Xndependent review by non-exam team personnel would have offered a
much better quality check.
Root Cause