IR 05000293/1996080
| ML20141E143 | |
| Person / Time | |
|---|---|
| Site: | Pilgrim |
| Issue date: | 06/20/1997 |
| From: | Conte R NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION I) |
| To: | Boulette E BOSTON EDISON CO. |
| References | |
| 50-293-96-06, 50-293-96-6, 50-293-96-80, NUDOCS 9706300252 | |
| Download: ML20141E143 (3) | |
Text
, l e
.
kh June 20, 1997 l
l E. Thomas Boulette, PhD l Senior Vice President - Nuclear Boston Edison Company
- Pilgrim Nuclear Power Station l 600 Rocky Hill Road Plymouth, Massachusetts 02360-5599 SUBJECT
- INSPECTION REPORTS 50 293/96-80 AND 50-293/96-06 I
Dear Mr. Boulette:
This refers to the June 17,1996, January 31,1997, and February 28,1997 correspondences in response to our letters dated April 16,1996 and October 30,1996 regarding the Pilgrim l Station. These correspondences dealt with your response to the 40500 Team Inspection l Report, performed January 1996, and the associated Notice of Violation 50-293/96-06-02.
Thank you for informing us of your corrective actions. We have reviewed this matter in accordance with NRC Inspection Manual Procedure 92901, " Follow-up - Operations." We preliminarily agree with your assessment of the root and contributing causes for the failure to -
have taken effective corrective actions for a number of events in 1995 and 1996 involving
,
procedural adherence and adequacy problems. The actions you have taken or have proposed l for the resolution of the corrective action process programmatic weaknesses, as wellas those l actions to address the cited examples of procedural non-compliance and procedure i inadequacies appear to be satisfactory. Those broad corrective actions will be examined during future inspections to assess their overall effectiveness for resolving identified problems and will also be a focus area for our scheduled 40500 team inspection starting the week of
- July 21,1997.
We continue to urge you and your staff to diligently examine significant plant events and
,
problems to ensure the root and contributing causes are thoroughly and clearlyidentified and l proper corrective actions are implemented to prevent recurrence.
Your cooperation with us is appreciated.
'
j
Sincerely, l
'
!
Original Signed By: I I
, i i l Richard J. Conte, Chief i
- Projects Branch 8
{ Division of Reactor Projects
!
'
Docket No. 50-293 License No. DPR-35 . ol l 9706300252 970620 llllhll!k!Ikkkkkk!
. i=^ , l i
'
ADOCK 05000293 G PDR
/
,
.
4.. -)
l i E. Thomas Boulette, PhD 2 cc w/o cv of Licensee Response:
L. Olivier, Vice President - Nuclear and Station Director
- T. Sullivan, Plant Department Manager
- N. Desmond, Regulatory Relations D. Tarantino, Nuclear Information Manager i
! cc w/cv of Licensee Resoonse:
i R. Hallisey, Department of Public Health, Commonwealth of Massachusetts '
The Honorable Therese Murray B. Abbanat, Department of Public Utilities
'
Chairman, Plymouth Board of Selectmen Chairman, Dur. bury Board of Selectmen l
Chairman, Nuclear Matters Committee l
Plymouth Civil Defense Director
'
P. Gromer, Massachusetts Secretcry of Energy Resources
,
'
J. Shaer, Legisic'.ive Assistant J. Fleming i A. Nocee, MASSPIRG
, Regional Administrator, FEMA i
Office of the Comn.issioner, Massachusetts Department of Environmental Quality s Engineering l Office of the Attorney General, Commonw salth of Massachusetts
!^ T. Rapone, naassachusetts Executive Office of Public Safety Chairman, Citizens Urging Responsible Energy j Commonwealth of Massachusetts, SLO Designee
-
. _ _ _ _ _ . _ . _ _ _ _ _ . . _ . _ _ _ _ _ _ _ _ _ _ _ _ _ . . _ . _ . . .
l l
l' *
l t
i E. Thomas Boulette, PhD 3 l'
Distribution w/ encl:
Region i Docket Room (with concurrences)
PUBLIC Nuclear Safety information Center (NSIC)~
D. Screnci, PAO l NRC Resident inspector l R. Conte, DRP E. Connor, DRP l C. O'Daniell, DRP Distribution w/enci (VIA E-MAIL):
P. Milano, NRR A. Wang, NRR W. Dean, OEDO R. Correia, NRR i F. Talbot, NRR '
DOCDESK l Inspection Program Branch, NRR (IPAS)
I i
l
,
l l
! l I
t .
'
OOCUMENT NAME: G:\ BRANCH 8\RPLYLTR\NOVRESP.PIL 1 To receive a copy of this document, indicate in the box: "C" = Copy without attachment! enclosure "E" = Copy with attachment / enclosure
'N' = No copy 1 i
OFFICE Rl/DRP Rl/DRP Rl/DRP /
NAME EConner 72C RLaura eg RConte 4'> y-p i DATE 6/18/97 6/1$/97 # ' " 6/1S/97 " " -
OFFICIAL RECORD COPY
,
. _ _ _ _
F....
a
.
l
.
'
. saatan emean Pilgrim Nuclear Power Station l
,
Rocky Hill Road Plymouth, Massachusetts 02360
[
J E. T. Boulette, PhD Senior Vice President-Nuclear ,
l
.
'
January 31,1997 l BECo Ltr. 2.97-009 i
,
U.S. Nuclear Regulatory Commission Attention: Document Control Desk Washington, DC 20555 -
l Docket No. 50-293 l License No. DPR-35 i
!
'
Roolv to Notice of Violation 96-06-02 l t
'
' 0
, We have provided in this letter our response to Violation 96-06-02 that describes procedural usage
\ - and adequacy problems at Pilgrim Station. As we stated in previous correspondence (BECo Ltr. j
'
- 2.53-057 dated June 17,1996), efforts were underway to resolve problems within these areas when the NRC issued the 40500 team inspection report (IR 96-80 dated April 16,1996). The results of the NRC report offered insights into our procedural weaknesses, and those insights were factored into our performance improvement plan.
As reerted, we have provided in this letter the reasons for the violation, corrective action taken )
and resuits achieved, actions planned to avoid further violations, and the date when full compliance will be achieved. Furthermore, we have included a clarification of our integrated plan to correct the identified problems and completion dates for corrective actions specified in our June 1996 letter.
Finally, we have included a summary of our common cause analysis. This informatiori is contained in the enclosure to this letter.
The letter contains the following commitments:
. Evaluate and redesign our process for changing procedures, as appropriate. The new process will be implemented by the end of September 1997. '
. Consolidate our Mission, Organization and Policy Manual and Nuclear Organization Procedures then review and revise, as necessary, our NOPs and administrative procedures by the end of 1997.
. Submit a change by the end of September 1997 to the NRC that will allow us to consolidate our deficiency report and nonconformance report processes with our problem report process .
. Provide training on a periodic basis to reinbrce management expectations on procedural adherence. The first of these sessions will be held prior to come ; coq our February outage.
,
'
}"% $
I '
. . . _ _ _ . . - - . , , _,
_. . . . . _ _ . _ _. . . .
.
. Evaluate and redesign our modification process as appropriate. The new process will be implemented by the end of September 1997.
! v . Complete initial training on our new root cause analysis (RCA) and human error prevention I
techniques by the end of September 1997. We will also have incorporated RCA refresher training into our continued training program by this date.
'
$
. T. Boulette, PhD Attachment: Reply to Notice of Violation JWK\dmc\radmisc\v96-068
, cc: Mr. Alan B. Wang, Project Manager l
Proiect Directorate 1-1 Office Of Nuclear Reactor Regulation Mail Stop: 1482 1 ' White Flint North 11555 Rockville Pike Rockville, MD 20852 U.S. Nuclear Regulatory Commission Region i 475 Allendale Road King of Prussia, PA 19406 d Senior Resident inspector Pilgrim Nuclear Power Station l
l
'
J
.
l l l a
-
.,. .,
,
.
>
.-
-
ENCLOSURE REPLY TO NOTICE OF VIOLATION BACKGROUN.D in April 1996, the NRC issued the results of the 40500 team inspection of the Pilgrim corrective action process (IR 96-80). The NRC identified what appeared to be a broad procedural usage and adequacy issue. Boston Edison (BECo) replied to that report in June 1996 (BECo Ltr #96-057) acknowledging the NRC findings were consistent with the results of self assessments performed by BECo personnel.
We also provided our corrective action plan. In October 1996, the f.'RC issued IR 96-06 containing the following violation:
"As a result of an inspection conducted January 22,1996 through February 9,1996, information received on June 17,1996, and additional inspection conducted during this inspection period (July 29 - September 23,1996), the following violation of NRC requirements was identified. In accordance with the NRC Enforcement Policy (60 FR 34381; June 30, 1995), the violation is described below:
Criterion XVI, " Corrective Action," of 10 CFR Part 50, Appendix B, states, in part, that measures shall be established to assure that conditions adverse to quality are promptly ,
identified and corrected. In the case of significant conditions adverse to quality, the measures i shall assure that the cause of the condition is determined and corrective action taken to I preclude repetition.
Contrary to tne above, corrective actions for problems identified since at least April 1995, were not effective in identifying and correcting overall procedural adherence an6. adequacy problems at Pilgrim. The following specific procedural adherence and adequacy problems were identified:
1. Procedure 2.2.87, Control Rod Drive System, Revision 53, was inadequate by not providing direction to operators moving reactor fuel to verify the correct orientation of the blade guide before control rod insertion. As a result, on April 30,1995, control rod 18-35 was inserted into the reactor core and physically jammed into a mispositioned blade guide causing blade guide and control rod damage.
2. Procedure 9.13, Control Rod Sequence and Movement Control, Revision 12, Attachment 3 listed control rods to be moved. During the power reduction required on October 6,1995, operators did not follow the reverse order of the pull sheet as evidenced when control rod 34-23 became mispositioned.
3. Nuclear Operating Procedure NOP 92A1, " Problem Report Program," Step 6.5.3 directs that Severity Level i problem report evaluations be forwarded to NSRAC for their review.
However, as of January 1996, the NRC identified that NSRAC had not been forwarded approximately one-third (60) of the Level I evaluations.
l
_ _ _ _ _ . . . - _ . . _ _ _ _ _ _ _ _ _ . _ . . _ _ _ _ . _ . . . _ _ _
.: .
.
.
p r
4. On July 31,1996 valve MO-1001-16A ("A" RHR loop heat exchanger bypass) became mispositioned when a reactor operator failed to follow the established RHR procedure when securing torus cooling. The valve was mispositioned for approximately 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br />.
5. In February 1995, a calibration problem on the "B" scale of the Rockwell Hardness Tester machine was adverse to quality and was not promptly identified or corrected. Quality control inspectors relied on verbal advice from an outside vendor rather than contacting the BECo measurement and test equipment personnel, as specified in the M&TE program, or l
i initiating a problem report in accordance with NOP 92A1, " Problem Report Program". Also, the degradation of a related BECo calibration block has not adequately been addressed to date.
This is a Severity Level IV violation (Supplement IV)."
In this response, we will address the overall problem of procedural usage and adequacy rather than each of the specific events described; however, each of the specific events have been entered into l our corrective action process. The details of the requested information follows.
REQUESTED INFORMATION
_ Common Cause Analysis Summarv
' The common cause analysis of problem reports was completed by a team trained in the Failure Prevention & Investigation Intemational (FPI) methodologies for determination and correction of human performance based errors. The report provides a detailed trend review of all problem report (PR) evaluations issued between June 1995 and June 1996. Of the 726 prs generated during this interval,227 were determined to be caused by human performance based errors. These results were further evaluated to determine the underlying causes of organizational, programmatic, and personnel performance problems. The report also contained the results of the first Pilgrim culture index survey.
Major findings from the common cause analysis report included:
. The overall number of human performance inappropriate actions documented by prs is low compared to industry averages and performance indicators. This is due to the perception that punitive measures will accompany reports of human performance issues and the lack of confidence that the corrective action process is effective for correcting human performance issues
. The modification process accounted for 17% of all inappropriate actions. These inappropriate actions were fairly evenly distributed between the design and implementing organizations.
,
. The personnel accountability system needs improvement. Accountability is perceived as
'
inequitable within and across organizations and is applied according to the individuals
- involved.
! . Of all trended events,15% were associated with procedure noncompliance. Most of these i deviations were associated with extemal factors (e.g., misjudgments, not meeting commitment)
O and not a product of deliberate violation.
d V
I
~ . - . - . -- -. .. . -_ -_ - ... . . --
_ . _ - - . - -
.
.- - - - _ . - - - . - - . - - - . . . . - - - - . - - . - . . -
'
-
l ,
i 1
.. .
j
-
.
- 4 Major recornmendations arising from the common cause analysis report were. 4
\
} (' . Executives and group managers should review the current personnel accountability system.
- Reward and punishment should be balanced throughout all departments. Awards should be considered for multi-discipline efforts to encourage teambuilding. Action should be taken to reverse the negative stigma associated with identifying and reporting human performance
[
errors.
e Group and department managers should explain the purposes of the PR process and the I
.
Human Error Prevention program to workers. Identifying problems via the PR process and 1
. using human error prevention and performance improvement techniques should be
encouraged. Leadership by example is encouraged.
i e An independent team should review (in-depth) procedum usage at Pilgrim Station. This review i
should include:
'
1. Level of detail within procedures as it relates to the complexity of the task, safety j significance, and skill of the craft.
l 2. Methodologies for simplifying the procedure revision process to make it easier, more
! cost effective, and timely, j 3. Required training of personnel to minimize level of detailin procedures.
i 4. Procedure revision tracking.
i 5. Procedure revision trending.
- e An independent team should review the administrative, technical, and review / approval
!
requirements of the modification process to minimize associated performance errors.
- N e Group and department managers should review all sections of the common cause analysis l j and culture index to use it as a platform for salf assessment within their organizations.
To assess the culture index at Pilgrim Station, a survey was conducted also using FPI methodologies.
3 The index is a leading indicator that forecasts whether performance will improve, degrade, or remain
! the same.
Reasons for the Violation As explained in our June 1996 letter and the common cause analysis / cultural index results, several weaknesses worked together to produce the overait procedural usage and adequacy problem described in the violation. The primary cause of these weaknesses was a corrective action process that dic' not provide consistent depth of analysis and performance monitoring capabilities to inform mant 4 : ment of adverse trends in procedural usage and adherance. Several programmatic weakness wert ,dentified that contributed to this deficiency.
One of these weaknesses was manifested in the form of inconsistent root cause analyses (RCAs).
Information of varying depth and scope, lack of independent review, missing er incomplete information, use of involved parties as evaluators, and lack of input from involvei parties were issues of concern. Other general concems with the root cause analysis process were iriadequate examination of procedural usage factors (e.g., adherence, usage, adequacy, and understanding) and poor quality of RCA documentation packages. As will be discussed, the quality of our RCA process
.
has a direct impact on our ability to identify and trend procedural usage causal factors.
i -
% -- p-- gr- P ~ -
f+- --' ,9 ---&-- +-- +
.
. . _ .
'
'
.. .
.
i p The root cause for the problems associated with RCAs was the inconsistent quality of training j provided to those involved in the RCA process. Contributing to the problem were procedures that did not require independent review of RCAs and allowed the use of involved parties as evaluators.
Another contributor to this condition was an inconsistent understanding within the organization of how organizations behave, how programs can fail, and how humans can err.
Second, the trending aspect of our corrective action process was weak particularly in the area of human performance. The ability to trend problem report (PR) and deficiency report (DR) data was hampered by the difficulty in translating causal factors from RCAs into the integrated action database (IADB). The cause for this condition was outmoded corrective action methods and procedures.
Third, independent oversight groups did not identify the broad-based proceduralissues within our organization. Contributing to this problem were the ineffective performance indicators used to analyze
- human performance issues. This in tum was caused partly by the RCA weaknesses already i discussed that resulted in poor data input to the trending process. Pocr quality data input to the trending process created obstacles to detecting the procedural problems.
1
- Fourth, our less-than-efficient procedure change process made it difficult for individuals attempting to !
make improvements. I Corrective Actions Taken To Prevent Recurrence i Human Error Prevention Training l O We contracted FPI intemational, who are proven performers and are widely recognized leaders in the l
( industry, to provide training to our organization on failure prevention methodology. This training provided the key elements of a strong self improvement culture as well as the knowledge and skills for investigating human performance, organizational, and programmatic failure modes.
i Approximately 85% of all Pilgrim Station executives and managers have already attended various l training sessions that introduced fundamental techniques for identification and correction of
- programmatic and human errors, l
Root Cause Analysis Training Also,30 workers have been trained as RCA experts using our new methods; every future RCA team l must have. at least one member trained to this level. Through this training, we expect to see
! consistent kC As that significantly improve the quality of data fed into our trending process.
Additionally, many managers and workers have attended human error prevention training.
Problem Report / Corrective Action Process The corrective action process has undergone significant revision as the result of insights gained
- during our line organization participation in the FPI training. Organization-wide, open forum training j sessions were personally conducted by the Plant Manager and operations support personnel on the j new process. Key elements of the training included focus on the need foridentification of problems at
! a very low thrcshold, particularly human performance errors.
e
'
f
!\
e
!
.
.
To encourage reporting of human performance errors, a Pilgrim Error Free Policy endorsed by the (9 entire executive team was introduced at the corrective action training sessions. This policy states that (~) no disciplinary action will accompany identified human errors unless the errors are willfulin nature.
The policy is aimed at eliminating the negative stigma perceived to accompany the reporting of human i errors. We feel the most effective way to solve human performance issues is to set high expectations
! for performance, hold managers accountable for results, and solve the environmental factors that l cause problems rather than place blame, i Also the PR initiation process was simplified to further encourage workers to identify problems.
Perhaps the most significant change, however, was the method by which prs are now screened and l
categorized. The four significance levels by which prs were previously categorized were condensed into two categories: significant conditions adverse to quality and non-significant conditions adverse to quality.
Under the revised process, only significant conditions adverse to quality require RCAs. Using the new root cause methods, procedural usage and adequacy causal factors will be identified and appropriate corrective actions determined.
Non-significant conditions adverse to quality are assigned either apparent cause or direct cause evaluations. All non-significant conditions adverse to quality will require the apparent or direct cause .
be identified for trending purposes and the immediate problem be fixed. Causal factors from all these '
evaluations will be fed into our improved trending process thereby improving the quality of our trend I reports. l l
Another improvement in the corrective action process relates to senior management involvement in )
[ l ()j thatPilgrim process. The senior Management licensed team continues individual to be at Pilgrim briefed on potentially continues significant prs as to partreview all new prs da of the station
- Plan of the Day Meeting. The Plant Manager, Engineering Manager, and other senior managers as l determined by the subject being reviewed will form the Corrective Action Review Board whose charter is to review all RCAs. RCAs will be performed on all problems that merit a high degree of attention.
,
Together, these changes in the corrective action process are expected to improve participation in our l l '
corrective action process.
Self Assessment Process and Oversight The self assessment process has been revised with emphasis on increased management participation ano oversight. Key elements include worker level assessments and quarterly group level management l assessments. Cross-functional assessments are conducted by each group every 18 months. The Nuclear Managers Committee (NMC) and the Nuclear Safety Review and Audit Committee review all group self assessments.
The independent Oversight Team (IOT) has been formed to monitor and trend performance. As part of their charter, the IOT will also independently review RCAs, self assessments, human performance corrective actions, and other data. These reviews will ensure high quality RCAs are consistently performed and will determine the effectiveness of human error corrective actions in preventing recurrence. The IOT willinclude the observations from management tours (e.g., performance review checklists) during their real-time trending of human performance.
O)
i v
. . . . . - . . ..
.; ,
l
. ,
The Quality Assurance Group will continue to oversee organization activities through various audits and assessments.
Management tours, observations, and discussions with workers continue with the expectation that workers will appreciate management commitment to reducing human errors. Managers are expected to reinforce the need for procedural compliance during routine tours. These tours have the added benefit of sensitizing managers to worker needs and concems.
Performance Indicators and Trend Reports The corrective action process has been revised to maximize the use of information from root cause, apparent cause, and direct cause analyses. Data is also provided from management observations of human performance during routine tours. A new work instruction, " Performance Evaluation Program" contains guidance on what types of errors are typically made for different types of work and will also provide a source of data for trending.
The capability to monitor and trend human performance has been improved by adding leading, real-time, and lagging indicators to the program although it is too early to measure their effectiveness. Our monthly human error performance reports include graphs of our human performance ratio, human performance prs, personnel error rate, significant personnel error rate, and LER 4-quarter rolling average. Use of these indicators in performance monitoring enhances our ability to identify current problems and determine whether trends are significant enough to merit immediate corrective action to avoid significant errors in the future.
Common cause analysis reports are expected to be generated on a 6 month frequency and culture index surveys aoproximately every 12-18 months.
MOP /NOP Revisions
Wo have completed a top down review of the Nuclear Organization Mission, Organization and Policy Ma,'ual (MOP), the Nuclear Operations Procedures (NOPs), and administrative procedures. From this review, we developed a plan for consolidation and revision as necessary to ensure the policies and procedures properly communicate management expectations and standards. Also expected is a clear statement of organization oversight responsibilities and simplified / streamlined processes.
Effectiveness of Actions Taken The December Human Performance Trend Report provides an indication that corrective actions taken have had a positive effect on the organization. During December,31 of 153 prs written were human performance related. Approximately 60% of the human pcrformance issues were self identified. The December self identification percentage represents a favorable increase. Also, from this data, we see that even though more prs are being submitted as the result of k wering the threshold for prs, the ratio of human performance problems to total problems has remained fairly constant.
During December,19 of the 31 human performance prs were associated with the incorrect use of administrative procedures. Although most of the errors have been generally minor in nature, this s
.
. . . . . . . . .. ...
_
. _ . _ _ _ __ _ _ __ _ . _ . . . _ _ _ _ _ __ _ _
l
- , -
.
> .
.
- (p) highlights the need focus maintain management to complete on attentionour review and revision of NOPs by the end of June 1 to detail.
V
'
'
Two other indicators, the personnel error rate and significant personnel error rate, took a tum in the adverse direction during the past month. Indeed, the station goal of 0.07 for the significant personnel i error rate (approximately 1 Level I human error PR per 120,000 man-hours) was not achieved at year i
end; actual year end performance was 0.1. This was due primarily to the fact that three relatively
- minor prs were issued on the lifted lead and jumper log process during December. Taken individually, these prs might have been treated as non-significant. However, they were viewed collectively as significant since they involved the same activity, and they were categorized accordingly.
l Therefore, this single process weakness was documented by 3 significance 1 prs and caused the disproportionate spike on the significant personnel error rate trend.
.
'
Another indicator trended at Pilgrim Station is the human performance ratio (HPR). The HPR is a
!
potential leading indicator that compares the number of significant human performance prs to the total number of human performance prs each month. Also trended on the same graph is the rolling average of monthly HPRs. In December, the Pilgrim Station HPR rolling average decreased for the
. sixth consecutive month even though the monthly HPR increased over the previous month's HPR
!
because of the 3 prs issued on the lifted lead and jumper log. The LER rate, a lagging indicator, also favorably declined in December.
i l l Intearated Corrective Action Plan Procedures Process Redesign O The scope of this effort is to redesign and implement the process for de i
approving procedures. The new process will assure procedure revisions are produced in a timely manner, and it will require a minimum of resources to maintain. Goals of this project include:
a reduce the effort required to produce quality procedures e improve the use of procedures in the work control process e reduce the cycle time for the development of new and modified procedures e improve the quality of procedures A multi-functional breakout team will be assigned to identify required process chrges, implement the new process, and train on it by the end of September 1997.
The Plant Manager will provide training on a periodic basis to reinforce management expectations on procedural adherence. The first of these sessions will be held prior to commencing our February refueling outage.
MOP /NOP Revisions A top down review of the MOPS, NOPs, and administrative procedures was completed. We will integrate into lower tier implementing procedures many of the higher level policies currently contained in MOPS with the expectation that a clearer tie between policies and implementing procedures will be provided. The remaining policies will be consolidated into a single NOP on policies.
Also, NOPs will be reviewed to ensure these high level procedures clearly reflect high management p standards for safety, compliance, arror avoidance and prevention. Also, some NOPs may be more effectively implemented as lower tier procedures. The NOPs will be revised as necessary to achieve 7 i l
l I
'
3 ,
.
.-
-
this objective. A further benefit of consolidating higher tier procedures into lower tier documents will 1 i be the ease with which process changes can be made. By consolidating procedures, fewer )
procedures will be affected when a policy revision is required. MOP and NOP revisions will be completed by the end of June 1997.
The lower tier station administrative procedures have been reviewed. Planned changes to these procedures will continue through 1997. Priority will be given to making the NOP changes to ensure
. that management expectations are first captured in the NOPs.
Other Procedure Changes A plan to integrate deficiency reports (DRs) and nonconformance reports (NCRs) from the quality assurance audit process into the PR process will provide the ability to more consistently capture data and trend human error performance. This effort requires submittal of a change to the Boston Edison Quality Assurance Manual for NRC review and approval. The proposed submittalis planned for completion by the end of September 1997. Once implemented, the changes will improve the quality and value of our trend reports.
Modification Process Redesign Changes to NESG Procedure 3.02, " Preparation, Review, Verification, Approval, and Revision of Design Documents for Plant Design Changes" and other station procedures are being developed to incorporate the modification team concept, streamline the process, consolidate forms / paperwork, and simplify the close-out process. The conceptual solution to each of these items has been agreed upon, and the details are currently under development.
To ease the process of assimilating the planned changes, they will be introduced to the organization in two phases. The first phase incorporates a more streamlined modification close-out process. This change will become effective following training prior to commencing our February outage. Rather than attempt the implementation of all changes to the modification process so close to our February refueling outage, we decided to wait until the end of the outage before implementing the second phase (i.e., design portion of the modification process). Implementation of these changes will be completed by the end of September 1997.
Training Additional initial RCA training classes will be conducted as appropriate. We will also complete initial training for the remaining managers and supervisors in methods for human error prevention and identification. We will complete initial human error prevention training for our work force after completion of refueling outage #11. All these training activities are scheduled for completion by the end of September 1997. Also, RCA refresher training will be incorporated into our continued training program.
Date When Full Compliance Will Be Achieved We will complete implementation of the plan including process changes and training by the end of 1997.
>
.
. . . . .
. .
. ..
_ .- - ..-. _.. . . - - . - . -- . .- ~..
!
<
t .
,
. .
,
g 10CFR2.201 Boston Edison Pilgrim Nuclear Power Staten Rocky Hill Road Plymouth, Massachusetts 02300
!
!
February 28, 1997 E. T. Boulette, PhD Senior Vice President - Nuclear BECo Ltr. 2.97-026 U.S. Nuclear Regulatory Commission Attention: Document Control Desk l Washington, DC 20555 l
l l Docket No. 50-293 i License No. DPR-35 )
Subject: Supplemental Reply to Notice of Violation 96-06-02
!
I Our first response to Violation 96-06-02 (BECo Ltr. #2.97 009 dated January 31,1997) described the causes of overall procedural usage and adequacy problems at Pl. im Station, their causes and our corrective actions to address them. As a supplement, we have povided in this letter, for each
- of the individual examples stated in the notice of violation, the reasons for the violation, corrective l action taken and results achieved, actions planned to avoid further violations, and the date when l full compliance will be achieved. This information is contained in the enclosure to this letter. i
'
l l This letter also contains the following commitments:
l l = Recertify the calibration of certain hardness tester calibration blocks and retire the blocks prior 4 to startup from RFO#11.
- Conduct a review of Pilgrim Station conduct of operations using licensed industry peers in l 1997.
!
= Complete installation of the cellular communications system by June 30,1997.
l l Please contact Mr. J.W. Keene at (508) 830-7876 if you have any questions concerning this letter.
,
' *
JWK\dmc\compmgmt\v96-06s ,
,
Enclosure: Reply to Notice of Violation
l
s V o9.7 t
. _ _
, _ . _ _ _ . _ . - _ . . . _ _ ..- . . _ . _ _ - - _ _ - - _ _ - . . -- _ _ .._ .. _ . _ . . .
'
-
.
, ..
. .
U.S. Nucisar Regulitory Commission
Page 2 cc: Mr. Alan B. Wang, Project Manager Project Directorate 1-1 Office Of Nuclear Reactor Regulation Mail Stop: 14B2 1 White Flint North 11555 Rockville Pike Rockville, MD 20852
'U.S. Nuclear Regulatory CommissionY '
Region I 475 Allendale Road King of Prussia, PA 19406 Senior Resident inspector Pilgrim Nuclear Power Station
,
j l
i
!
l
1 l
. . - - . . . . . _ _
___ _ _ . . . . _ _ . _ _ _ . _ . _ _ _ _ _ . . - - _ __ _ _ _ _ _ _ _ _ _ _ . -
.
. ,
.
ENCLOSURE REPLY TO NOTICE OF VIOLATION BACKGROUND l
,
in October 1996, the NRC issued IR 96-06 containing the following violation l
"As a result of an inspection conducted January 22,1996 through February 9,1996, information received on June 17,1996, and additional inspection conducted during this inspection period (July 29 - September 23,1996), the following violation of NRC requirements was identified. In accordance with the NRC Enforcement Policy (60 FR 34381; June 30,
-1995), the violation is described below:
Criterion XVI, " Corrective Action " of 10 CFR Part 50, Appendix B, states, in part, that measures shall be established tv assure that conditions adverse to quality are promptly identified and corrected. In the case of significant conditions adverse to quality, the measures ,
shall assure that the cause of the condition is determined and corrective action taken to '
preclude repetition.
l Contrary to the above, corrective actions for problems identified since at least April 1995, were not effective in identifying and correcting overall procedural adherence and adequacy problems at Pilgrim The following specific procedural adherence and adequacy problems I
were identified:
1. Procedure 2.2.87, Control Rod Drive System, Revision 53, was inadequate by not providing direction to operators moving reactor fuel to verify the correct orientation of the blade guide before control rod insertion. As a result, on April 30,1995, control rod 18-35 was inserted into the reactor core and physically jammed into a mispositioned blade guide causing blade guide and control rod damage.
2. Procedure 9.13, Control Rod Sequence and Movement Control, Revision 12, Attachment 3 listed control rods to be moved. During the power reduction required on October 6,1995, l operators did not follow the reverse order of the pull sheet as evidenced when control rod j 34-23 became mispositioned.
3. Nuclear Operating Procedure NOP 92A1, " Problem Report Program," Step 6.5.3 directs I
'
that Severity Level I problem report evaluations be forwarded to NSRAC for their review.
However, as of January 1996, the NRC identified that NSRAC had not been forwarded approximately one-third (60) of the Level i evaluations. l l 4. On July 31,1996 valve MO-1001-16A ("A" RHR loop heat exchanger bypass) became
! mispositioned when a reactor operator failed to follow the established RHR procedure i when securing torus cooling. The valve was mispositioned for approximately 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br />.
5. In February 1995, a calibration problem on the "B" scale of the Rockwell Hardness Tester j machine was adverse to quality and was not promptly identified or corrected. Quality
!.
1
~
_ - __ ._
___ ..__ _ _ _.___ _ _ _ _ _ . _ _ _ _ _ _ _ _ _ _ _ _ _ . _ _ . .
.
.
.
. .
.
control inspectors relied on verbal advice from an outside vendor rather than contacting the BECo measurement and test equipment personnel, as specified in the M&TE program, or initiating a problem report in accordance with NOP 92A1, " Problem Report Program". Also, the degradation of a related BECo calibration block has not adequately been addressed to date.
This is a Severi!y Level IV violation (Supplement IV)."
In this response, we will address the individual problems of procedural usage and adequacy stated above. The details of 'he requested information follows.
REQUESTED INFORMATION A. Misoriented Blade Guide Procedure 2.2.87, ' Control Rod Drive System', Revision 53, did not provide direction to operators moving reactor fuel to verify the correct orientation of the blade guide before control rod insertion. As a result, on April 30,1995, control rod 18-35 was inserted into the reactor core and physically jammed into a mispositioned blade guide causing blade guide and control rod damage. (Reference problem report (PR) 95.9255)
1. Reasons for Violation The direct cause of this event was poor communications between the control room panel C905 operator and the refueling bridge senior reactor operator (SRO) which led to a control rod being inserted with no concurrent monitoring of the cell from the refueling bridge personnel. Contributing causes included 1) a procedure (2.2.87) which did not specifically require monitoring by personnel on the refuel floor,2) the refueling bridge SRO allowed himself to be distracted by another refueling activity while the latching evolution was continuing,3) failure of the individuals who initially installed the blade guide to recognize it was not properly seated in the cell, and 4) the limited experience of the
! individuals who were performing the complex and infrequently performed tasks even though those individuals had received training prior to the outage.
! 2. Corrective Actions Taken and Results Achieved
! The Chief Operating Engineer immediately halted refueling activities with concurrence of i the Plant Manager. Refueling activities were not resumed until the Operations Section Manager, Chief Operating Engineer, Plant Manager, and Vice President-Operations discussed with all operations personnel the significance of the event, its relationship to previous events, and reinforced management expectations that the pace of activities should ensure careful performance of the task. Also, procedure 2.2.87 was revised to require verification and monitoring when these tasks are being performed. The SRO involved is no longer performing licensed operator activities.
! Inspections of affected components indicated they were all capable of continued service j
with the exception of control rod blade 18-35 which was replaced on May 1,1995. No
'
other blade guides were mispositioned following these corrective actions.
i Also, Licensed Operator Requalification Training (LORT) on this event was completed in
! January 1997 prior to RFO#11.
'
- .. - - --- -- - , - . - . - -- - - - ,
.-- - - -~ - - .- ~.- - - - - - - - - . . - . - . - . - . - . -
. .
,
. . .
l
l
,
3. Corrective Actions Planned j l No further corrective actions are planned.
i )
4. Completion Date for Full Compliance ;
Full compliance was achieved in January 1997 upon completion of LORT on this event.
B. Mispositioned Rod -
l Procedure 9.13, Control Rod Sequence and Movement Control, Revision 12, Attachment 3 '
L listed control rods to be moved. During the power reduction required on October 6,1995,
!
operators did not follow the reverse order of the pull sheet as evidenced when control rod 34-23 became mispositioned. (Reference PR 95.9528) )
l 1. Reasons for Violation l l
The root cause of this event was related to less than adequate human factors and I human error. Control room personnel were reacting to a rapidly degrading condition in ]
the screenhouse that necessitated a reduction in power to allow a seawater pump to be i secured. The pull sheet was misread during a period in which numerous activities
- required monitoring creating an undue sense of haste. Additionally, the operators involved did not effectively use self-checking techniques.
A contributing cause was inadequate communications between the control room and screenhouse. An inoperable page in the screenhouse necessitated the use of communications in altemate locations. This caused a delay in getting critical information to the control room adding to the sense of urgency felt by control room personnel.
Also, procedures were less than optimum. The control rod withdrawal sequence sheet (procedure 9.13, Attachment 3) was cluttered, and the format was not conducive to l inserting control rods in reverse order under the conditions experienced during this event. l Also, procedure 2.4.154 contained unnecessary steps to be followed in a transient condition, resulting in delays in securing the seawater pump (i.e., required closing a valve in the discharge path prior to tripping the pump).
Training omissions occurred. Although both operators involved had approximately nine I years experience, neither had previously been involved with a transient that required a rapid power reduction by inserting rods in reverse order of the pull sheet. Moreover, simulator training did not provide scenarios that would have prepared them to perform this activity under abnormal circumstances.
2. Corrective Actions Taken and Results Achieved Upon discovery of the mispositioning, core thermal limits were verified, and the correct control rod pattem was established. The Operations Department Manager was notified, and an electronic mail message was sent to all operations personnel alerting them of the event.
t Procedures 9.13 and 2.1.14 were revised to provide a method for rapidly lowering power l
l with a selected set of control rods. This control rod set is the rapid power reduction l
(RPR) array and provides the required power reduction with only eight discrete rod
!
i
l 3 l
L -. . _. _ _ . _ _ _
.. - - _ __
.
.
-
.
. j
. .
.
movements. The RPR is graphical in nature thereby improving the ability to read the pull sheet.
Alllicensed operators have received training on the methods to rapidly reduce power using the RPR array. Operators commented favorably on this method.
Communications deficiencies between the control room and the screenhouse were corrected under maintenance request (MR) 19502521, and a new cellular communications system is being implemented.
The Operations Department Manager discussed this event with all licensed operators during Plant Status Update training (session O-RQ 04-01-04). The review included:
. A review / technical discussion of the event
. Changes made to procedures to establish the RPR array l
. Management expectations for the use of the RPR array Management expectations for reactiv;ty control turnover
'
.
. Management expectations for independent verification of control rod manipulation (when required, when not required).
. Management expectations for command and control / oversight by NWEs and NOSs
. Management expectations for promptly informing management of anomalies noted during steady state and transient operations Following the technical evaluation, procedure 2.4.154 was revised to allow tripping the seawater pump prior to closing the valve in the discharge path.
A practice requiring reactivity changes te be made one method at a time (i.e., reduce core flow, then insert control rods) was established through issuance of NOP 96A3, Reactivity Management Program in June of 1999.
No further control rod mispositioning events have since occurred at Pilgrim Station.
3. Corrective Actions Planned Complete installation of the new cellular communications system by June 30,1997.
4. Completion Date for Full Compliance Full compliance was achieved with the issuance of NOP 96A3, " Reactivity Management Program", in June of 1996 and completion of operator training on the NOP in January 1997.
C. NSRAC Distribution Nuclear Operating Procedure NOP 92A1, " Problem Report Program," Step 6.5.3 directs that Se verity Level i proLM report evaluations be forwarded to NSRAC for their review. However, as of January 1996, the NRC identified that NSRAC had not been forwarded approximately one-third (60) of the LevelI evaluations. (Reference PR 96.0025)
- --- . . - - -- .. . -. . . . . . . .. - . _ - .
!
. .
.
.
! 1. Reasons for Violation The cause of this problem was a failure to follow the requirements of nuclear operating procedure (NOP) 92A1, Section 6.5.3[5] which states, "The PR Coordinator should forward all Significance Level 1 completed evaluations to the NSRAC Coordinator for
,
information purposes only " This failure to folicw procedure is attributable to inattention I
to detail on the part of the cognizant Problem Report Coordinator. Contributing to this failure was the lack of an administrative process to ensure the required evaluations were sent for NSRAC review.
2. Corrective Actions Taken and Results Achieved The Problem Report Coordinator responsible for forwarding the required evaluations is no longer employed by Boston Edison Company. Also, the sixty items that should have been sent to NSRAC were identified and forwarded to NSRAC for their review.
Additionally, a reporting mechanism has been established in the corrective action j program dat& base to ensure all required evaluations are forwarded for distribution to j NSRAC.
The NSRAC routinely receives all required information. No further problems have been 1 identified, l
'
3. Corrective Actions Planned No further corrective actions are planned.
4. Completion Date for Full Compliance Full compliance was achieved upon completion of the improvements to the corrective action database in February 1996.
D. Mispositioned Valve On July 31,1996, valve MO-1001-16A ("A" RHR loop heat exchanger bypass) became mispositioned when a reactor operator failed to follow the established RHR procedure when securing torus cooling. The valve was mispositioned for approximately 11 hours1.273148e-4 days <br />0.00306 hours <br />1.818783e-5 weeks <br />4.1855e-6 months <br />. (Reference PR 96.9383)
1. Reasons for Violation The first causal factor was failure to follow procedure. The operator did not intentionally fail to perform the procedural step to open the subject valve nor did he forget how to close the valve; but rather, the operator forgot to perform the step. Although operators are committed to strict adherence to procedures, they are not necessarily required to have the procedure in hand while executing an evolution, particularly if the evolution is performed frequently. In this case, the operator had referred to the procedure prior to commencing the evolution. Contributing to the operator's error were distractions that diverted his attention during this evolution (i.e., a communications page that required a response).
l Another reason for the error was failure of the control room supervisor to maintain proper
'
oversight of ongoing activities. Contributing to the supervisor's error was overconfidence
'
in the operator's ability not to make an error.
l
i l - - - _ _ . .- _
- --- . . . . - . _ . - ~ . . - - - _ _ _ - - . . . - - - . - . - - - . - - - - - .
'
'
. . .
-
.
.
The remaining causes are associated with the failure to discover the mispositioned valve i during two shift tumovers. The cause was inattention to detail during shift turnover walkdowns of control boards.
2. Corrective Actions Taken and Results Achieved All operations shifts were briefed on the incident and the procedure 1.3.34 requirements.
Also, management expectations were reviewed for 1) procedural adherence, particularly the 2.2. series operating procedures which do not have to be in hand,2) supervisory oversight of critical evolutions, and 3) control board walkdowns at shift turnover. 1 The operator and supervisor involved in the mispositioning event were counseled by the Operations Department Manager, as were the operators and supervisors who failed to 1 discover the mispositioned valve during tumover. All the errors were logged in the
,
operations human performance matrix for trending. The supervisor involved was )
i'
j removed from the watchbill to receive additional training because of a declining performance trend for this individual. The supervisor successfully completed additional training and was placed back on the watchbill after successfully completing interviews 1 with the Operations Training Department Manager, Operations Department Manager, and the Plant Manager.
Refresher training on self-checking techniques is periodically provided to licensed operators'who are also reminded that managers expect these techniques to be applied when following procedures and during preshift tumover control board walkdowns.
No further valve mispositioning events have sinc : curred.
3. Corrective Actions Planned The Operations Department Manager will have an independent assessment performed l by licensed industry peers of Pilgrim Station conduct of operations in 1997. l
.
4. Completion Data for Full Compliance Full compliance will be achieved by the end of 1997.
E. Hardness Tester Calibration In February 1995, a calibration problem on th6 'B' scale of the Rockwell Hardness Tester machine was adverse to quality and was not promptly identified or corrected. Quality control inspectors relied on verbal advice from an outside vendor rather than contacting the BECo measurement and test equipment personnel, as specified in the M&TE program, or initiating a problem report in accordance with NOP 92A1, ' Problem Report Program'. Also, the degradation of a related BECo calibration block has not adequately been addressed to date.
(Reference PR 96.0369 and Quality Assurance Surveillance Report 96-066)
Rockwell Hardness Tester HDT-002 is maintained by BECo M&TE personnel and operated by receipt inspection personnel. M&TE personnal are responsible for ensuring the tester is calibrated in accordance with the M&TE Program as controlled by procedure 1.3.36. The tester is calibrated by the vendor (Wilson Instruments) annually under this program. The tester is issued to receipt inspection personnel on a long-term basis, and they are responsible for performing calibration checks prior to use.
l
.
l
[ . . - , -
-- -
'
a .
~
.
.
The timeline for this event is as follows:
January 1994 Calibration performed by vendor March 1994 Repair and calibration by vendor November 1994 Calibration by vendor; replaced damaged 1/16" ball cap February 1995 Receipt inspectors question Channel B calibration. From this point on, Channel B is used twice by receipt inspectors March 1995 Routine annual calibration check performed by BECo M&TE; results satisfactory May 1995 The ball penetrator is impacted by a unistrut channel; vendor assistance is requested by receipt inspection personnel June 14,1995 Calibration check performed by BECo M&TE; Channel B is found out-of-tolerance; a " limited use" status is applied to the tester (the same technician and calibration block were associated with the March 1995 calibration check)
June 21,1995 Calibration of the tester and replacement of ball penetrator performed by vendor; tester found to be in calibration; calibration block (s/n 87H34072)
is found out-of-tolerance by the vendor; BECo M6TE removed the block
from service Procedural adherence problems associated with this event occurred in February 1995 when receipt inspection personnel failed to notify BECo M&TE personnel as required by procedure 1.3.36 when they questioned the calibration of the Channel B scale. The BECo M&TE personnel are responsible for contacting the vendor if their expertise is required. Also, receipt inspection personnel did not issue a prcblem report as required by NOP92A1. Another procedural adhereace problem occurred following the June 14 calibration by BECo M&TE when they failed to promptly issue an out of calibration report as required by procedure 1.3.36. .
1. Reasons for Violation The reasons for the violation were determined to be knowledge based in that receipt inspection personnel were not fully aware that hardness tester calibration was the responsibility of the M&TE group and, hence, was subject to the procedural requirements of 4.3.36. Also, they were unaware that a problem report should be generated when an out-of-tolerance condition is discovered. Moreover, the M&TE supervisor did not realize procedure 1.3.36 required an out of calibration report be issued promptly upon discovery of the potential out-of-tolerance condition of the tester.
2. Corrective Actions Taken and Results Achieved Personnel in the receipt inspection area have been replaced by others as part of a procurement services contract. They have been trained on procedure 1.3.36 and the need to contact M&TE personnel with any test equipment problems instead of calling
, .
.
..
.__ . ._ -
i . . , c.. -
4 - . ,
.-
l l vendors directly. They have also been trained on the problem reporting process l described in NOP92A1.
The M&TE supervisor and his manager reviewed the 1.3.36 requirement for prompt documentation on an out of calibration report of M&TE found out of calibration tolerance.
The expectation of procedural adherence was also reinforced. M&TE personnel have i also been retrained on the importance of reporting problems in a timely manner and in l accordance with NOP92A1.
Aside from the procedural issues, concem about the potential for parts to have been l inappropriately accepted needed to be evaluated. This concem was alleviated through a record search that produced the information used for the above time line. It shows that the tester was in calibration at all times even prior to the June 21 calibration (i.e., the calibration data taken before and after replacement of the ball penetrator was the same, and no adjustments were necessary). Even though the tester had been in eclibration, Channel B of the tester was only used twice to test parts (reference MRIRs 95-1285 and
)
95-1190) between February 1995 when calibration first became suspect and June 1995
,
when the vendor confirmed the tester was still in calibration. The tested components -
were either retested satisfactorily using Channel A or installed in non-safety applications. l
,
! Moreover, because the tester was always in calibration, it shows that the calibration block j (s/n 87H34072) was in tolerance until June 1995 when the discrepancy was noted.
i The hardness tester procedure (Quality Control Instruction 7.15) was revised to ensure l notification of M&TE personnelif calibration checks are found to be cut of tolerance.
Also, it was confirmed that training on the hardness tester includes proper installation of the ball penetrator since proper positioning is not described in the vendor manual nor is the need for delicacy. )
, No other calibration blocks have been found out-of-tolerance, nor have any further .
failures been identified to promptly issue out of calibration reports when required.
3. Corrective Actions Planned Based upon historical performance of numerous hardness tester calibration blocks, one calibration block going out-of-tolerance can be viewed as an isolated occurrence. i However, a policy will be implemented to recertify or retire all hardness tester calibration i blocks on their fifth year anniversary and every five years thereafter. Additionally, the - j other five calibration blocks (s/n 89120978, 89P18946, 89120668, 89120491, and ]
89C94684) bought under the same purchase order as the subject calibration block (NST :
009016, MRIR 90-0503) will be recertified to ensure no common cause failure mode was ,
manifested. They will then be retired. These corrective actions will be completed prior to j restart from RFO#11.
l 4. Completion Date for Full Compliance ;
Full compliance will be achieved when the calibration blocks are checked then retired
) prior to restart from RFO#11.
j e
)
,
i
! 8
l l l
t _ . . __ _ . _ _. .- . _ _ . . __ _ __a