ML13329A207

Responds to NRC Re Violations Noted in Insp Repts 50-206/92-26,50-361/92-26 & 50-362/92-26 on 920827-1021. Formal Training of M&TE Users on Proper Usage of M&TE Conducted

ML13329A207
Person / Time
Site:	San Onofre
Issue date:	12/21/1992
From:	Morgan H SOUTHERN CALIFORNIA EDISON CO.
To:	NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM)
References
NUDOCS 9212280275
Download: ML13329A207 (22)

Text

Southern California Edison Company P. 0.

BOX 128 SAN CLEMENTE, CALIFORNIA 92674-0128 H. E. MORGAN TELEPHONE VICE PRESIDENT AND SITE MANAGER 714-368-9470 SAN ONOFRE December 21, 1992 U. S. Nuclear Regulatory Commission Attention:

Document Control Desk Washington, D.C. 20555 Gentlemen:

Subject:

Docket Nos. 50-206, 50-361, and 50-362 Reply to a Notice of Violation San Onofre Nuclear Generating Station, Units 1, 2, and 3

Reference:

Letter from Mr. S. A. Richards (USNRC) to Harold B. Ray (SCE), dated November 20, 1992 The referenced letter forwarded a Notice of Violation resulting from the NRC inspection conducted from August 27, 1992 through October 21, 1992, at the San Onofre Nuclear Generating Station, Units 1, 2, and 3. This inspection was documented in NRC Inspection Report Nos. 50-206/92-26, 50-361/92-26, and 50-362/92-26.

In accordance with 10 CFR 2.201, the enclosure to this letter provides the Southern California Edison (SCE) reply to the Notice of Violation.

If you have any questions regarding SCE's response to the Notice of Violation or require additional information, please call me.

Sincerely, Enclosure cc:

J. B. Martin, Regional Administrator, NRC Region V J. 0. Bradfute, NRC Project Manager, San Onofre Unit 1 M. B. Fields, NRC Project Manager, San Onofre Units 2 and 3 C. W. Caldwell, NRC Senior Resident Inspector, San Onofre Units 1, 2, and 3 921228O275 92221 PDR 4DOCK 05000206 0

~~PDR J~~'

ENCLOSURE REPLY TO A NOTICE OF VIOLATION The Enclosure to Mr. Richards' letter dated November 20, 1992 states in part:

"A. Technical Specification 6.8.1 for San Onofre Nuclear Generating Station, Units 1, 2, and 3, requires that written procedures be established, implemented, and maintained covering activities referenced in Appendix A of Regulatory Guide 1.33, Revision 2. Appendix A of Regulatory Guide 1.33 specifies that safety related activities should be covered by written procedures, including procedures for control of measuring and test equipment (M&TE).

"Section 6.2.4 of procedure SO123-II-1.2, TCN 1-4,

'Preparation And Responsibility Of The M&TE traveler,'

requires that, 'Without exception all M&TE's used in conjunction with a Maintenance order or any other approved Station procedure shall be recorded in the Traveler.

"Contrary to the above, as of October 1, 1992, the inspector identified 24 instances in which M&TE usage was not properly documented on travelers in accordance with procedure SO123-II-1.2.

These instances included two cases in which the M&TE used were not properly documented and also needed to be evaluated due to calibration failures (12-8526 used in M092041235000 and Ml-3469 used in M091111631001).

Additional examples of improperly documented M&TE usage were subsequently identified by the licensee.

"This is a Severity Level IV violation (Supplement I) applicable to Units 1, 2, and 3."

RESPONSE TO ITEM A

1.

BACKGROUND Measuring and Test Equipment (M&TE) Program Weaknesses In January 1992, Quality Assurance (QA) initiated a routine Audit of SCE's Measuring and Test Equipment (M&TE) program to verify compliance with TQAM chapter 4-E.

Although the findings of the audit identified that the program was being adequately implemented in accordance with procedural requirements, deficiencies existed that pointed to potential weaknesses in the program which required management attention.

Based on the audit results, Maintenance management initiated a self-assessment of the program to address the potential programmatic weaknesses identified. A Quality Action Team (QAT) was subsequently formed in June, 1992 to perform an in-depth evaluation of the M&TE program.

The QAT was tasked with reviewing the M&TE process, identifying weaknesses, and recommending ways of improving and simplifying the process. The QAT started their review by assessing the specific weaknesses identified in the QA Audit. The weaknesses identified by the audit included the use of M&TE with accuracy inappropriate to its application and routing some calibration failure notices to the wrong organization. In addition, the QAT also planned to assess any other weaknesses identified during the QAT's in-depth review of the M&TE traveler process.

SCE was in the process of evaluating the M&TE program when the Resident Inspector identified the M&TE violations cited in his inspection. SCE believes that as the QAT progressed, it would have identified the deficiencies associated with the travelers.

SONGS Test Equipment Management (STEM) and San Onofre Maintenance Management System (SOMMS)

Databases The usage of M&TE is recorded on Maintenance Orders (MO) and that information is transcribed from the MO to the SOMMS database, the computer tracking system for MOs. M&TE used in conjunction with a MO is also recorded on a M&TE traveler and that information is transcribed from the traveler to the STEM database, the computer tracking system for M&TE usage and calibration.

2.

REASON FOR THE VIOLATION Our assessment concluded that the reasons for the violation include:

1) the use of a cumbersome process for using M&TE which led to a failure of M&TE users to follow procedures, and 2) inadequate training of users on the M&TE process.

Failure to Follow Procedures Personnel failed to correctly follow the M&TE procedure regarding documentation of M&TE usage on M&TE travelers.

This was caused in part by programmatic weaknesses which made the M&TE traveler process cumbersome and therefore prone to personnel error:

The travelers are not always kept with the work packages and personnel therefore overlooked recording M&TE usage.

The M&TE Traveler program requires four different manual entries of M&TE data performed at different stages of the process. This leads to transcription errors.

The M&TE traveler is handled by different organizations at different locations which leads to misplacement of the M&TE traveler and additional transcription errors.

Inadequate Training Maintenance supervision did not adequately communicate to personnel the significance of the M&TE traveler.

Personnel using M&TE did not fully understand the reasons behind the program requirement to record on two separate documents (MO and Traveler) the same information on M&TE usage.

In addition, no formal training was conducted on the M&TE Process. This contributed to the failure of personnel to consistently and accurately document use of all M&TE on travelers as required by procedures.

3.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVED Corrective actions taken to date include: 1) formal training of M&TE users on its proper usage, and 2) an audit of the STEM (M&TE usage) database to identify and resolve all M&TE usage discrepancies.

M&TE Training SCE completed formal training of M&TE users on November 25, 1992, regarding the proper use and significance of the M&TE traveler. The training will ensure that personnel properly document the use of M&TE on travelers as required by the current procedure. Beginning November 26, 1992, only those personnel who have received this M&TE training are being allowed to check out M&TE. This requirement was documented by issuance of a memorandum on December 17, 1992.

Verification of STEM Database In order to identify other uses of M&TE which were not properly documented, SCE developed and implemented a computer program to compare the entire STEM database against the SOMMS database for the 26,027 uses of M&TE on plant equipment over the past 18 months. The computer program identified discrepancies between M&TE usage recorded in STEM and SOMMS. Any M&TE calibration failures associated with those database discrepancies were then evaluated. These evaluations were performed under the Calibration Failure Notice process.

The majority of the calibration failures associated with these M&TE usages did not adversely impact plant equipment and no further action was required. A small percentage resulted in Nonconformance Reports (NCRs) being initiated to assess the operability of the plant equipment affected by the remaining M&TE usages. There were no operability impacts identified in these NCR assessments.

The audit also confirmed that the cause of the database discrepancies was a failure to follow procedures due to a cumbersome M&TE traveler process.

4.

CORRECTIVE STEPS THAT WILL BE TAKEN TO PREVENT RECURRENCE Our corrective actions to prevent recurrence will include: 1) monthly monitoring of the STEM database, and 2) implementation of appropriate recommendations of a Quality Action Team to address program deficiencies.

Monthly Monitoring of the STEM Database SCE will run the new STEM database comparison computer program monthly and evaluate and address any discrepancies identified. The monthly use of this program provides a means for management to continue to monitor the accuracy of the M&TE traveler database.

It will be continued indefinitely or until superseded by other QAT recommended improvements.

M&TE Program Enhancements In an effort to address weaknesses identified in the internal audit of the M&TE program, SCE formed a Quality Action Team (QAT) to perform a broad and in-depth look at the M&TE program.

The QAT continues to evaluate the M&TE process and will provide recommendations to SCE management for enhancing the implementation of the M&TE program in general. For example, the QAT is reviewing the feasibility of implementing a major modification to the current traveler process. The current four step manual data entry and dual (SOMMS and STEMS) tracking system would be replaced with a single simplified on-line computer tracking system.

Such a tracking system would eliminate the documentation redundancies and reduce the opportunities for data transfer error.

The QAT is expected to complete their assessment and provide recommendations for improving the M&TE program by April 1993.

5.

DATE WHEN FULL COMPLIANCE WAS ACHIEVED Full compliance was achieved on December 1, 1992, when all M&TE uses associated with unreviewed calibration failures due to deficiencies in the STEM database were evaluated and Nonconformance reports were initiated, as required, to evaluate the potential impact of M&TE calibration failures on plant equipment.

REPLY TO A NOTICE OF VIOLATION The enclosure to Mr. Richard's letter dated November 20, 1992, states in part:

"B. Technical Specification 6.8.1 for San Onofre Nuclear Generating Station, Units 1, 2, and 3, requires that written procedures be established, implemented, and maintained covering activities referenced in Appendix A of Regulatory Guide 1.33, Revision 2. Appendix A of Regulatory Guide 1.33 specifies that safety related activities should be covered by written procedures, including procedures for control of M&TE.

"Sections 6.2.4 and 6.2.5 of procedure S0123-II-1.5, TCN 1-4, "Evaluation of Calibrated Items After M&TE Failure," requires that the cognizant department supervisor detail the specific reasons that retests or recalibrations are not required if M&TE fails calibration. The procedure states, "This detail shall include identifying a component as non-safety related if this is the reason for not performing a retest or recalibration."

Otherwise, the supervisor shall initiate a nonconformance report or initiate the proper work documents to perform remeasurements, retests, or recalibrations with known accurate M&TE.

"Contrary to the above, as of October 1, 1992, M&TE used in conjunction with the MOs specified were not properly evaluated in accordance with procedure S0123-II-1.5.

Specifically, evaluations for M&TE Ml-1596 (used in M09004232400), 11-6427 (used in M089082495000), M2-3992 (used in M090100691000), M1-1973 (used in M09105011000), Ml 2634 (used in M091041634000), and M2-4857 (used in M091121502), did not provide adequate justification of why a retest or recalibration was not required nor were nonconformance reports issued to document that the equipment was operating properly.

"This is a Severity Level IV violation (Supplement I) applicable to Units 1, 2, and 3."

RESPONSE TO ITEM B W

1.

BACKGROUND Measuring and Test Equipment (M&TE) is utilized to measure, test, and calibrate permanent plant equipment.

M&TE is regularly calibrated to ensure equipment accuracy.

A Calibration Failure Notice (CFN) is generated by SCE when M&TE fails its calibration, is lost, or is damaged. The CFN is forwarded by the M&TE Supervisor to the responsible department supervisor to determine whether the M&TE calibration failure affects the operability of quality affecting plant equipment on which the failed M&TE was used.

The responsible department supervisor is required to either retest or recalibrate the plant equipment or provide justification for why a retest or recalibration is not required in accordance with procedure S0123-II-1.5, "Evaluation of Calibrated Items After M&TE Failure."

During routine resident NRC inspection activities, the Resident Inspector reviewed approximately 100 M&TE CFN evaluations and found six evaluations that did not include sufficiently documented justification of evaluation results as required by procedure S0123-II-1.5.

SCE reviewed these six evaluations and determined that five of the six evaluations involved calibrations performed on non-safety related equipment, although this would not have been evident from a brief review of the packages. Under SCE's M&TE program, procedure SO123-II-1.5 does not require the retest or recalibration of non-safety-related plant equipment as a result of CFNs issued against the M&TE used on that equipment. However, that reason was not specifically provided as justification in the CFN evaluations as required by the procedure.

The remaining evaluation involved a calibration performed on safety-related equipment. The safety-related equipment was not affected by the calibration failure because a valid recalibration test was performed on the affected equipment following the initial calibration with the failed M&TE. However, the evaluator did not document the valid recalibration as justification for not performing a retest or recalibration in accordance with procedure S0123-II-1.5.

Our review of the six evaluations determined that the original conclusions were supported in each evaluation but that the documentation was not complete. Additionally, as part of our routine review of CFN evaluations, we have occasionally discovered and corrected similar deficiencies in documentation which did not result in requiring a change to the original conclusions regarding the operability of the affected plant equipment.

Our assessment of these CFN evaluation deficiencies is that their safety significance is minimal since only 1 evaluation out of the 100 reviewed involved safety-related equipment. We consider these CFN evaluation documentation deficiencies to be administrative in nature since they do not impact plant safety.

2.

REASONS FOR THE VIOLATION Our assessment concluded that the reason for the violation was that CFN evaluators lacked knowledge of procedural requirements. A contributing cause to the violation was the lack of formal training for CFN evaluators.

CFN Evaluators Lacked Knowledge of Procedural Requirements The personnel who performed the CFN evaluations were not knowledgeable of the procedural requirements for properly documenting CFN evaluations. Therefore, the evaluators incorrectly concluded that their documented justification was appropriate.

Contributing Cause to the Violation -

Lack of Formal Training Personnel required to perform CFN evaluations had not been provided formal training on the proper methods for documenting M&TE calibration failure evaluations in accordance with procedure S0123-II-1.5.

3.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVED Our corrective actions taken to date include: 1) a memorandum was issued to Maintenance supervision and CFN evaluators to emphasize the purpose and importance of CFN evaluations, 2) formal training was provided for CFN evaluators, and 3) SCE revised the six CFN evaluations.

Memorandum Issued to M&TE Supervisors and CFN Evaluators On October 14, 1992, Maintenance supervision issued a memorandum to M&TE supervisors and all CFN evaluators to emphasize the purpose and importance of CFN evaluations and adherence to procedures.

Training for CFN Evaluators Formal training was conducted for CFN evaluators on the proper methods for performing CFN evaluations. This training was completed on November 25, 1992.

Beginning November 26, 1992, only those personnel who had received the CFN evaluation training were allowed to perform CFN evaluations. This administrative control was documented by issuance of a memorandum on December 17, 1992.

Revision of the Six Identified CFN Evaluations SCE revised the six CFN evaluations to provide proper documentation of the evaluations in accordance with procedure S0123-II.1.5.

4.

CORRECTIVE STEPS THAT WILL BE TAKEN TO PREVENT RECURRENCE Our corrective actions to prevent recurrence include:

1) provide periodic retraining for CFN evaluators, and 2) development of a check list to enhance the CFN evaluation process.

Periodic Retraining for CFN Evaluators SCE will implement periodic retraining for all CFN evaluators on the proper methods for performing CFN evaluations. The appropriate programs will be revised to include this retraining by July 15, 1993.

Development of Check List to Enhance CFN Evaluation Process Maintenance is developing a check list to assist completion and documentation of CFN evaluations and to enhance the process. The check list will clearly indicate the criteria to be evaluated, the documentation required for the evaluation, and any remaining actions required to comply with the CFN evaluation procedure. Use of this check list will simplify the CFN process and will be incorporated into procedure S0123-II-1.5 by February 15, 1993.

5.

DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Full compliance was achieved by December 18, 1992, when the six identified CFN evaluations had been reviewed and revised in accordance with procedural requirements. The original conclusions documented in the CFN evaluations did not change.

ENCLOSURE REPLY TO A NOTICE OF VIOLATION The Enclosure to Mr. Richards' letter dated November 20, 1992 states in part:

"C.

10 CFR Part 50, Appendix B, Criterion XVI, requires, in part, that measures shall be established to assure that conditions adverse to quality, such as deficiencies, deviations, and nonconformances, are promptly identified and corrected. The measures shall assure that the cause of the condition is determined and corrective action taken to preclude repetition.

"On June 21, 1990, a Quality Assurance (QA) audit of the M&TE program identified instances in which M&TE uses were not being properly documented on M&TE travelers in accordance with station procedure SO123 11-1.2, 'Preparation And Responsibility Of The M&TE Traveler.'

"Contrary to the above, as of October 1, 1992, the licensee had not taken adequate actions to correct the deficiencies found in the 1990 QA audit, as evidenced by the fact that the Resident Inspector identified 24 instances in which M&TE usage was not documented in travelers as required by station procedure S0123-II 1.2.

"This is a Severity Level IV violation (Supplement I) applicable to Units 1, 2, and 3."

RESPONSE TO ITEM C

1.

BACKGROUND 1990 QA Audit of Measuring and Test Equipment (M&TE) Program From April 1990 to July 1990, Quality Assurance (QA) performed a routine Audit of SCE's M&TE program to verify compliance with Topical Quality Assurance Manual Chapter 4-E "Calibration Program."

The findings of the audit identified a program deficiency in the recording of M&TE information in the M&TE tracking system (STEM).

This was based on discrepancies identified between Maintenance Orders (MO) and the STEM database. This program deficiency was documented on a Problem Review Report (PRR) SO-137-90.

The response to PRR SO-137-90 stated that the cause of the database discrepancies was due to a misconception on the part of the personnel as to which M&TE needed to be recorded on the travelers. The personnel were only recording M&TE used for quantitative measurements and not M&TE used for qualitative measurements. To address this condition, Maintenance revised the M&TE procedure to require both quantitative and qualitative M&TE to be recorded on the M&TE travelers.

In November 1990, QA closed PRR SO-137-90 based on the revised Maintenance procedure and the completion of a one time required reading assignment by Maintenance personnel.

QA concluded that the response was acceptable and scheduled a followup in the next audit scheduled for January 1992.

Followup Audit of M&TE Program In January 1992 through July 1992, QA conducted the re audit of the M&TE program. The 1992 M&TE Audit performed by QA reviewed the STEM database deficiency identified in the 1990 Audit. The Auditor reviewed a sample of selected-MOs that included 138 M&TE instruments and compared the listing to the M&TE tracking system, STEM. No deficiencies were identified. The audit sample was adequately sized but was biased towards work with high safety significance. A sample representative of the total cross section of M&TE usage would have been more likely to identify the discrepancies noted in this NOV.

The sampling used in the audit was weighted to capture more safety significant work activities and focus on those items which would be expected to have a higher quality impact if deficient. This selection caused QA to look primarily at Mos associated with Technical Specification surveillances and safety related equipment rather than a random sampling of MOs.

2.

REASON FOR VIOLATION Our assessment concluded that the reasons for the violation include: 1) SCE did not perform an adequate evaluation of the cause of the M&TE traveler discrepancies when initially discovered, and 2) a missed opportunity to identify earlier the continuing M&TE problem due to a biased sample used by QA when performing a followup audit.

Insufficient Evaluation of the Problem Once the problem was identified in 1990 by PRR SO-137 90, a sufficiently rigorous root cause evaluation was not performed. When Maintenance management assessed the problem identified in the PRR, they concluded that personnel did not believe all M&TE usage needed to be recorded on M&TE travelers. Maintenance erroneously assumed that fixing this misperception would correct the problem. A rigorous root cause evaluation would have identified other problems with this program.

In the PRR response, Maintenance stated that the omission of the M&TE information from STEM was caused by the misconception that M&TE that was used only for informational purposes did not need to be included on a traveler. M&TE users thought only M&TE used for quantitative measurements needed to be recorded on the travelers. As part of the PRR response, Maintenance corrected this problem by revising the procedure to require the listing of all M&TE on travelers and indicating whether or not the M&TE was used for quantitative measurements.

The cause of the discrepancies noted in 1990 were not only due to the issue noted by Maintenance in the PRR response, but were also due to the failure of personnel to follow procedures. As a result, Maintenance did not place sufficient emphasis on compliance with the procedural requirements because they erroneously believed that the problem was based on a misconception of the procedural requirements rather than a failure to follow procedures.

In addition, the PRR response did not contain steps to ensure that the procedural enhancements and the required reading assignment were adequate and effective. There was no action planned or taken to more closely monitor the recording of M&TE usage on travelers. Because Maintenance did not believe the problem to be a failure to follow procedures, they did not plan to take action to audit the STEM database to ensure data was documented on travelers and accurately entered into that database.

Missed Opportunity to Identify the M&TE Problem The 1992 QA followup audit of the M&TE program evaluated the continuing effectiveness of the PRR corrective action using a biased sample. The use of a biased sample, intended to focus on the most safety significant work, prevented a timely SCE identification of the ineffective corrective action from the initial PRR. The audit sample was adequately sized but was biased towards work with high safety significance. A sample representative of the total cross section of M&TE usage would have been more likely to identify the discrepancies noted in this NOV.

3.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVED Corrective actions taken to date include: 1) an audit of the STEM database to identify and resolve all M&TE discrepancies and 2) formal training in the conduct of root cause evaluations for Maintenance personnel performing such evaluations.

Special Audit of STEM Database The usage of M&TE is recorded on Maintenance Orders (MOs) and that information is entered into the SOMMS database, which is the computer.tracking system for MOs.

M&TE used in conjunction with a MO is also recorded on a M&TE traveler and that information is also entered into the STEM database, which is the computer tracking system for M&TE usage and calibration. Any discrepancies identified with the STEM database for permanent plant equipment were addressed under the Calibration Failure Notice process.

SCE assessed the magnitude of the problem and its cause by developing and implementing a computer program that compared the entire STEM database against the SOMMS database for the 26,027 uses of M&TE on plant equipment over the past 18 months. The computer program identified discrepancies between M&TE usage recorded in STEM and SOMMS and SCE evaluated the M&TE calibration failures associated with those database discrepancies. The discrepancies involving permanent plant equipment were evaluated under the Calibration Failure Notice process.

The majority of the calibration failures associated with these M&TE usages did not adversely impact plant equipment and no further action was required. A small percentage resulted in Nonconformance Reports (NCRs) being initiated to assess the operability of the plant equipment affected by the remaining M&TE usages. There were no operability impacts identified in these NCR assessments.

In the process of verifying the accuracy of M&TE traveler information in the STEM database, it was confirmed that the cause of the database discrepancies was a failure to follow procedures due to a cumbersome M&TE traveler process.

Root Cause Program Improvements In 1990, SCE recognized the need to strengthen the Root Cause program for SONGS. The Nuclear Oversight Division (NOD) created the Safety Engineering Group to formally develop and administer a Root Cause program. Root Cause procedures were developed and implemented along with a Root Cause training program. Root cause evaluation training was provided to personnel in organizations outside of NOD that were performing root cause evaluations. Also, the Safety Engineering group assigns a Root Cause engineer to each of these organization to assist in root cause evaluations that are performed.

From June, 1992 through October, 1992, Maintenance personnel in positions which might be required to perform Maintenance Division Evaluation Reports (MDER) attended formal root cause evaluation training. In August 1992, a new Maintenance procedure for MDERs, SO123-I-1.42, was issued to require formal root cause evaluation training for personnel performing root cause evaluations. This training and the procedure will ensure that Maintenance personnel will perform effective root cause evaluations.

4.

CORRECTIVE STEPS THAT WILL BE TAKEN TO PREVENT RECURRENCE Refinement of QA Sampling Techniques In general, the sampling techniques used by QA, which include the use of biased samples, are an efficient and effective method for identifying program problems during audits. However, the use of biased sample techniques may allow problems that would more likely be detected by random samples to go undetected. SCE will evaluate our use of various sampling techniques and will take action to provide more balanced samples. This will be accomplished by providing training to all QA Auditors based on the results of our evaluation. This will reduce the liklehood that the sampling technique will mask the existence of problems.

This action will be completed by March 30, 1993.

5.

DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Compliance was achieved on December 1, 1992, when the SOMMS and STEM database comparison was completed and the M&TE uses associated with unreviewed calibration failures due to deficiencies in the STEM database were evaluated and corrected as required.

REPLY TO A NOTICE OF VIOLATION The Enclosure to Mr. Richards' letter dated November 20, 1992, states in part:

"D. 10 CFR Part 50, Appendix B, Criterion XVI, requires, in part, that measures shall be established to assure that conditions adverse to quality, such as deficiencies, deviations, and nonconformances, are promptly identified and corrected. The measures shall assure that the cause of the.

condition is determined and corrective action taken to preclude repetition.

"On September 27, 1991, an audit of the employee personnel records qualification program by the site Quality Assurance (QA) organization identified instances in which documentation of station personnel qualifications, in accordance with station procedure S0123-VI-33, "Personnel Records Qualification Program," had not been performed for several individuals.

"Contrary to the above, as of October 15, 1992, the licensee had not taken adequate actions to correct the deficiencies found in the 1991 QA audit, as evidenced by the fact that the Resident Inspector identified four instances in which documentation of personnel qualifications had not been performed in accordance with station procedure S0123-VI-33.

"This is a severity Level IV violation (Supplement I) applicable to Units 1, 2 and 3."

RESPONSE TO ITEM D

1.

BACKGROUND Site Quality Assurance (QA) conducted an audit of the Personnel Records Qualification (PRQ) program during the period of July 15 through October 17, 1991.

As a result of this audit, QA determined that some SCE job positions identified in ANSI N18.1-1971 as requiring a Qualification Resume (QR) were not included in SCE's PRQ program.

Additionally, some individuals filling job positions that were included in the PRQ program did not have a QR on file.

Therefore, on September 27, 1991, QA issued a Corrective Action Request (CAR) P-1386 to the Budgets and Administration Division (B&AD), the controlling organization, for failure to implement the PRQ program in accordance with the existing procedure.

In response to this CAR, B&AD performed an evaluation of the PRQ program and developed a plan of corrective actions for the deficiencies noted in the CAR. Corrective actions taken and planned by B&AD included the following:

Phase I:

(Action Taken to Resolve Problem)

Updated Attachment 1 to procedure S0123-VI-33 to reflect current, appropriate job titles.

Verified that a completed QR for each individual designated in procedure S0123-VI-33 was on file in the Corporate Documentation Management (CDM) files.

Phase II: (Planned Corrective Action to Prevent Recurrence)

Evaluate the work process detailed in procedure S0123-VI-33 and evaluate alternative work processes to improve the program.

Revise procedure S0123-VI-33 as required.

Request changes to licensing documents (i.e.,

Technical Specifications) as required.

Implement changes to process as required.

B&AD revised procedure S0123-VI-33 to reflect current, appropriate job titles, generated new QRs for all personnel identified in Attachment 1 of procedure S0123-VI-33, and contacted Site QA to request verification of this corrective action on May 28, 1992.

QA verified that the procedure was revised and QRs in CDM were complete and up-to-date. Subsequent to this verification, QA incorrectly concluded that the PRQ program was effective without implementation of the phase II corrective actions. Accordingly, QA closed the CAR and intended to verify the effectiveness of the PRQ program during the next routinely scheduled audit.

In closing the CAR, QA documented that B&AD had planned additional actions to improve the work process for the PRQ program, and QA considered these additional actions to be program enhancements only and therefore, not necessary for CAR closure. In addition, QA believed the closure of the CAR was appropriate because of the low safety significance of the deficiencies in that it was verified that personnel with out-of-date or missing QRs were found to meet ANSI N18.1-1971 qualification requirements.

2.

REASONS FOR THE VIOLATION Our assessment concluded that the reasons for the violation include: 1) failure to follow the procedure when QA closed the CAR without completion of all of the corrective actions, and 2) failure of the B&AD group to complete all planned corrective actions in a timely manner.

Failure To Follow Procedure QA failed to follow Quality Assurance Procedure (QAP)

N16.03, "Instruction for Issuance and Control of Corrective Action Request (CAR)," when it closed CAR P-1386. QAP N16.03 requires QA to verify implementation of corrective actions and to verify documentation of all completed actions prior to final closure of the CAR. B&AD had completed actions to correct the deficiencies, but had not completed planned corrective actions to prevent recurrence of similar deficiencies when QA closed the CAR.

Failure To Complete Planned Corrective Actions In A Timely Manner B&AD management and supervision failed to ensure that corrective actions were completed in a timely manner. In their response to CAR P-1386, B&AD indicated that the four planned corrective actions identified in the CAR would be completed by July 6, 1992.

However, as of November 21, 1992, B&AD had not completed the actions.

3.

CORRECTIVE STEPS THAT HAVE BEEN TAKEN AND THE RESULTS ACHIEVED Our corrective actions taken to date include the following immediate and interim corrective actions implemented to correct and prevent recurrence of the identified deficiencies.

Immediate Corrective Actions Implemented to Correct the Identified Deficiencies QA issued a new CAR to track the completion of the previously planned corrective actions required to correct the root cause of the deficiencies noted in CAR P-1386.

This new CAR will ensure that the corrective actions will be completed in a timely manner by B&AD.

QA reviewed a sample of recent CAR closures to determine whether other instances of premature CAR closure existed. No other premature CAR closures were noted and it was concluded that the failure to follow the CAR procedure was an isolated occurrence.

After the Resident Inspector alerted SCE to the PRQ program deficiencies, B&AD coordinated an evaluation of all occupational codes created since implementation of the PRQ program to determine ANSI 18.1-1971 applicability. New QRs were generated, as appropriate, and procedure S0123-VI-33 was revised to include the new occupation codes determined to require ANSI 18.1-1971 qualification.

B&AD also verified that a current QR exists for all personnel in each occupational code identified in the revised procedure. QRs were generated or revised, as appropriate. All personnel who required a new or revised QR were determined to meet ANSI N18.1-1971 qualification requirements.

With the completion of these actions, the PRQ program was made current.

Interim Corrective Actions Implemented to Prevent Recurrence of Identified Deficiencies B&AD has administratively established a central point of contact within the B&AD organization to be responsible for ensuring that:

1) QRs are completed, as required, and

2) the completed QRs are forwarded to CDM. Accordingly, the procedure has been revised to clarify B&AD program administration responsibilities.

B&AD has also administratively implemented monthly data verification to ensure that all personnel requiring ANSI 18.1-1971 qualification as identified in procedure S0123-VI-33 have a current QR on file.

Specifically, this validation will compare the QRs on file at CDM with the occupation codes in the Employee Information System to ensure all required QRs are on file.

A memorandum was issued on December 17, 1992, to nuclear organization managers and supervisors emphasizing their responsibilities for determining the applicability of, and ensuring compliance with, ANSI 18.1-1971 qualification requirements in the areas of: 1) hiring new employees,

2) promoting existing employees, and 3) other situations which may result in an employee requiring ANSI 18.1-1971 qualification.

4.

CORRECTIVE STEPS THAT WILL BE TAKEN TO PREVENT RECURRENCE Our corrective actions to prevent recurrence will include: 1) QA review of this NOV and SCE's response, 2) QA evaluation of the CAR closure process, 3) management and supervisor training for PRQ program responsibilities, and

4) implementation of a new PRQ program.

QA Review of NOV and SCE's Response QA will review this NOV and the response with all affected QA personnel to ensure continued compliance with the procedure requirements for closure of CARs.

This review will be completed by February 15, 1993.

QA Evaluation of CAR Closure Process QA will evaluate the CAR closure process to determine if additional program provisions need to be established to ensure adequacy of QA verification of completed corrective actions prior to CAR closure. This evaluation will be completed by March 30, 1993.

Manager and Supervisor Training for PRO Program Responsibilities B&AD will provide training to nuclear organization managers and supervisors to review the deficiencies identified in the PRQ program. This training will emphasize that, as appropriate, managers and supervisors are responsible for ensuring their employees comply with requirements of procedure S0123-VI-33.

This training will be completed by April 15, 1993.

Implementation of a New PRO Program A team has been formed to evaluate the PRQ program.

Based on the recommendations of this team, SCE will implement enhancements to the PRQ program which will further ensure compliance with the regulatory requirements. The new PRQ program will be implemented by April 15, 1993.

5.

DATE WHEN FULL COMPLIANCE WILL BE ACHIEVED Full compliance was achieved by November 20, 1992, when the following was completed: 1) the existing PRQ program was made current, and 2) B&AD implemented a monthly data verification to ensure that all personnel requiring ANSI 18.1-1971 qualification have a current QR on file.