IR 05000341/1992018
| ML20127A720 | |
| Person / Time | |
|---|---|
| Site: | Fermi  |
| Issue date: | 01/07/1993 |
| From: | Burgess B, Rescheske P, Wright G NRC OFFICE OF INSPECTION & ENFORCEMENT (IE REGION III) |
| To: | |
| Shared Package | |
| ML20127A694 | List: |
| References | |
| 50-341-92-18, NUDOCS 9301120050 | |
| Download: ML20127A720 (38) | |
Text
..
..
-
-
.
.
_
Ib'
....
.
.
U.
S.
NUCLEAR REGULATORY COMMISSION-REGION III-Report No. 50-341/92018(DRS)
Docket No. 50-341 License No. NPF-43
'
Licensee: The Detroit Edison Company l
6400 North Dixie Highway Newport, MI 48166 Facility-Name:
Fermi.2 Nuclear Power Station Inspection At:
Newport, MI 48166 Inspection Conducted:
November 02 - 23, 1992
.
OYI/12d b} (N kb Inspectors:
Old
'PegcJyi RJ.. Rescheske, RIII Datd
'
Team Leader
~
R. A.
Langstaff, RIII M.
E.
Bielby, RIII
R.
S. Cain,.EG&G c
L. T. Ostrom, EG&G f'
n 6)/ 3-
. Reviewed-By: d 'Yfua MNfdeu.-
o/
-Bruce L.
Burgess.f RIII Date Team Manager Approved By:,CM aMM MA/
o/ 67 /93 Geoffrey-C. Wright,(Chief Datie
,'
Operations Branch, RIII
.
$
9301120050 930108 PDR ADOCK 05000341
,
G PDR
.
.,, -, _.,.. _.
.
. ~.
.
... _, _, _ _.....,
.
_-
.
.
_ _ =, -.
.
.
' '
Inspection Summary Insoection on November 02 - 23. 1992 (ReDort No. 50-341/92018(DRS))
Areas Inspected:
Routine, announced team inspection, to verify that the Fermi EOPs were technically correct and useable, and to determine that the quality of the EOPs could be controlled and maintained over time.
The inspection was conducted in accordance with NRC Inspection Procedure No. 42001.
Results:
One previously identified Open Item (341/90007-04,'LRP))
was closed.
Seven previously identified operator exam issues were reviewed and resolved.
Three open items were identified.
The results of the inspection were as follows:
-
(1)
The EOPs were technically correct and met the BWR Owners Group accident mitigative strategy.
Plant specific deviations from the vendor guidelines were adequately justified and documented.
Supporting information, including calculations, was also adequately documented.
,
(2)
Several minor procedure deficiencies were identified (e.g,
omission of support procedure references in the_ flowcharts)
during the desktop reviews of the EOP flowcharts and; support'
procedures.
(Open Item 341/92018-01a(DRS))
(3)
No significant procedural deficiencies-were noted during the walkthroughs of the EOP flowcharts and support procedures; the operators could generally accomplish the EOP tasks as written, and the required equipment was generally available and pre-staged.
However, several weaknesses were identified (e.g.,
labeling) which have the potential'to.cause operator delays and confusion.
(Open Item 341/92018-01b(DRS))
(4)
The verification and validation (V&V) program was generally adequate, however, sufficient guidance was not provided to ensure successful completion of the V&V for the flowchart EOPs.
Significant weaknesses were identified in the-implementation of the V&V program, particularly for,that conducted prior to EOP flowchart issuance.
In this regard, the V&V effort was both-incomplete and ineffective, especially from a haman factors standpoint.
(Open Item 341/92018-02(DRS))
(5)
The EOP flowcharts and Writers Guide were developed and verified without appropriate consideration of human performance principles.
As a result, the Writers Guide did not effectively control the presentation of information in
,
,-n
--.-,-n- - -
,,--g
,
-
,
,--
,, -, -
-
n-
,-
.,---m n
-
__
_
_
._
_ _
_
-.
.
the EOP flowcharts, and the flowcharts were poorly organized, formatted, and written.
The weaknesses identified in-the'EOPs.have the potential for affecting operator performance in high stress situations.
(Open Item 341/92018-03(DRS))
(6)
Although training was not specifically evaluated and the EOPs were not exercised on the simulator, several operator
-
training concerns were identified.
(7)
Notwithstanding the deficiencies identified by the Team, the annual-Quality Assurance audits of_the EOPs were relatively
'
adequate in scope and frequency, and appeared effective in identifying and resolving deficiencies-in the areas audited, particularly when compared to other utilities.
.
.
i -
i
?
,
!
1
'
l-
_.. _ -... _ _ _ _,. _ _ _. _, _, _., _ -,. _.
. -
.
.
..
-
...
.
-..
.
.
.
CONTENTS Egranraph Pace 1.
Persons Contacted.......................................
2.
Overview................................................
3.
Action on Previously Identified Items...................
4.
Desktop Review..........................................
5.
Walkthroughs............................................
_
6.
Verification and Validation (V&V)......................
7.
Human Factors Review...................................
8.
Training and Qualification Effectiveness...............
'
9.
Quality Verification Effectiveness.....................
10.
Open Items.............................................
11.
Exit Meeting...........................................
Appendix A:
Description of Inspection Activities
-
-
Appendix B:
Detailed comments on the EOPs and Support Procedures Appendix C:
Detailed comments from the Human Factors Review Appendix D:
List of Acronyms Used in the Inspection Report A
_ _ _ _ _ _ _ _
_ _. _ _. _ _ _. _ _ _. _ _ _ _ _ _ _ _ _ _ _. _ _ _. _.. _ _ - _ _ _ _ _ _ _ _ _ _ _ _. _ _. _ _ _ _ _ _ _ _ _ _ -. _ _ _ _ _ _ _. - - _ _ _ _ _ _ _ _ -. _ _ _. _. _ _ _ _ _ _ _. _ - _ _. - _ _. _ _.. _ _ _ _ _ _ _ _ _ _ _. _
___
..
.....
.
.
Report Details 1.
P_elpons Contacted Detroit Edison Company W.
S.
Orser, Senior Vice President
- D.
R. Gipson, Vice President, Nuclear Operations G.
Diserens, Quality Assurance Specialist J.
P.
Flint, EOP Coordinator
- L.
Goodman, Director, Nuclear Quality Assurance R.
Henson, Operation Engineer M.
E.
Hoffmann, Principle Quality Engineer
- R.
M.
McKeon, Plant Manager
- R.
A.
Newkirk, General Director, Regulatory Affairs R.
M.
O'Sullivan, Operations Support
- 0 P. Ockerman, Training
- J.
M.
Pendergast, Compliance Engineer D.
J.
Piening, Jr.,
Senior Nuclear Training Specialist
- J.
H.
Plona, Superintendent, Operations
- A.
C.
Settles, Director, Nuclear Licensing
- G.
E.
Smith, Operations Support Engineer
- R.
B.
Stafford, General Director, Nuclear Assurance
- J.
A.
Tibai, Supervisor, Compliance
- J.
G.
Walker, General Director, Nuclear Engineering Recion III NRC
- K.
Riemer, Resident Inspector, Fermi Plant W.
J.
Kropp, Senior Resident Inspector B.
L.
Burgess, Chief, Operational Programs Section, Division of Reactor Safety (DRS)
- G.
C.
Wright, Chief, Operations Branch, DRS
-
Individuals indicated by an asterisk attended the management exit meeting held on November 23, 1992.
Other persons were contacted during the inspection incle. ding members of the licensee's operations, training, and quality assurance staffs.
2.
Overview a.
DackgrouDd Emergency Operating Procedures (EOPs) have undergone significant changes due to the 1979 accident at the Three Mile Island (TMI) facility.
The post-TMI
_______ __
_ _ _ _ _ _ _
_
I
.
.
procedures are symptom-oriented rather than event-based.
Symptom-oriented EOPs provide the operator guidance on how to verify the adequacy of critical safety functions and how to restore and maintain these functions when they are degraded.
Symptom-oriented EOPs are written in a manner that the operator need not diagnose an event to maintain the plant in a safe shutdown condition for accidents that are within the scope of the EOPs.
EOPs were required to be established, implerented, and maintained by NUREG-0737 and Supplement 1.
In 1990, the NRC issued the Gafety Evaluation for the Fermi Procedures Generation Package (PGP) which was based on Revision 3 of the BWR Owners Group (BWROG) Emergency Procedure Guidelines (EPGs).
Revision 4 of the EPGs was issued in March 1987, and was_ implemented at the Fermi Plant in 1988.
An NRC EOP Team Inspection was conducted in July 1988 (Inspection Report No. 50-341/88200, dated September 08, 1988).
The inspection included a review of the draft version of EOPs for the upgrade to Revision 4 EPGs.
In July 1991, flowchart format EOPs were issued at Fermi.
The purpose of this inspection was to determine that-the Fermi EOPs, including support procedures, were technically correct and useable; verify that specified actions could be accomplished by the operations staff using existing equipment, controls, and instrumentation; determine that the Writers Guide was adequate and that the EOPs were prepared in accordance with this document; and verify that there were assurances that the quality of the EOPs would be controlled and maintained over. time, such as with continuous verification and validation, and management and quality assurance involvement.
The Fermi EOPs consisted of flowcharts and text format support procedures.
Although the licensee continued to maintain the text format EOPs, the flowchart EOPs were used almost exclusively by the operators.
The majority of the supporting procedures necessary for performing
~
the EOPs were maintained as attachments to the text-format EOPs. -Several other supporting procedures _were portions of normal system operating procedures.
The EOPs were controlled and maintained by-Administrative Procedure NPP-PR1-03, " Emergency Operating Procedure Development Process;" and_the-Fermi Writers Guide, " Appendix E: Emergency' Operating Procedures" and " Appendix F: Emergency Operating Procedure Flowcharts."
The administrative procedure
,.
-.
_
.
.
.
. -
_
.
..
..
...
.
-
.- -....
-
-
-_.
_
_
.
.
also contained the verification and validation (V&V)
program, and the requirements regarding preparing and upgrading the EOPs.
The Fermi Plant Specific Technical Guideline (PSTG) was based on Revision 4 to the BWROG EPGs.
Deviations from the EPGs were required to be
-
technically justified and documented in the-Differences Document.
The EOPs were required to be developed from the PSTG using the Writers Guide.
Supporting docur
,n included the.PSTG to EOP Implementation List ocumented the differences between the PSTG and \\
nat EOPs, and the Engineering Design
'
Calck (EPG Appendix C Calculations) for the vario'
specific parameters used in the EOPs.
J Prior oing the EOPs, Procedure NPP-PR1-03-
,
requireo that V&V be conducted _to confirm technical
accuracy and-written correctness, and useability of the EOPs.
b.
Inspection Methodoloav
,
The inspection consisted of a technical and human factors desktop review of selected Fermi EOPs and support procedures; review of the Writers Guide and V&V program including implementation for the conversion to flowchart EOPs; control room and in-plant walkthroughs conducted with. operations staff; interviews with EOP'
preparers, users,-and training staff; and an assessment of quality assurance effectiveness in the area of EOPs.
A detailed listing of these activities, including a list of the documents utilized.during the inspection, is given in Appendix A of this report.
While training.
was not specifically evaluated, several observations in this area resulted from the inspection.
The EOPs were not exercised on the simulator, c.
Inspection Summarv-The Team concluded that the-Fermi.EOPs were technically correct, and were adequate for mitigating the accident scenarios described in the BWR Owners Group EPGs.
However, the EOP flowcharts and Writers Guide were developed and verified without adequate consideration of human factors principles.
The Team determined that the EOP flowcharts lacked consistent application of the human engineering principles to support. operator-performance.
The result was that under a high stress accident sittation, an increased potential existed for
,-
L delays due to operator error and confusion.
'
.
a w.,
-- b
,,, -, - -.
..--y.~
. - - - -
--.--..m
-
--r
,w
.e---
.
.
...
. _...
-.
_ -.
- -
.
..
. _. -. -.-_
--
.
.
The Team concluded that the EOP support procedures were technically correct.
In general, the operators could accomplish the procedures as written, and the required equipment was available and pre-staged.
However, several weaknesses were identified (e.g.,
labeling)-
which have the potential to cause operator delays and confusion.
The verification and validation (V&V) program was generally adequate in stated requirements; however, sufficient guidance was not provided to ensure successful completion of the V&V for the EOPs.
The Team determined that the V&V offort for the transition to flowchart EOPs was both incomplete and ineffective, from a human factors and independence of review standpoint.
The EOP Writers Guide was not considered adequate to control the presentation of information in the EOPs.
The Writers Guide did not contain the guidance necessary to assure consistently structured, easily understood procedures, that minimized error and supported operator performance.
The Writers Guide contained guidance inconsistent with human factors principles, nonrestrictive and ambiguous guidance, and omissions in guidance.
These weaknesses were reflected in the EOPs.
The Team concluded that the quality of the EOPs may not be controlled and maintained-over time, such as, in procedure revisions and changes in personnel.
This was primarily due to the weaknesses identified in the Writers Guide, and the V&V program ineffectiveness.
In the areas where weaknesses were identified, the Team determined that the Fermi EOP program'and implementation generally did not meet industry and NRC accepted standards and guidance (e.g., NUREG-1358 and NUREG-0899), and in some cases did not meet the intent of the licensee's plant administrative procedures.
All inspection findings and conclusions were discussed in detail with licensee staff during the inspection-and at the conclusion of the inspection ~in a debrief.
The detailed-findings-identified in this inspection report, including those in Appendix B, were representative of the results from the desktop reviews-and walkthroughs, and may not be inclusive of all deficiencies within the EOPs.
Three open items'were' identified to document the identified weaknesses.
A licensee response to these items was requested.
-_
_.
-
_
_
..
.
... _ _ _ _ - _ _ - _ _ _.
l
.
.
3.
Action on Previously Identified Items a.
(Closed) Open Item (341/90007-04(DRP)):
Use of reactor core isolation cooling (RCIC) as a pressure control method.
This open item documented the inability of the operators to use RCIC in the full flow test mode following a reactor scram from 100% power on April 10, 1990.
In the test mode, RCIC takes suction from the condensate storage tank (CST) and discharges to the CST.
To open the high pressure coolant injection (HPCI) test return valve E41-F011, the discharge piping was required to be vented since RCIC/HPCI discharge pressure was on one side of the valve and CST pressure was on the other side.
During the event, the operators initially failed to vent the piping.
Subsequently, when the piping was vented, attempts to open the valve failed.
The licensee's investigation revealed that the use of the CST-to-CST pathway was severely hampered by an undersized motor operator on the test return valve, resulting in the need to shut down the RCIC turbine and vent the test return line prior to use.
Corrective actions included revision and verification of the affected procedures supporting the EOPs; additional operator training; engineering evaluations; and review of the EOPs for other potential hardware, procedure, and training weaknesses.
All actions have been completed with the exception of Design Change No.
11655, scheduled for the next refueling outage, which would replace the HPCI test return valve E41-F011, increasing the size of the motor operator.
This open item is considered closed.
b.
The inspectors reviewed seven issues involving the EOPs identified by NRC Examiners during the December 1991 Operator License Requalification Examination at Fermi (Examination Report No. 50-341/OL-91-02).
The concerns were, in general, a result of operator training weaknesses and/or human factors related problems in the EOPs.
The concerns involved the following issues:
RCIC termination, level band during an anticipated transient without a scram (ATWS), anticipation of emergency depressurization, alternate boron injection with standby feedwater, safety relief valve (SRV)
opening during reactor pressure vessel (RPV) flooding, steam line isolation, and wording of a level override statement.
The inspectors verified that the licensee had reviewed each issue and taken corrective action.
The Team considered several of these issues to be further examples of human performance problems inherent in the EOP flowcharts.
Results of the inspection related to training and human factors are discussed in the following paragraphs.
i i
_ _ _ _ _ _
- - -
. - - -
.. - - - -
- -
. -.. -.
. -. _ - - -. _
-
t
.
.
4.
D_erdtpn Review A desktop review of the EOPs and EOP program, and supporting procedures and documentation was performed by the Team.
No technical problems were noted.
The desktop review included a
comparisons of the Fermi Plant Specific Technical Guidelines (PSTG) to Revision 4 of the BWR Owners Group Emergency Procedure Guidelines (EPCs), and review of the documentation used to justify deviations from the EPGs.
The calculational basis for the plant specific parameters and curves used in the EOPs was also reviewed.
The EOP flowcharts were
!
reviewed using the EPGs, PSTG, and the Writers Guide as a basis.
The text format EOPs were used for reference during l
the inspection, and were reviewed as part of the EOP technical basis.
The plant procedures referenced or required during performance of the EOPs were also reviewed (e.g.,
EOP support procedures and portions of system operating procedures).
The results from the human factors review of the Writers Guide and EOPs are in Paragraph 7.
The Team also reviewed documentation related to the installation of hardened containment venting capability (Generic Letter 89-16).
The licensee installed this modification during the third refueling outage.
The Team reviewed portions of the modification package, applicable drawings, and_affected procedures, including the EOP
" Primary containment control" and supporting procedures.
Based on the review of the flowcharts and support procedures, the Team concluded that the EOPs were technically correct and met the BWR Owners Group accident mitigative strategy.
Plant specific deviations from the vendor guidelines were adequately justified and documented.
Support documentation, such as the basis for plant specific setpoints, curves, and tables used in the EOPs, was adequately documented, including calculations and assumptions.
This information was also correctly incorporated into the EOPs.
The Team identified minor procedure deficiencies, such as omission of support pr ocedure references in the flowcharts.
Representative examples are provided in Appendix B of this. inspection report.
Licensee resolution of these items will be tracked as an Open Item (341/92018-Ola(DRS)).
-
In general, the development of the EOPs was not-consistent- -
with NRC and industry accepted standards and guidance.
Foi example, the Team noted that the Revision 4 text EOPs were
,
basically written'using the exact wording from the vendor guidelines, rather than language familiar to the operators.
The vendor guidelines were not intended to be used in this
'
manner, since the guidelines were not procedures and have not been human engineered.
Although this method of
,
,
_. _ _ _ _ _ _
_ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _. _ - _ _
_
.
.
procedure writing accomplished technically correct EOPs, it did not incorporate the human factors principles which support operator performance.
In the transition to flowchart EOPs, the exact wording from the text EOPs was carried over to the flowcharts, with only minor changes.
The primary benefit of converting text to flowcharts is the procedural simplification provided by flowcharting techniques.
For example, the benefit in using symbolism was lost in most cases because-the text was not simplified.
In
addition, certain techniques for managing procedure steps were not considered, such as, separating the RPV Control EPG into ATWS and non-ATWS flowcharts.
Also, information was not always placed where it was used, such as, the Fermi EOP curves, cautions, and tables, which were placed together on a separate sheet, rather than located on the applicable EOP flowcharts.
'
The EOP flowcharts and Writers Guide were developed and verified without adequate consideration of human factors principles.
For example, operations and technical persons were used, rather than a mult1 disciplinary team approach with a human factors specialist.
In general, technical and operations personnel have difficulty in objectively considering the human performance aspects of procedures.
As a result, while the EOPs were technically correct, the flowcharts were poorly organized, formatted, and written.
,
5.
Walkthrouchs The objective of the walkthroughs was to verify-that operator actions required by the procedures could be implemented in a timely manner with minimum potential for error or time delays which could affect safety.
'
Walkthroughs of the EOP flowcharts and supporting procedures were conducted by the Team, accompanied by licensed or non-licensed operators who would normally perform the procedures.
Senior Reactor Operators (SROs) were-used for
'
flowchart walkthroughs in the control room, Reactor Operators (ROs) were used for support procedure walkthroughs-in the control room and relay room, and non-licensed operators (NLos) were used for in-plant support precedure walkthroughs.
No significant procedural deficiencies were noted; however,
- -
weaknesses were identified (e.g.,
labeling) which have the potential to cause operator delays or confusion during performance of the EOPs.
These weaknesses should have been identified by the licensee's V&V.
The Team also identified severa1' concerns related to human factors and training which are discussed in Paragraphs 7 and 8, respectively.
The results from the walkthroughs were as follows, with
._
. u..
_ _., _ _,.._.- _. _ - _ _ _ _ _ __ _ _ _
_. - _, _.
...
_. _ _ -,
- - _ _ _ _ _ _ _ _ _
_
.
.
additional representative examples relating to specific procedures provided in Appendix B of this inspection report.
Licensee resolution of these items will be tracked as an open Item (341/92018-Olb(DRS)).
a.
The SRos coald simulate performance of the EOP steps in the control room and generally follow the flowchart, including transitioning between and within the charts.
When ROs were requested to interpret various flowchart steps, some operators needed to read the step several times before understanding the information or direction.
b.
In general, the operators had little difficulty locating in-plant or control room components required for accomplishing EOP tasks.
The support procedures could generally be accomplished as written.
However, the team identified weaknesses in plant labeling, which have the potential to cause delay and operator confusion.
In some cases, the labeling method did not provide clear direction to the operator, particularly for terminal locations in the relay room; in other cases the label was missing, deficient, or not approved for use; and in several examples the label and procedure did not match.
Several of these discrepancies were immediately corrected by the licensee.
The Team noted that special EOP labeling was used on some components which could be helpful to the operator, especially for relays and terminals inside electrical panels.
However, in some cases, the labeling was either not consistent, misleading, or insufficient to adequately direct the operators to the component.
The following examples were from EOP Support Procedure 29.000.01, RPV Control, " Interlock Defeats."
(1)
Sections 5 and 6,
" Defeat of RWCU Isolations" and
" Defeat of ARI Logic Trips," required an operator to lift leads from terminal strips.
For several of the terminals, multiple leads entered one side of the terminal strip.
Consequently, part of an electrical circuit could still be made if only one lead was lifted.
During the procedure walkthrough, an operator questioned whether lifting one lead would be sufficient or whether
l
-_
.
_ _ _ _ _ - _ _ _ _ _ _ _ - _ _ - _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
- _ _ _ _ _ - _ _ _ _ _ _ _ _ - _ _ _ - -
__ _ _ - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
.
.
lifting all leads would be acceptable.
The operator would have consulted the control room for further guidance before proceeding, possibly resulting in delay in completing the task.
In this example, the EOP labeling did not appear sufficient to adequately direct an operator to lift the required lead.
(2)
Section 7,
" Defeat of RPS Logic Trips," required an operator to jumper between relays using specific terminals.
The terminals to be jumpered could only be identified by the terminal numbers embossed on the relay cases.
Because the numbers were black raised letters and in a recessed area, it was difficult for the operator to identify which terminals were to be jumpered.
Further, although the EOP labeling identified the terminal number to be jumpered, the labels were located on different corners of the relay than the terminal to be jumpered.
Consequently, the location of the labels could mislead an operator to jumper from an incorrect terminal.
(3)
Section 9,
" Defeat of Standby Feedwater Level 8 Trip," required an operator to remove a plug-in relay.
The EOP label which identified this relay was located on a nearby terminal strip, rather than directly below the specified relay.
The location was not consistent with other EOP labeling for relays.
During the procedure walkthrough, the operator was delayed (for approximately onc minute) while locating the specified relay.
_
c.
The required equipment for performing the support procedures was generally available and pre-staged.
Large equipment, such as hoses, portable pumps, ladders, and heavy tools, was maintained in dedicated EOP support lockers in the plant near where the procedures were to be performed.
Equipment.and tools were generally adequately controlled and maintained.
Equipment required for performing several of the support procedures was contained in EOP support packets located in a drawer in the shift supervisors office.
In addition to necessary tools, electrical jumpers, and keys, the support packets contained a copy of the procedure section to be performed.
In the case of tasks performed in the relay room, a map of the relay room indicating the applicable electrical cabinets was also included.- Except for the maps, the contents of the support packets were audited on a regular basis.
l
-
_ _. _ _ - _ _ _ - _ _ _ - - _ _ _ - - _ _ _ _ _ _ _ - _ _
_ _ _ _ _ _ _ _ _ _ _ _ - - _ - - _ _ _ -
.
.
_m
.
-.
Although the equipment was controlled and maintained in a drawer, the lack of understanding the organization of
,
the drawer and the lack of a list detailing the contents led to some confusion during several procedure walkthroughs.
For example, a shift supervisor spent several minutes trying to locate a nonexistent packet for the EOP-support procedure, " Reactor Pressure Vessel Injection Using SLC Test Tank."
The packet did not exist because equipment was not required.
d.
Most electrical cabinets required to be accessed-by EOP support procedures were accessible.
However, potential accessibility problems were identified during the walkthroughs for Section 4 of the Interlock-Defeats support procedure 29.000.01, " Defeat of all MSIV and Main Steam Line Drain Valve Isolation. Signals."
Electrical cabinets H11-P622 and H11-P623, located in the relay room, had terminal strips _ located deep inside the cabinets which required; access.
These terminal strips could only be reached.by stepping completely into the cabinet.
Because of the narrow width of the cabinets, the relatively low light conditions,-and the moderately high location of the terminals to be used (about six feet), lifting leads and jumpering between terminals would be performed with some difficulty.
In addition to having the potential to cause delays, since the panels were energized, performing these-actions posed an electrical hazard for the operator and a risk of unintentionally shorting out other circuits in the cabinet. For these reasons, the licensee should consider alternative methods for performing some of the tasks in the relay room.
6.
Verification and Validation (V&V)
The purpose of a verification and validation.(V&V) program for EOPs is to verify that the EOPs are useable, ensure written correctness, and ensure technical accuracy'.
The V&V program is intended to control the quality of the-EOPs over
. time as they are revised due to plant design changes, regulatory _information, vendor notices, or revisions to procedures referenced by or referencing the EOPs.
In addition, V&V serves as a check against weaknesses in development _of the Writers Guide and EOPs.
The Team reviewed the Fermi V&V program contained in L
Administrative Procedure.NPP-PR1-03, " Emergency Operating L
Procedure Development Process," Revision 3, and L
documentation for V&V conducted for the EOPs and support
,-
.
._
_.
.~a.
--
-
-
-
...-
. -. - -
-
,.
.
-
.
.
procedures since December 1990.
The documentation included V&V for the two major revisions to the EOP flowcharts; i.e.,
the original issue in July 1991, and the revision in October 1992 due to the power uprate.
The Team identified significant weaknesses in the implementation of the V&V program, particularly for the transition to flowchart EOPs.
Licensee resolution of the weaknesses described below, identified in the V&V program and implementation, will be tracked as an Open Item (341/92018-02(DRS)).
a The Fermi V&V program contained many of the elements a.
essential to a good program, and was potentially capable of resulting in a satisfactory verification and validation effort.
However, the program was nonrestrictive and lacked clearly defined criteria, which contributed to the weaknesses identified in the V&V for the transition to flowchart EOPs.
(1)
The program failed to define criteria for the scope and depth of the V&V required for a particular procedure change.
As a result, V&V was incomplete for the transition to-flowchart EOPs.
(2)
The program lacked clearly defined criteria for.
selection of V&V individuals and V&V team composition.
Though the program listed the types of participants in the team, qualifications were not defined.
For example, " individuals qualified in human factors" was used as the criteria for a human factors specialist.
As a result, human factors reviews were conducted by individuals not knowledgeable in human _ performance principles.
(3)
The program did not adequately restrict involvement in the V&V by the EOP coordinator or those directly involved in EOP development.
In
,
i addition, the program failed to distinguish between the individuals performing _the procedure (e.g., operators) and those conducting the V&V (observers).
As a result, there.was little or no independence of review.
b.
While the V&V program was generally adequate in stated
'
requirements, sufficient guidance was not-provided to ensure successful-completion of the V&V for the EOPs.
Significant. weaknesses were identified in the implementation.of the V&V program, particularly for that conducted prior to EOP flowchart issuance.
In this regard, implementation of.the program was not consistent with NRC and industry accepted standards and-guidance; in some cases, the V&V effort did not meet the intent of licensee's V&V program.
The ineffective l
l
l
-..
--.
.
-
-.
.
.
-
-
-__
_ _ _ _ _ _ _ _ - _ - _ _ _ _ _ _ _
.
.
or lack of V&V contributed to the human factors deficiencies identified in the EOPs, and reinforced the Team's concern that the EOPs may not be controlled and maintained over time.
(1)
The transition to flowchart-format EOPs was a complete re-write and a significant change to operator use of the procedures.
In this regard, the licensee did not meet the intent of the program because a full V&V was not performed for the flowcharts prior to their issue.
The V&V which was conducted, was not performed by a multidisciplinary team with a human performance specialist.
Furthermore, the V&V team was composed of persons directly involved in EOP development.
As a result, one of the primary benefits of the V&V process, that being independence of review, was eliminated.
(2)
The majority of the deficiencies in the procedures were directly related to the lack of human factors expertise used in the V&V of the flowcharts.
Independent verification by a person knowledgeable in human performance principles would have identified the weaknesses in development of the flowcharts before the procedures were issued.
For example, an independent human factors review would have detected the inappropriateness of combining logical " ANDS" and " ors" in the procedures.
(3)
For the simulator validation of the flowcharts, the validation team was also the operating crew, and all individuals were operators.
Not only was there a complete lack of independence in this function, this was in conflict with the intent of the licensee's validation program, which called for a validation team separate from the validation operators.
(4)
A single person performed both the technical accuracy and written correctness verifications for the transition to the flowchart EOPs; the same individual was directly involved in developing the writers guide and the flowcharts.
Therefore, there was a lack of independence in this function, in conflict with the intent of the licensee's verification program.
The Team further noted that this verification was not complete in verifying written correctness (e.g., not all flowchart elements were reviewed), and was not effective in identifying all errors in the flowcharts (e.g.,
omissions in support procedure references).
1
- --
%_________ _ _ _ _ _ _ __. _ ___.. _ _
_. _.... _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _. _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
. _ _ - - _ _ _ _ _ - _ _ _ _ _ _ _ _ _
l
.
=
(5)
Regarding support procedure validation walkdowns, a single operator routinely performed the walkdown and evaluated their own performance.
This did not meet the definition of independent validation.
Based on the labeling deficiencies and other concerns identified by the Team, this validation was not completely effective.
7.
Human Factors Review The Writers Guide and Emergency Operating Procedures were reviewed for conformance with accepted human factors principles as described in NUREG-0899, NUREG-1358, and NUREG/CR-5228.
Weaknesses in areas considered related to potential human error were identified in the writers guide and the flowcharts.
Although individually these weaknesses would probably not cause the procedures to be ineffective during an emergency condition, taken collectively they represented a significant reakness in the EOPs, which could increase the potential for operator error, delays, and confusion in high stress situations.
Licensee resolution of the weaknesses and related examples identified during the human factors review will be tracked as an Open Item (341/92018-03(DRS)).
These weaknesses are summarized below with representative examples given in Appendix C of this inspection report.
a.
Writers Guide (WG)
Fermi Writers Guide, " Appendix F: Emergency Operating Procedure Flowcharts," was the Fermi flowchart EOP writers guide (WG).
The Team concluded that use of the current WG to control the presentation of information in the EOPs, will not assure consistently structured, easily understood procedures, that minimized error and supported operator performance.
The WG contained guidance inconsistent with human factors principles, nonrestrictive and ambiguous guidance, and to a lesser extent, omissions in guidance (e.g.,
lack of sufficient examples).
These weaknesses could result in increasing inconsistency in the EOPs over time and changes in procedure writers.
b.
Emeroency Operatina Procedures The human factors review of the EOPs identified numerous examples related to the weaknesses in the WG and in the development of the EOPs.
General examples incluce:
unnecessary steps (including redundant information), very wordy steps (where symbolism is basically lost), complex steps (such as those combining the logic terms and/or), potentially confusing steps
-
- - _ _ - _ _ _ _ _ _ _ _ _ _ _ - _ _ _
_ _ _ _ _ - _ _ _ _ _.
_ _ _ _ _ _ _
.
~
(such as those containing inappropriate negatives),
excessive transitioning (which was basically due to the method used to develop the flowcharts), ambiguous action steps, inconsistent and undefined terminology, necessary information not contained on the flowcharts (such as curves, tables, and cautions), and a general lack of emphasis techniques.
In addition, some steps were written contrary to the WG.
c.
Suonort Procedures A cursory human factors review of the support procedures was conducted.
From this review, it was determined that the procedures were generally written according to the Writers Guide for the text procedures with few exceptions.
,
d.
Interviews Interviews were conducted with SROs, ROs, training personnel, and procedure development personnel.
The interviews clarified and augmented other inspection findings, including concerns related to developing and verifying the EOPs.
In general, interviews with the SROs verified that they were well trained on the procedures, and understood the meaning and purpose of the procedure steps.
The ROs interviewed appeared not as well trained nor as familiar with the procedures.
A summary of the interviews is given in Appendix C.
8.
Trainina and Oualification Effectiveness Although training was not specifically evaluated by the Team, several observations resulted from the interviews and
-
procedure walkthroughs with the operations staff, a.
It was evident from the walkthroughs that training on the support procedures was generally effective.
The NLos and ROs understood the task and could perform the procedure with little difficulty.
Additional emphasis in training regarding the contents and organization of the drawer containing the EOP support procedure packets would be beneficial.
The Team also noted that in at least one case, the operator had never performed the task of pulling a relay similar to that required in, for example, RPV Control support procedures for Interlock Defeats, Sections 1, 3,
8, and 9.
b.
Although the SRos appeared to be well trained on the flowcharts, it was also apparent that training sometimes was used to overcome human performance problems in the EOPs, rather than changing the
1
_
- _ ~ - - - _. _. - -.. _ _ _ _ _ _.. _ _
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
_
_
.
.-
procedure.
In this regard, several problems with transitioning were mentioned by the operators.
For-example, because EPG step numbering was used in the flowcharts, on several occasions during training, operators would inadvertently transfer to step C1-1.2-of flowchart "C1 - RPV Flooding" (Procedure 29.100.01)
instead of step Cl-2 as directed by the second override.
Instead of recognizing ~the difficulty in the procedure and simplifying the numbering scheme, the operators were retrained to transfer to the correct step.
c.
It was apparent that the operations staff had not received sufticient training on-the methods available for commenting on procedures.
No two operators explained the process the same during the walkthroughs and interviews, 9.
Quality Verification Effectiveness The licensee conducted annual Quality Assurance (QA) audits of the EOPs.
Notwithstanding the' deficiencies identified by the Team, the audits were relatively adequate in scope and frequency, and appeared effective in identifying and resolving deficiencies, particularly when-compared to other utilities.
The EOP audits routinely included observation of operator training on the simulator exercising the EOPs, in-plant walkthroughs of the.EOP support procedures, and review of EOP technical basis information.
The 1992 EOP audit'was in progress at the time of this inspection, and the licensee discussed the results of this audit with the Team.
The most significant finding from the audit was identification:of-several errors in the Appendix C Calculations which had resulted from the revisions to the calculations for the recent power uprate.
Deviation event report No. 92-0577 documented this QA finding.
(The licensee determined that the errors had negligible affect on the results of the calculations-and did not impact the curves in-the EOPs.)
The Team also reviewed QA Audit No. 91-0197,. which-was conducted in 1991, and the documentation-related to corrective action taken in response to the audit findings.
The audits had not identified the human factors related deficiencies documented in this inspection report.
This wasi primarily due to the lack of human factors expertise-and unfamiliarity with the EOP-related standards and guidance, such as NUREG-1358.
Although it might not have been
-
y expected that the audits would have identified human factors
-
related concerns, the audits should have identified the
'
- -
-
__
_
_ _. _. _ _ _ _
.. _ -
.
_.. _... _ _. _ _ _ _. _
- _ _ _
_.
. -. - _ _ _
.
_
._
_-
.
,
weaknesses in the V&V program and implementation documented in this inspection report.
Further, quality assurance-involvement during EOP development would have been beneficial from the standpoint of independence of review.
10.
Open Items Open items are matters which have been discussed with the licensee which will be reviewed further by the NRC and which
'
involve some action on the part of the NRC or licensee or both.
The three open items disclosed during this inspection are described in Paragraphs 4, 5,
6, and 7.
11.
Exit Meetina The management exit meeting was held with licensee representatives (denoted in Paragraph 1) on November 23, 1992.
The-Team Leader summarized the purpose, scope, and findings of the inspection and the likely informational content of the inspection report.
The licensee acknowledged this information and did not identify any information as proprietary.
,
,
d
..,. - -.. -.
. _. - _, - - - - - -. -
.
..
. -
_ : : ',_
.
- - -. -.
..
- -
-..
_ - _ - - -
.
.
_..-
- _..
-...-
.. _
...
- - -
_
.,
<
.
.
Y Appendix A Description of Inspection Activities All EOP flowcharts were reviewed during the desktop review as described in Paragraph 4.
These procedures wore also walked through as described in Paragraph 5.
Selected tasks in-the support procedures listed below were walked through in the control room and in-the plant (Paragraph 5).
The text format EOPs were used as a reference during the desktop review (Paragraph 4).
EOP Flowcharts (latest revision approved October 31, 1992)
29.100.01 RPV Control Sheet 1, RPV Control, Revision 2 Sheet 2,-RPV Control (Continued), Revision 2 Sheet 3, Level Contingencies, Revision 2 Sheet 4, Pressure Contingencies, Revision 1 Sheet 5, C1 - RPV Flooding, Revision 2 Sheet 6, C2 - Level / Power Control, Revision 2 29.100.02 Primary Containment Control
-
Sheet 1, Revision 2 Shtet 2, Revision 1 29.100.03 Secondary Containment'and Rad Release Control, Revision 2 29.100.04 Curves, Cautione. and Tables, Revision 1
,
Text Format EOPs (latest revision approved October 31,'1992)
29.000.01 RPV Control (RC), Revision 23 29.000.02 Primary Containment Control (PC),' Revision 17 29.000.03 Secondary Containment-Control (SC). and Radioactivity Release Control (RR), Revision 17
,
l.
~,
.a
,
-
-
.
- - - - - - -..-
.-..
-
..
-
a
.
EOP Support Procedures (attachments to the text EOPn)
29.000.01 RPV Control Alternate Boron Injection Alternate Control Rod Insertion Operation of Control Rod Drive Pumps Reactor Pressure Vessel Injection Using SLC Test Tank Interlock Defeats Primary Containment Water Level Determination Manual Operation of ARI 29.000.02 Primary Containment Control Drywell Average Temperature Calculation
--
Primary Containment Venting and Purge Primary Containment Water Level Determination Interlock Defeats Drywell Cooling Water Restoration 29.000.03 Secondary Containment Control and Radioactivity Release Control Interlock Defeats Other Supportina Procedures (nortions reauired for EOP performance)
23.107, " Reactor Feedwater and Condensate Systems," Revision 55 23.107.01, " Standby Feedwater System," Revision 16
_
23.139, " Standby Liquid Control System," Revision 20 23.142, " Auxiliary Boiler System," Revision 18 23.202, "lligh Pressure Coolant Injection System," Revision 45 23.203, " Core Spray System," Revision 20 23.205, " Residual Heat Removal System," Revision 38 23.206,
" Reactor Core Isolation Cooling System," Revision 39 23.208, "RHR Complex Service Water System," Revision 31 23.707, " Reactor Water Clean Up," Revision 62
........
.
.. _
-
.
.
Qther Documents Utilizgd Durina the Inspection Boiling Water Reactor Owners Group (BWROGj* Emergency Procedure Guidelines (EPGs), Revision 4, March 1987 (NEDO-31331)
NUREG-0899, " Guidelines for the Preparation of Emergency Operating Procedures," August 1982 NUREG-1358, " Lessons Learned from the Special Inspection program for Emergency Operating Procedures," April 1989 NUREG/CR-5228 (2 volumes), " Techniques for Preparing Flowchart-Format Emergency Operating Procedures," January 1989 Letter from F.
E.
Agosti, DECO, to E.
G. Adensam, NRR, transmitting the Fermi Procedures Generation Package (PGP), July
31, 1986 Letter from P.
L.
Eng, NRR, to B.
R. Sylvia, DECO,: transmitting the Safety Evaluation of the Fermi PGP, March 13, 1990 NRC Emergency Operating Procedure Team Inspection Report No. 50 -
341/88200, September 06, 1988 Generic Letter GL 89-16, " Installation of Hardened Wetwell Vent,
September 01, 1989 Fermi Administrative Procedure NPP-PR1-03, " Emergency Operating Procedure Development Process," Revision 4, August 06, 1991 Fermi Writers Guide, " Appendix E: Emergency Operating Procedures," Revision 3, August 02, 1991 Fermi Writers Guide, " Appendix F: Emergency Operating Procedure Flowcharts," Revision 0, August. 08,-1991 Fermi Emergency Operating Procedure Support, Documentation:
Section 1 - Plant Specific Technical Guidelines (PSTGs),
Revision 10, October 31, 1992 Section 2 - Differences Document, Revision 10, October 31, 1992 Section 3-- PSTG to EOP Implementation List, Revision 8,
- .
October 31, 1992 l
Section 4 - Appendix C Calculations (various)
Section 5 - Emergency Operating Procedure Ongoing Evaluation l
Program, Revision 0, February 10, 1989
i
_.. _... _ _ _ _.
. _
. ~... -...___ _ _
..._. _ _ _ _._. - _..... -. _._ -_.. _ _ _.-.
_ _.
..___ -._
.
.
,
i Procedure NPP-OP1-10, " Audits," Revision 12 (specifically,
attachments related to inventory of EOP equipment)_
!
Operations Procedure Comment Form," and Comment Forms:
"
Professional Advice, Comment / Feedback Form"
"
Urgent Required Reading 92-UO6, Summary of October 31, 1992, changes to EOPs (as a result of the power uprate, etc.)
,
k
?
h n
[
$
1 i>
!
. -
p I
..
Y N
I r-p c-wv--wwwm.-w.yr,-
.+e ee-
.g" w
wng-
ey y
--g.y yyr-y y-y++--
v-
'gry >yv-ww>-w y
--
u
~
m 9.
v
. _ _ - - _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
l l
.
.
I APPENDIX _D Detailed Comments on the EOPs and Supportino Procedures 29 100.01 Sheet 1.
RPV Control o
Step RC/Q-4 omitted the reference to the EOP support procedure for " Manual Operation of ARI."
o Step RC/L-2.1 referenced procedures 23.203 - 23.205, however, there was no procedure 23.204.
O The second override in step RC/P-2 referenced the EOP support procedure for " Defeat of MSIV cnd Main Steam Line Drain Valve Level 1 Isolation Signals."
However, the override in the flowchart stated "OPEN MSIVs, B/P DW PNEUMATIC SYSTEM AND LOW RPV LEVEL ISOLATION TO RESTORE MN CONDENSER AS A HEAT SINK," with no mention of defeating isolation signals.
29.100.01 Sheet 2.
RPV Control o
Step RC/Q-7.2 referenced coveral sections of an EOP support procedure (Alternate Control Rod Insertion).
However, the wording in the flowchart referencing the individual sections did not always match the section headings of the EOP support procedure.
For example:
(1)
The flowchart reference read " DRIVE RODS, BYPASSING RWM IF NECESSARi," whereas the support procedure was titled
" Manually insert control rods as follows" (2)
" RESET SCRAM, DEFEAT RPS LOGIC IF NECESSARY AND RE-SCRAM" was titled " Reset the scram, drain the scram discharge volume, and initiate a manual scram."
(3)
" VENT DRIVES OVERPISTON VOL." was titled " Vent Control Rod Drive over piston volumes."
29.100.01 Sheet 3.
Level Contincenclog
Step RC/L-4.1 listed subsystems in bullet form.
Two of the subsystems were required to be injecting.
One of the subsystems listed was "COND/FW (23.107) AND/OR SBFW (23.107.01)".
It was not clear which combination of these systems was required to count as one subsystem; for example (1) condensate, feedwater and SBFW; (2) condensate and feedwater; (3) condensate; (4) feedwater; or (5) SBFW.
l l
_ _ _... _ _ _ _ _. _ _. _ _ _ _ _ _ _ _ _ _ _ _ _
_ _ _ _ _. _.. _ _. _ _ _ _ _ _. _ _ _ _ _ _ _. _ _. _ _ _. _ _ _ _ _ _ _ _
_.. _
.
_
-.
.-....
, -... -
.
.__
-
.
.
o Step RC/L 5.3 referenced page 131 of the EOP support procedures for defeat of isolations.
The referenced page provided no useful information in that only a list of interlock defeats was provided.
The applicable isolations to be defeated with appropriate references to the EOP support procedures was provided later in the flowchart step.
29.100.01 Sheet 5.
C1 - RPV Floodina o
Step Cl-4 was not consistent with other steps, in that, it referenced an EOP support procedure without using parentheses for the page number reference.
29.100.02 Primary Containment Control o
The step below flowchart stop TW/L-4 did not reference the normal operating procedure 23.144, which was inconsistent with the text EOP.
o Text EOP step TW/L-5.3 referenced EOP support procedure
" Primary Containment Water Level Determination," page 133.
This reference was omitted in the flowchart EOP.
o Text EOP step DW/T referenced EOP support procedure
"Drywell Average Temperature Calculation."
This reference was omitted in the flowchart.
29.100.03 Secondary Containment and Rad Release Control o
Step SC/L-1 had a decision step in which the conditions would always be satisfied if the steps were followed literally.
The step specified that the water level be greater than maximum normal after operation of the pumps.
The previous step required that the' water level exceed maximum normal and that a pump be operated, thereby satisfying both conditions of the decision step.
29.100.04 Curves, Cautions, and Tables o
Caution 1 - Although the chart included a column for maximum run temperature for "DW RUNS," no useful information was provided in that all of the values were listed as N/A.
o-The chart for " HCL" did not have the right hand axis labeled (Torus water level).
--
,
a
-
. - _ _ - _ _ _.
.
.
29.000.01 Alternate Boron Iniection Section 1, Boron Iniection with the Standby Feedwater System o
Step 7.1 directed the operator to "open demin water to resin feed and precoat tanks header valve..."
However, the valve was labeled " DEMINERALIZED WATER SUPPLY VALVE" at the local control panel used to open the valve.
29.000.01 Alternate Boron Injection E_ection 2. Boron Iniection from RWCU usina the SLC Tank o
Step 6 required the operater to follow the " Backwash Section" of procedure 23.707, " Reactor Water Clean Up,"
until a hold pump stopped and valve G33-F153 opened for the second rinse cycle.
There was no indication in procedure 23.707 that the operator should exit the procedure at a particular point when executing the EOPs.
Consequently, an operator could (mistakenly) continue in the procedure.
-
o Step 7 required closing a " condensate fill valve."
However, the control for the valve was labeled "RWCU FILTER /DEMINS SUPPLY VALVE."
o Step 9 required opening a "Precoat Tank Drain Valve to Radwaste."
However, the valve was labeled "PRECOAT TANK DRAIN TO CHEMICAL WASTE TANK VALVE."
o According to the operator on the procedure walkthrough, step 20 would more appropriately be performed before step 18 to prevent dropping a hose down a crane access.
O Step 25 required that a valve be verified closed from the control room.
However, because no location for this step was given in the procedure, the operator on the procedure walkthrough in the plant did not immediately recognize that the location was the control room, o
Step 26.1 required holding two toggle switches located inside electrical cabinet G33-P001.
However, the procedure did not state that the switches were inside the cabinet.
Furthermore, there was no special EOP labeling inside the cabinet to identify the switches.
O Step 26.1 required holding two toggle switches down for an extended period of time.
The licensee should consider whether an operator can realistically hold the switches down for the required time and the consequences if a switch is momentarily released.
l
.
.
29,000.01 Alternate Control Rod Insertion o
Step 3 of venting the scram air header, required the operator to disconnect piping at C11-R013 (Scram Air Header Pressure Gauge located at RB1-G11).
The pressure gauge label description did not include the system number C11-R013.
Because of this omission, the operator on the procedure walkthrough needed to trace the system to verify the proper gauge before disconnecting the air line.
As a result, the operator was delayed in completing the task.
O Step 3 of venting the scram air header, required the operator to disconnect piping to C11-R013.
The operator could not positively identify the required wrench in the dedicated Alternate Control Rod Insertion tool box.
Also,
_
the Audit Form (inventory list for support procedure equipment) did not include a wrench for venting the scram air header.
As a result, the operator could have been delayed in completing the task until a wrench was obtained.
19.000.01 Interlock Defeats Section 4.
Defeat of al] MSIV and Main Steam Line Drain Valve Isolation Sionals o
Steps 2 through 5 in the procedure required either lifting leads or jumpering between terminals on terminal strips BB and CC, which were located deep inside electrical cabinets H11-P622 and H11-P623 in the relay room.
To reach these terminal strips, an operator would have to step inside the cabinets, potentially coming in contact with energized electrical circuits.
As such, performing these steps posed an electrocution hazard for the operator and a risk of unintentionally shorting out other circuits in the cabinet.
-
The procedure required installing jumpers at terminals BB-5 and AA-12 in electrical cabinet H11-P622, and at terminals BB-3 and AA-12 in electrical cabinet H11-P623.
These terminals each had two leads installed on the side of the terminal strip denoted by EOP label plates.
Consequently, adding another lead using jumpers would have been difficult to accomplish on the side of the terminal strip denoted by the label plates.
Jumpering from the terminals was possible because only one lead entered the terminal strip on the opposite side.
The licensee should consider additional labeling and/or a clarification in the procedure as to wnich side to jumper.
_ _ _ _ -
_
-....__.. -
-..
-. - - - -._
. -. - - -. - - - -
-
-
- - -. -
,
.
29.000.01 Interlock Defeats section 5, Defeat of RWCO Lgolations o
Steps 2.2 and 3.2 each required lifting a single lead from terminal strip point CC-24 in relay room electrical cabinets H11-P622 and H11-P623, respectively.
However, for both-cabinets, terminal strip point CC-24 had one lead entering on one side and two leads entering on the opposite side.
Although the special EOP label plate was on the side with a single lead, the operator questioned whether lifting the
,
single lead would be sufficient because an electrical connection would still exist on the other side.
The operator also questioned whether lifting all loads would be acceptable because he was not sure if an electrical connection was intended to be made.
The operator stated that he would have stopped and consulted the control room for further guidance, which could have caused a delay of several minutes.
29.000.01 Interlock Defeats Section 6.
Defeat of ARI Loaic Trips o
Steps 1.1 and 2.1 required lifting a single lead from terminals C-123 and F-123, in relay room electrical cabinets H11-P857 and H11-P870, respectively.
However, for both cabinets, the terminals had one lead entering on one. side-and two leado entering on the opposite side.
The inspectors
,
had the same concerns with these steps as with those in Section 5 of the Interlock Defeats procedure (see above).
o The terminal strips in panel H11-P857 were correctly labeled A,
B, C,
D, E,
and F, but with magic marker.
This was not a labeling method approved for use at Fermi.
29.000.01 Interlock Defeats Section 7.
Defeat of RPS Lqgjc Trips o
This procedure required jumpering between terminals on
.
different relays inside electrical cabinets H11-P609 and i
H11-P611.
However, the terminals were only identified by
"
black raised lettering on a black recessed surface on the relays, which was difficult to read.
Although-special EOP
- -
labeling identified-which terminals to be jumpered, the EOP labels were at different corners of the relay than where the terminals to be jumpered were located.
Consequently, the labeling was potentially misleading.
J
,..
,
--.. -,
,e,,w.
<.,.,,..r-,w_-...._y,,mm,.
,_mg
,w..
....
-,,_,,E.
..e...
,,.,.,.,m,
,,,.., -.... _,.,
.._,,,,____.m..__....-~,,c.
._.
,
+m,~,=
,,,
e--,-v
_
'.
.
O The procedure specified that jempers be installed in electrical cabinets H11-P609 and H11-P611 in the relay room.
However, each cabinet had three bays (east, center, and west) and the procedure did not specify in which bay the specific devices were located.
29.000.01 Interlock Defeats Section 9.
Defeat of Standby Feedwater Level 8 Trio o
The procedure required the operator to remove a plug-in relay.
The EOP label which identified this relay was located on a nearby terminal strip, rather than directly below the specified relay.
This was not consistent with other EOP labeling for relays.
29.000.01 Interlock Defeats Section 10. Defeat of RHR Shutdown Coolina Isolations o
The procedure required installing jumpers at terminals BB-16 and DD-6 in electrical cabinet H11-P622, and'at terminal DD-6 in electrical cabinet H11-P623.
These terminals each had two leads installed on the side of the terminal strip denoted by the EOP label plates, and one lead entering on the other side.
The inspectors had the same concerns with these steps as with those in Section 4 of the Interlock Defeats procedure (see above).
The licensee-should consider-additional labeling and/or a clarification in the procedure as to which side to jumper.
29.000.02 Primary Containment Ventina and Purae Section 4.
Low Pressure Containment Vent and Purae o
The valves referred to in step 6.8.2-were:
T41-F400, T41-401, T41-402, and T41-403, located in the Nitrogen Skid Enclosure.
These valves were labeled:
400, 401, 402, and 403, using magic marker.
This'was not a labeling method approved for use at Fermi.
29.000.02 Interlock Defeats L
Section 1.
Torus Water Manacement System Interlock Defeat o
Steps 2.a and 2.b required the operator to lift and separate.
the leads from F42.in relay room panel H11-P877.- There were l
four leads on these terminals and the labeling did not make L
it clear if all or some of-the leads were to be lifted.
l During the walkthrough of the procedure, the operator. stated
'
that. direction would be needed from the control room as to which leads needed to be lifted.
l l:
-
-
- - - -
..
.
-
.
,
,
,. -
..
-.
_
_
.
.
.
a
.
,
23.142 Auxiliary Boiler System o
Step S.2.13 referred to instrument tap P61-F293A(B)..
The label was missing.
J o
A ladder was not available on the second level for venting of the feed tank.
'
23.707 Reactor Water Clean Up o
Section 6.3.2, Step 11.a, required verification that the RWCU vessel had started depressurizing as indicated on Pressure Indicator G33-PI-R177.
However, the pressure indicator was labeled "RWCULF/D INFLUENT PRESS." without-an instrument number.
The step also misspelled the word depressurizing ("depressuring").
7-
_ _ _ _
.,
_
w y
w-a y
e
-,,
c
-
v 3-wr-
-
gi i-1-,---v
-"'
y T^
n't
- ' ' '
"
E
__
-
.-
.
- _ _.
-
... =.. _ _. -
.
.
.
Appendix C Detailed Comments from the Human Factors Review Writers Guide (WG)
o The WG provided guidance that did not optimize the use of'
symbolism and resulted in procedures that were excessively wordy and increased the clutter of the flowchart, o
The Fermi Writers Guide for plant procedures was comprised of a general section and 6 appendices (e.g.,' Appendix F was the EOP flowchart writers guide).
Each appendix contained a list of acronyms, abbreviations, and verbs which could be used in procedures, rather than having one source for this information.
Also, there were a few verbs used in the EOPs which were not listed in any of these appendices; e.g.,
reset, commence, keep, stabilize, manually, and maintain.
o Combining logical ANDS and ors in the same procedure step was not allowed.
However, in steps RC/P-7.1, RC/P-7.2, C2-2, and C2-4 of Procedure 29.100.01, and numerous other steps, logical F.NDs and ors were combined.
Guidance was.
provided on how to make the clauses distinctive, however, this guidance was not always clear.
Also, decision boxes were to be used to format steps containing both logical ANDS and ors, but no guidance was provided on how to. format a decision box.
O The WG guide specified using all uppercase letters for the text and using bolded letters for emphasis.
However, since all the text was in uppercase the bolding added little j'
emphasis.
In addition, the WG stated to use text of a size that was legible, but did not give a size; therefore, text size varied throughout the flowcharts.
,
o Guidance was lacking as to how to format. time-dependent steps.
Some operators had been trained to note times on the r
'
flowchart, however, this was not discussed in the WG, nor.
was it found that the operators consistently followed this practice.
An example of a time-dependent step was Step Cl-4 in procedure 29.100.01.
o The WG did not provide a mechanism for formatting notes in the flowcharts, but briefly discussed the-concept of-notes.
An example of a note was not provided-in the WG.,
In numerous cases in the flowcharts, supplemental information, which would-be better formatted as a note,-was presented in the step, thus increasing the1 number of words in the step.
!
_
. -
.. -..
.
.
.
_,
_
_
.
,
_
_._ _
-
. _. -
o s
Emeraency Operatina Procedures o
The flowcharts lacked effective use of symbolism, in that, the amount of verbiage in the procedures had not been significantly reduced (from the text EOPs).
It was not uncommon for a step to have 20 or more words, and in some cases over 50 words.
Also, efforts had not been made to simplify step wording, or to keep the wording concise.
For example, Step C1-1.3 of Procedure 29,100.01 read in part,
" START AND SLOWLY INCREASE RPV INJECTION WITH THE FOLLOWING UNTIL 2 1 SRV OPENS AND RPV PRESS >' MIN ALT RPV FLOODING
"
This PRESS, IRRESPECTIVE OF NPSH AND VORTEX LIMITS:
...
example demonstrated a number of the weaknesses found in the procedures in regard to step length, complexity of the step wording, unrestricted use of ambiguous terms, and imbedded logic.
Numerous other examples existed in the procedures.
An example of ineffective use of symbolism, was that in addition to providing symbols, the flowcharts generally contained the wcrds for the symbols.
For example, the word concurrently was always used when the symbol for concurrently was presented.
O The weakness deemed to be most significant in the procedures was the improper combining of logical AND and OR statements in the procedures.
These steps were found in all procedures reviewed.
In one flowchart, for example, six steps contained both logical " ANDS" and " ors" and the conjunctive usages of the words.
Although some efforts were made to make the clauses distinctive, they were not always effective.
Emphar.s techniques should have helped alleviate this problem, however, since all the letters in the flowcharts were uppercase and both logical " ANDS" and " ors"
-
and conjunctive " ands" and " ors" were in bold type, the emphasis was lost.
In several cases, steps were formatted with imbedded logic.
For example, the phrase "if necessary" was used in several procedure steps.
O The EOP steps were phrased in EPG language rather than language familiar to the operators.
For example, a procedure asked if "all control rods are s 00."
However, the indication for all rods being less than s 00 was momentary and the operators commonly used the phrase "ala rods are in."
Another example was step RC/L-2 (Procedure 29.100.01) which stated " RESTORE AND KEEP LEVEL 173 TO 214 IN. WITH 2 1 OF THE FOL)OWING:"
The operators commonly used
"any" rather'than
"2."
. - _ _ _ _ _ _ _ _ _ _ _.
\\.
o Step numbering was consistent with the EPGs rather then unique to tne flowcharts.
Therefore, not all substeps were numbered and the numbering scheme was cumbersome.
Although this was not a problem for the operators during the interviews and walkthroughs, it did appear that it could be a potential source of confusion in high stress events.
For example, an operator stated that on several occasions during training, operators would inadvertently transfer to step Cl-1.2 of flowchart "C1 - RPV Flooding" (Procedure 29.100.01)
instead of step C1-2 as directed by the.econd override.
o Procedure step formatting was not conci atent.
For example, the WG stated that the logical ifs, THENS, and WHENS, be kept separate from the text of the step, however, this was not always the case.
In some cases the words were not offset and were not distinctive from the stop text.
This could lead to confusion on the part of the operators if they expected to see the logic terms offset and the terms were imbedded in the text.
o In several instances, negatively phrased steps were used in the procedures which made the step much more difficult to understand.
Two examples were "NOT DECREASING" and " SCRAM CONDITION AND Rx PWR CANNOT BE DETERMINED TO BE < 3%"
o There were numerous examples in the flowcharts where redundant steps and unnecessary information were presented, due in most cases to the literal translation of the text procedures to the f' owcharts.
These extra steps could potentially delay the performance of important steps.
For example, it was not uncommon in the procedures for a step to specify some parameter to be maintained at a certain value, and then the next step to ask if the parameter can be maintained at this value.
As an example, Step TW/L 5.1 (Procedure 29.100.02) read " KEEP TORUS LEVEL < SRVTPLL," and the following decision step read "CAN TORUS LEVEL T'
KEPT <
SRVTPLL7" This method of formatting significantly increased the total number of steps in the procedure without benefitting the performance of the procedure.
As another example, a table containing the number of open SRVs for the Min Alt RPV FLOODING PRESS was located in three places on the flowchart "C1 - RPV Flooding," but was not in l
a forth location where it was also used.
The duplication of the same information on a flowchart created unnecessary
,
clutter.
One such table strategically placed was sufficient l
if it was properly referenced at the various places needed l
In the flowchart.
I l
The procedures used words and phrases such as " irrespective L
of..." and " restore and keep..." which increased the complexity of the procedure steps.
l l
..
..
_
_
._
~-
__
_
.. _ _
_
_.
\\.
,
o There was a lack of effective use of space on the flowcharts.
For example, the text in the procedures was in all uppercase letters with little white space around them.
In numerous cases the words were run up to the edge of text boxes and were difficult to read.
Different text size was used on several of the flowetarts.
,
l Step SC/R-4 (Procedure 29.100.03) was an example of not providing adequate white space.
This step was a hold step, which was a rectangular box with the corners shadad.
Part of the letter
"D" in " EXCEED" was obscured by one of the
,
shaded corners, and the lettering of the word had been compressed due to the word processing need to fit the text in the space provided.
On this particular flowchart this same condition appeared in six steps.
O The procedures had numerous transition points.
For example, Sheet 1 of Procedure 29.100.01 had eight separate transition points to other sections of the EOPs.
In addition, the procedure had numerous exit points to other procedures, and references to support procedures and enclosures.
There were also numerous transition points within the proceduras.
Since simulator exercises were not observed, it is difficult to judge whether operators would have difficulty transitioning between the flowcharts.
In several cases, flowpaths were continued onto a second flowchart.
For example, the RC/Q flowpath in RPV Control was continued on a second sheet.
In this case, consideration should be given to breaking this leg out into an ATWS leg and a non-ATWS leg, each on separate flowchart.
This would minimize the number of sheets the SRO would have to layout in an ATWS condition.
O The procedures used numerous terms with the same meaning.
An example of this was the use of the words " keep" and
'
" maintain."
o Though the use of the words increase and decrease was strictly prohibited by the WG, these words appeared in numerous flowchart steps, including C1-1.3, Cl-3, and C2-3 of Procedure 29.100.01.
In addition, according to the WG, the use of words such as slowly and rapidly was to be avoided.
However, these words were used in several steps including steps C2-3.2 and C1-1.3 of Procedure 29.100.01.
Although these words were undefined in the WG, the SRos knew what the words meant in-the context of the procedures.
O Supplementary information was commonly contained in procedure steps rather than formatted as notes.
This contributed to the excessive wordiness in the steps.
..
-
.
.
l z
s
.
o All the curves, cautions, and tables appeared on a single chart rather than the individual flowcharts to which they applied.
This resulted in an additional chart which needed to be handled when the flowcharts were used.
In the event the Safety Parameter Display System (SPDS) is lost during an emergency, the operators would need to rely of the hardcopy chart for this information.
Jnterviews-j i
o Interviews with the SROs verified that they were well
'
trained on the procedures, and understood the meaning and-purpose of the procedure steps.
The SROs could work through the procedures and transition within the flowcharts and to the support procedures.
O Comments from the SROs indicated that they would like to see more symbolism used in the procedures.
For example, a
"WHEN" or hold point on a flowchart was drawn as a-box-with shaded corners.
When asked what.the box meant, an SRO stated that it was a stop sign.
When the operator was then asked what he wotld like to see to represent a hold he said a stop sign.
o A number of SROs interviewed stated they would like to see the EOP curves placed on the individual flowcharts, as well as on the curves,-cautions, and tables chart.
One operator stated that the names of the curves were confusing because the name of the curve did not always map with its use.
-
o Overall, the operators accepted the flowcharts. ;However, there was not an overall enthusiasm about the flowcharts.
_
At the extremes, one SRO' stated he really liked the flowcharts and did not want to use anything else, and another SRO stated he did not like the flowcharts at all and would like to use the text procedures.
o
.The operators felt that if the total number of flowcharts could be reduced it would be' beneficial.
The SROs stated
.that it was difficult to have so many flowcharts and, at certain times, an operator could be using as many as seven.
- According'to the licensee, a table.would be cleared in'the-control room in an event and this space would be' utilized, but there was not a dedicated space for flowchart use.
o The ROs interviewed appeared not as-well trained nor as familiar with the procedures.
When Ros read through the procedure steps containing both logical " ANDS" and " ors" they had difficulty immediately grasping the intent of the step.
.
.- -_- _
-.
.
.
O One concern noted during the interviews was that no two operators explained the process by which operator comments on the procedures were handled the same, o
Training personnel interviewed discussed how training was conducted on the procedures.
A point made by the training staff was that they did not want to see changes in the procedures because additional training would be required.
O It was apparent from interviews with those involved in the procedure development process that only two individuals were involved in developing the Writers Guide and the flowcharts.
One individual served as the writer of both the WG and the procedures and one served as the reviewer.
Neither had
broad experience nor training in human factors, although, one individual had limited training in human factors principles.
The same two individuals also had major input into the V&V of the procedures.
These two individuals were no longer involved in the procedure development process.
O Two of the current procedure development personnel were interviewed.
One individual was fairly new to the job and had few insights into the process.
The other individual had been very involved in the recent V&V of the support procedures and appeared quite competent to perform the written correctness review of the procedures.
_
.
.
.
.
_ _ _ _
--
.
j,. -,.
Annendix D Linf. of Acronyms Used in the Insoection Report ARI Alternate Rod Insertion ATWS Anticipated Transient Without Scram BIT'
Boron Injection Temperature BWROG Boiling Water Reactor Owners Group CRD Control Rod Drive CST Condensate Storage Tank DECO-Detroit Edison Company EOP Emergency Operating Procedure EPG Emergency Procedure Guideline HPCI High Pressure Coolant Injection MSIV Main Steam Isolation Valve
<
NLO Non-Licensed Operator NPSH Net Positive Suction Head PGP Procedures Generation Package PSTG Plant Specific Technical Guidelines PWR Power QA Quality Assurance RCIC Reactor Core Isolation Cooling RHR Residual Heat Removal RO Reactor Operator RPS Reactor Protection System RPV Reactor Pressure Vessel RWCU Reactor Water Cleanup RWM Rod Worth Minimizer SLC Standby Liquid Control SPDS Safety Parameter Display System SRO Senior Reactor Operator SRV Safety Relief Valve SRVTPLL SRV Tail Pipe Level Limit STM Steam TMI Three Mile Island V&V Verification and Validation WG Writers Guide
.
, - -
,
_
.
._
__ __,