ML20129A247

Evaluation of Dcrdr Summary Rept for Maine Yankee Atomic Power Station, Technical Evaluation Rept

ML20129A247
Person / Time
Site:	Maine Yankee
Issue date:	05/08/1985
From:	SCIENCE APPLICATIONS INTERNATIONAL CORP. (FORMERLY
To:	NRC
Shared Package
ML20126M781	List: ML20126M778 ML20129A247
References
CON-NRC-03-82-096, CON-NRC-3-82-96 SAIC-85-1060, NUDOCS 8506040512
Download: ML20129A247 (24)

Text

.. - - - -

DCIOSURE 2 SAIC-85/1060 EVALUATION OF THE DETAILED CONTROL ROOM DESIGN REVIEW

SUMMARY

REPORT ,

FOR MAINE YANKEE ATOMIC POWER STATION May 8, 1985 Prepared for U.S. Nuclear Regulatory Commission

. Washington. 0.C. 20555 Contract NRC-03-82-096 ,

REFERENCES

1. Letter from J. H. Garrity (MYAPCo) to R. A. Clark (NRC) dated August IP, 1982,.with attachment, " Program Plan Report for the Detailed Control Room Design Review."

2. "NUREG-0737, Supplement 1 - DCRDR -Program Plan Letter" from J. R. Miller (NRC) to J. H. Garrity (MYAPCo), dated October 4,1983.

3. NUREG-0737, Supplement 1, " Requirements for Emergency Response Capability," U.S. NRC, Washington, D.C., December 1982, transmitted to reactor licensees via Generic Letter 82-33, December 17, 1982.

4. Letter from G. D. Whittier (MYAPCo) to J. R. Miller (NRC) dated February 28, 1985, with attachment, "A Human Factors Engineering Critical Assessment of the Design of the Control Room at Maine Yankee Atomic Power Company."

5. NUREG-0660, Vol. 1, "NRC Action Plan Developed as a Result of the TMI-2 Accident," May 1980; revision 1, August 1980.

6. NUREG-0700, " Guidelines for Control Room Design Reviews,"

September 1981.

7. NUREG-0800, " Standard Review Plan," section 18.1, " Control Room," and Appendix A " Evaluation Criteria for Detailed Control Room Design Reviews

. (DCRDR)," September 1984.

1 I

9-d However, eight requirements of NUREG-0737, Supplement 1, Section 5 could not t

be properly evaluated because of insufficient infomation. The staff recommends that these open items be resolved by requesting supplemental information from the licensee and, possibly, perfoming an on-site audit.

4 a

- , , v- -- e,, ,- m , ~- --,wy-y r-v -,w-

w-5.. discussion addressing the concerns raised in the enclosed TER regarding the assessment process, e.g., how was the cut-off point for safety significance estab'. Shed?

6. further description of the actual processes used to select design improvements;

)

, 7. confirmation that the process for verifying the adequacy of l corrections, as described at the in-progress audit, has been implemented;

8. further discussion regarding the coordination of initiatives, I

especially emergency procedure upgrading, SPOS, and DCRDR.

l The staff also reconnends that a meeting be held with representatives from MYAPCo in early July,1985 so that the staff's information needs are clearly g understcod. In addition, based on the staff's present knowledge of the task analysis effort at Maine Yankee, it appears that a Pre-Implementation Audit may be needed at some later date to confirm that the task analysis has been adequately performed. This matter will be discussed at the meeting and, if necessary, a date scheduled._ A final item for discussion at the meeting will' be a clarification of the implementation schedule for control room upgrades.

In conclusion, the licensee provided sufficient evidence for the staff to confirm that an acceptable control room survey 1.as been done at Maine Yankee.-

r - , e - r -r.

~

> tasks required for emergency operations to list information and controls required to perform these tasks, and to identify the characteristics of the i needed instruments and controls.

1 As described in the Summary Report, the ' analysis was accomplished by table top discussions between multidisciplinary team members to identify the intent or functional objectives of the existirq emergency procedures (p.

20). The functional objectives were used to verify. the adequacy of the individual steps of the procedures. Functional objectives were then used to generate task objectives, task descriptions, and finally the physical tasks needed to be accomplished. The physical task list was sufficiently detailed so that a. list of required information and controls and applicable charac-i teristics could be generated (p. 21). The result of the task was a list of information and control characteristics which should exist in the contrci room.. -

j No additional information is provided in the Summary Report as to how function and task analyses actually were conducted. No data analysis or collection forms were provided for review. The licensee has described a

procedure for verification of Emergency Operating Procedures (EOPs) steps but has not indicated how analyses were conducted to establish these steps.

There is no evidence of a procedure for the identification of systems /

subsystems and their associated functions. Furthermore, there is no evi-dence of an analytical process being implemented to determine operator tasks or information and controls and their required characteristics necessary for

, the operator to fulfill successfully his assigned tasks. The requirement of i Supplement 1 to NUREG-0737 dictates that these classes of information must be established independent of the control room, prior to the development of i

E0Ps. The methodological approach and the depth of the analyses are diffi-

! cult to assess with the limited information provided.

It appears that 'the basis of the analyses conducted was the set of

symptomatic emergency operating procedures developed by Maine Yankee in l response to the requirement of Supplement I to NUREG-0737. The licensee has failed, however, to discuss the basis for and development of its E0Ps.

Based on a memo (Reference 10) from the NRC to CEOG, the licensee's owners group, the CE EPGs do identify basic parameters and control functibns needed to satisfy safety functions. The EPGs did not explicitly identify the 9

l

~

l behavioral elements, and to provide an opportunity to question operators on information and control requirements beyond those implied or prescribed in I the E0Ps. The procedures which were walked through did not include all the operator-panel interfaces involved in emergency operations.

Because the " task analysis" was based on existing E0Ps, the NRC audit

(

! team concluded that a system function analysis was not adequately conducted.

The E0Ps, as written, did not provide a basis for the function analysis t i

necessary to define the associated operator tasks. The audit team further concluded that an a priori, independent analysis of operator tasks was not thoroughly conducted and that Maine Yankee had performed a verification of I task performance capabilities and validation of control room functions l rather than a task analysis which should be completed, independent of the )

control room, prior to validation. I

~ l i

In the Summary Report, the licensee provided a written description of '

its walk-through activity used to " validate control room functions." The validation effort was conducted to analyze selected operating scenarios (procedures). This provided a method to determine whether the control room operator could implement the procedures with the existing instrumentation, controls, and control room design. Eleven E0Ps were said to be used in the validation effort (page 16 of the Summary Report), but it is not clear how many procedures were analyzed. This uncertainty is due to the fact that at the time of the in-progress audit Maine Yankee indicated only seven pro-cedures were analyzed. On page 27 of the Summary Report it is indicated that nine procedures were used by the Maine Yankee Review Team and on page 28 of the Summary Report it is indicated that 15 procedures were analyzed.

In any case, the full spectrum of operator tasks required for emergency operation was not analyzed. Consistent with audit team findings, this activity, as described, appears to have provided a means to validate a subset of emergency operating procedures and to verify that controls and displays in the control room will allow operators to execute selected E0P tasks. The walkthroughs did'not establish, a priori, the information and control requirements against which the existing control room instrumentation should be compared.

The licensee also briefly described an activity entitled, "Co'ntrol Room l Operator Task Analysis" (p. 20), which was conducted to identify operator 7

~

reduce operator effectiveness. To facilitate communication, interviews were conducted by an independent consultant who was familiarized with plant design and terminology although he had little expertise in the area of human factors. The actual survey questionnaire used has not been provided for review. No information has been provided on analyses performed on data )

collected or on how information acquired was actually utilized by the I licensee in its assessment. The degree to which industry-wide reports were reviewed and documented is not clear. Furthermore, it is unclear whether plant documents for the last five years were reviewed to determine how an operator may have erred because the procedures, instruments, and controls did not comply with accepted human factors engineering principles. Such information and examples of checklists or questionnaires employed for data collection would provide greater confidence in the review of operating experience performed by the licensee.

2. System Function Review and Task Analysis Supplement 1 to NUREG-0737 states that the licensee is required to perform a " function and task analysis (that had been used as the basis for developing emergency operating procedures) to identify control room operator tasks and information and control requirements during emergency operations."

In other words, the objective of the task analysis is to establish the input and output requirements of control room cperator tasks. These information requirements are then to serve as benchmarks for examination of the adequacy of control room instrumentation, controls, and other equipment.

Maine Yankee conducted its system function and task analysis by:

(1) defining the intent or functional objective of existing emergency

, procedures, generating task oojectives, and defining physical tasks needed to be accomplished; and (2) conducting walk-through/ talk-throughs to deter-

. mine whether the control room operator could implement the procedures with the existing instrumentation, controls, and control room design.

At the in-progress audit. Maine Yankee described its task analysis activity which was based on symptomatic emergency operating procedures (EOPs) that had been developed in response to the requirements of Supplement I to NUREG-0737. A subset of these E0Ps were walked through in order to i observe operators' performance of tasks, to generate comments regarding 4

6

REVIEW PHASE Maine Yankee's review phase activities included:

1. Review of operating experience,

2. Control room operator task analysis,

3. Control room inventory,

4. Control room survey, and

5. Validation of control room functions.

For the most part, the above activities are those recommended by NUREG-0700 - l guidelines as contributing to the review phase objectives. Activities 2, 3, i and 4 contribute to the accomplishment of specific DCRDR requirements  !

contained in Supplement I to NUREG-0737 Activities 1 and 5 are discussed l in this evaluation only to the extent that they impact or augment those tasks required by NUREG-0737, Supplement 1.

l

1. Review of Operating Experience A review of operating experience is not explicitly required by NUREG-0737, Supplement 1. However, it is an activity recommended by NUREG-0700 guidelines as contributing to the accomplishment of review phase activities.

As described by the licensee, its review of operating experience included: (1) a review of Maine Yankee License Event Reports and Plant Information Reports; (2) review of the operating history of other power plants; and (3) conduct of interviews with control room operators to identify specific aspects of the control room which have reduced or could reduce operator effectiveness.

It was difficult to assess the adequacy of Maine Yankee's operating experience review due to lack of detail concerning procedures employed.

Twelve operators were interviewed on an individual basis to gather informa-tion on specific aspects of the contro' om which have reduced or could 5

senior Human Factors Specialist, Professor John Senders of the University of Maine. It is evident that Maine Yankee is attempting to instigate a long-term commitment to the integration of human factors to the plant operations by establishing an in-house awareness of human factors principles. The DCRDR team was organized with representatives from several levels of manage-ment.

The Steering Committee of the DCRDR team included representatives from operator training, control room simulation, and engineering. Human factors and operations management expertise were also included. The Steering ^

Committee was responsible for providing guidance to the Working Group, reviewing and analyzing data outputs, and developing the assessment criteria for HEDs.

The Working Group consisted of a core of two engineers, the human .

factors consultant, a human factors engineer, and various engineering assis-tants. Other support staff were available on an as-needed basis.

Other than the information above, the Summary Report provides little new information about the DCRDR team or qualifications of team members. It was pointed out in the review of the Program Plan and at the in-progress audit that Maine Yankee should increase the level of participation by a human factors specialist in conduct of the DCRDR. Nuclear systems engineer-ing and training expertise should also be increased. Based on information '

provided in the Summary Report, the degree to which the licensee adopted these recommendations is not clear.

In summary, it appears that Maine Yankee has organized a DCRDR team l composed of qualified personnel. However, the adequacy of the team and DCRDR task accomplishment cannot be fully evaluated until Maine Yankee I provides information relevant to the level of effort expended by specific team members. At a minimum, the licensee should provide information to show team members, their relevant areas of expertise, and degree of participation in the accomplishment of each of the DCRDR tasks.

l

Report (SER) indicating the acceptability of the DCRDR (not just the Summary Report). The NRC's evaluation encompasses all documentation as well as briefirgs, discussions, and audits if any were conducted.

PLANMING PHASE l

Maine Yankee's planning phase activities included: (1) preparation and l submission of a Program Plan, and (2) establishment of a review team.

1. Preparation and Submission of a Program Plan The Program Plan submitted for Maine Yankee did not fulfill the requirements of Supplement 1 to NUREG-0737. Although many of the elements of a review were addressed, the Maine Yankee Program Plan only provided brief descriptions of what would be accomplished. Specific areas of the -

work were not described in sufficient detail to provide assurance that the licensee understood the processes necessary to complete the tasks and therefore meet the requirements. The results of the evaluation of the Maine Yankee Program Plan are detailed in Reference 2.  !

\

{

\

l

2. Establishment of a Qualified Multidisciplinary Review Team The licensee's Program Plan included a brief description of the staff and management organization that was established to conduct the control room ,

design review. From additional information provided by the licensee at the audit and in the Summary Report, it appears that the structure and manage-ment of the DCRDR are flexible enough to permit a multidisciplinary effort.

l Management seemed to be actively interested and involved. Management of the DCRDR was the responsibility of the chairman of a steering committee who I

supervised the activities of a working group, reported progress to Maine Yankee management concerning activities of the review team, managed resources allocated to the design review, and coordinated review team per-sonnel assignments.

l l The Maine Yankee DCRDR team itself consisted of two groups: a Steer-ing Committee, and a Working Group. The NRC audit team concluded that most members had suitable qualifications. Some team members attended a formal human factors engineering course at the Maine Yankee Plant conducted by a 3

• I

2. .Use of function and task analyses to identify control room operator tasks and information and control requirements during emergency operations; l 3. A comparison of display and control requirements with a control room inventory;

f

4. A control room survey to identify deviations from accepted human  ;

factors principles;

5. Assessment of human engineering discrepancies (HEDs) to determine which HEDs are significant and should be corrected;

6. Selection of design i mprovements that will correct these discrepancies; -

7. Verification that selected design improvements will provide the i necessary correction;

8. Verification that improvements can be introduced in the control room without creating any unacceptable human engineering

. discrepancies; and

9. Coordination of control room improvements with changes resulting -

from other improvement programs such as SPDS, operator training.

E new instrumentation (Reg. Guide 1.97, Rev. 2), and upgraded emergency operating procedures.

A DCRDR is to be conducted according to the licensee's own Program Plan (which must be submitted to the NRC). According to NUREG-0700, it should address the previously stated requirements and be conducted in accordance with the following four phases: (1) planning, (2) review. (3) assessment, and(4) reporting. The product of the last phase is a Summary Report which i

must include an outline of proposed control room char.ges, their proposed schedules for implementation, and summary justification of human engineering i discrepancies with safety significance to be left uncorrected or partially j

corrected.. Upon receipt of the licensee's Summary Report and prior .to l implementation of proposed changes, the NRC must prepare a Safety Evaluation l 2

.,_.,~r . ., - . - - , . - - , . , - , - . , , , - n n. .,---w - ., . , , - .

f l

~

EVALUATION OF THE DETAILED CONTROL ROOM DESIGN REVIEW SUM 8.ARY REPORT FOR MAINE YANKEE ATOMIC POWER STATION This report documents the Science Applications International Corpora-tion (SAIC) evaluation of the Summary Report of the Detailed Control Room Design Review (DCRDR) submitted to the Nuclear Regulatory Commission (NRC) ,

by Maine Yankee Atomic Power Company (MYAPCo) for Maine Yankee Atomic Power -

Station on February 28,1985 (Reference 4). This evaluation also considers

-information obtained from the previously submitted Program Plan (Reference 1). Further information regarding DCRDR activities was acquired at an in-progress audit conducted at the plant on February 13-17,1984 (Reference 3).

Results of the SAIC evaluation follow a brief overview of the -

background leading up to preparation and submission of the Summary Report by the licensee.

BACKGROUND Licensees and applicants for operating ifcenses are required to conduct a Detailed Control. Room Design Review. The objective of the review is to

"... improve the ability of nuclear power plant control room operators to prevent accidents or cope with accidents if they occur by improving the information provided to them" NUREG-0660. Item I.D.1 (Reference 5). The need to conduct a DCRDR was confirmed in NUREG-0737 (Reference 6), and the requirements to be met in such a review were contained in Supplement I to NUREG-0737 (Reference 7). Guidelines for conducting a DCRDR are provided in

~

NUREG-0700 '(Reference 8), while NUREG-0800 (Reference 9) presents the assessment processes for use by the NRC.

The DCRDR requirements as stated in Supplement 1 to NUREG-0737 can be ,.

summarized in terms of nine specific issues, a list of which provides a convenient outline of the areas covered in this technical evaluation. The nine issues include: ,

1. - Establishment of a qualified multidisciplinary review te'am; 3

1 4

i

.-- , , - - - - - , - ~ ~ - . - , - , - - . . . - . . ,- -., ,

TABLE OF CONTENTS Section Page BACKGROUND . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 PLANNING PHASE . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

1. Preparation and Submission of a Program Plan . . . . . . . 3

2. ' Qualifications and Structure of the DCRDR Team . . . . . . 3 REVIEW PHASE . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1. Operating Experience Review ............... 5

2. System Function Review and Task Analysis . . . . . . . . . 6

3. Comparison.of Display and Control Requirements With a Control Room Inventory . . . . . . . . . . . . . . . . . . 9

4. Control Room Survey ................... 10 ASSESSMENT AND IMPLEMENTATION PHASE ................ 10

1. Assessment o f HEDs . . . . . . . . . . . . . . . . . . . . 11

2. Selection of Design Improvements . . . . . . . . . . . . . 13

3. Proposed Schedules for Implementing HED Corrections ... 14

4. Verification That Improvements Will Provide the Necessary Corrections Without Introducing New HEDs . . . . . . . . . 14

5. Coordination of the DCRDR With Other Improvement Programs. 15 CONCLUSIONS AND RECOMMENDATIONS .................. 15 REFERENCES . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 i

J e

ii

FOREWORD l

This Technical Evaluation Report (TER) was prepared by Science I Applications International Corporation (SAIC) under Contract NRC-03-82-096, Technical Assistance in Support of NRC Licensing Actions: Program III. The evaluation was performed in support of the Division of Human Factors Safety.

Human Factors Engineering Branch (HFEB). SAIC previously evaluated Maine Yankee Atomic Power Company's Program Plan submitted for the Detailed Control Room Design Review (DCRDR) conducted for Maine Yankee Atomic Power Station (Reference 1). Results of that evaluation are described in a memorandum prepared by HFEB and transraitted to the licensee (Reference 2).

Subsequent to issuance of the memorandum, an in-progress audit was conducted at the plant (Reference 3).

This report includes the SAIC evaluation of the licensee's Summary Report (Reference 4) and considers DCRDR activity information presented in the Program Plan and findings provided at the audit.

o 4

i

REFERENCES

1. " Program Plan Report for the Detailed Control Design Review " Maine Yankee Atomic Power Company, April 12, 1982.

2. Memo from William Russell (NRC) to Gus Lainas (NRC), NRC Comments on the Maine Yankee Power Company Detailed Control Room Design Review Program Plan Submittal . September 26, 1983.

3. Memo from Voss Moore (NRC) to James Miller, NRC, Results of In-progress audit of Maine Yankee Detailed Control Room Design Review, May 17, 1984

4. "A Human Factors Engineering Critical Assessment of the Design of the

Control Room at Maine Yankee Atomic Power Company," Maine Yankee Atomic Power Company, February 28, 1985.

5. NUREG-0660 Vol.1 "NRC Action Plan Developed as a Result of the TMI-2

) Accident," U.S. Nuclear Regulatory Commission, May 1980; Revision 1 August 1980.

l 6. NUREG-0737, " Clarification of TMI Action Plan Requirements," U.S.

g Nuclear Regulatory Commission, November 1980.

7. NUREG-0737, Supplement 1. " Clarification of TMI Action Plan Require-ments," U.S. Nuclear Regulatory Commission. December 1982.

8. NUREG-0700 " Guidelines for Control Room Design Review " U.S. Nuclear i Regulatory Comission, September 1981.

, 9. NUREG-0800, " Evaluation Criteria for Detailed Control Room Design Review," U.S. Nuclear Regulatory Commission October 1981.

10. Memo from H. Clayton (NRC) to Dennis Ziemann (NRC), Meeting summary -

l

~ task analysis requirements of Supplement I to NUREG-0737, August 29, 1984 meeting with Combustion Engineering Group (CEOG) Operations

, Subcommittee, September 7, 1984.

I 1

i 1

18

o A description of the human factors evaluations which will he an integral part of the verification that improvements will provide the necessary corrections without introducing new HEDs.

e Coordination with other improvement prograss, o A prioritized list of HEDs identified during the review, o The proposed design changes to evaluate against provided schedules.

e Suasary justifications for HEDs with safety significance to be left g uncorrected or partially uncorrected.

i i.

• e >

~

! t O

I e

+

L i

! 17

limited discussion of most activities has prevented a complete assessment of the licensee's efforts or its understanding of the requirements. Areas which should be expanded include the following: (1) qualifications of the team; (2) conduct of a function and task analysis; (3) assessment of HEDs; j (4) selection of design improvements; (5) verification that improvements -l will correct HEDs without introducing new HEDs; and (7) coordination of HED l resolution implementation.

Due to the brevity of discussion in the Summary Report and the lack of information regarding identified HEDs, proposed corrective actions, and -

justifications for HEDs left uncorrected, we conclude that a more definitive presentation is necessary to establish the degree to which the requirements

, of Supplement 1 to NUREG-0737 have been met. Therefore, we recommend that a i

pre-implementation audit be conducted to clarify the points raised in this evaluation report and to provide Maine Yankee with additional feedback. The -

concerns raised as a result of this evaluation are summarized below and h should be addressed by the licensee during the audit and discussed in detail in a supplementary Summary Report.

e Identification of task assignments and levels of effort for DCRDR

> team members.

e Identification of the scope and depth of the System Function Review and Task Analysis including discussion of the bases of E0Ps and '

assurance that all tasks involved in emergency operations were j analyzed. A description of the process used to identify plant-specific information and control needs and to establish the charac-teristics required of needed instruments and controls. A description of the auditable record that contains the data gene-rated from the function and task analysis, e A more detailed description of the HED assessment process.

o A more detailed description of the process used to select design improvements and to ensure the integration of design modifica-tions/ changes.

16 l

O t (1) Testing the proposed change on the Control Room mockup using a

" cut and paste" technique; (2) Testing the proposed change on the Maine Yankee simulator; and (3) Installing the final change on the Control Room control board.

During each of the steps a human factors evaluation would occur prior to proceeding. Those items of a nonhardware nature (e.g., procedures, train-ing) would be laplemented and then apparently would not be reviewed further.

This process appeared adequate to the audit team. However, it was ,

recommended that those HEDs corrected by nonhardware improvements be subject to periodic verification to ensure the adequacy of the improvement. ,

! As there is no description in the Summary Report, it is unclear whether recommendations of the audit team were heeded. The licensee should provide documentation to show how Maine Yankee implemented these human factor evaluations.- Until further information is provided, this task cannot be fully evaluated.

5.- Coordination of the DCRDR with Other Programs 1

Although coordination activities were not addressed in detail in either of the licensee's submittals Maine Yankee provided the audit team with information relating to the current status .of the SPDS, Reg. Guide 1.97 '

instrumentation, and other improvement programs.. The Summary Report does i

indicate that all changes resulting from DCRDR activities are sent to the steering committee of the Energency Response Capabilities Integration Program, a larger program which coordinates MUREG-D737. Supplement 1 initia-tives. This arrangement may be entirely satisfactory. but greater confi-

] dance in its efficacy would have been fostered if details regarding the ,

mechanics and scheduling of the coordination activities had been provided.

CSICLUS!MS AND RECWWENDATIGIS Maine Yankee's' Summary Report for the DCRDR demonstrates an attempt to

. seet many of the requirements of Supplement I to MUREG-0737. However, 15

There is no information provided on the processes that were used to  ;

examine various alternative solutions, their integrated effects on operator performance, and the arrival at a final solution. Therefore, it is difficult to determine whether the process to select design improvements was conducted with full consideration of alternative solutions that would provide the optinua human factors design. Furthermore, it is unclear wnich members of the DCRDR team were involved in this activity and whether both CPA & PTL improvement recommendations were sent to Maine Yankee for final technical review or as a matter of company policy.

The lack of information regarding how design improvements were selected and which HEDs have been designated for improvement renders this section of the Summary Report impossible to evaluate. .

i

3. Proposed Schedules for Implementing HED Corrections -

i Sections 8 and C of the main control board (MCB) are scheduled to be  !

upgraded during Maine Yankee's 1985 refueling outage. These sections are the primary operating sections of the control boards and, as such, contain i most of the safety-related displays and controls. Upgrading these sections i

will alleviate the highest priority HEDs.

4 Section A of the MCS and the Electric Board will be upgraded and their associated HEDs resc1ved during the 1987 refueling outage.

Any remaining HEDs and the annunciator panels will be addressed during the 1988 refueling outage.

4 Due to the fact that the Suasary Report does not identify or prioritize i' HEDs, this schedule cannot be evaluated.

t

4. Verification That Improvements Will Provide the Necessary Corrections Without Introducing New HEDs >

This requirement was not discussed in either the licensee's Program i Plan or Summary Report per se. However, at the audit, the licensee indicated that verification of correction effectiveness was to be' conducted for hardware' changes in a three-step fashion including: i i

14 i

e

,, - - , - - , , - , , , --y-, - , < , . , , , . - - . , g,g,.,-, , . - - ~ - - - - - - - - - - , . --n-- ,- -..-- , , - - - ,, ,, -

not possible to make an informed evaluation of Maine Yankee's assessment methodology.

2. Selection of Design Improvements As described in the Summary Report, each identified HED was placed into one of 17 categories (e.g., labels, procedural, enhancements, scaling, etc.). Each category was analyzed by the Steering Committee and, if neces-sary, subgroupings in categories were generated. An initial review was performed to determine if similar problems (subgroups) were assessed to the same criterion value. Those groupings or subgroupings for which the

, majority of HED criterion values exceeded the assessment threshold were I recommended for corrections; the others were not.

t

[' ,

An additional screening of HEDs not recommended for resolution was -

t conducted by the Steering Committee to ensure that those HEDs with safety E

significance were correctly taken into account. This is a commendable l

procedure.

Recommendations for design improvements (HED corrections) were treated

in one of two ways. Those requiring design changes, in accordance with the Maine Yankee Operational Quality Assurance Manual, were submitted to Maine Yankee management via the licensee's standard design change procedure (CPA)

[ for approval. Changes (paint, tape, label) which did not require a plant .

design change were submitted to management in memoranda fer approval. All e

changes were sent to the Emergency Response Capabilities Integration Program (ECRIP) steering committee for integration with remaining NUREG-0737, '

! i Supplement 1 programs.

1 Although it appears that the selection of design improvements was an integral part of the DCRDR performed at Maine Yankee, the licensee's l submittal provides limited information describing the actual processes that were used to select improvements for identified HEDs. Although those groups for which the majority of HEDs exceeded the criterion value were recommended for correction, there is no identification of criterion value and no identi-fication of HEDs that are above or below this value.

13

l .

were described only briefly in -the Summary Report. The methodology relied

-on questionnaires, interviews, and observational data obtained from opera-l tors during previous DCRDR activities. Data from these sources were used to compute a safety and a frequency factor for each HED. A probability of

, error estimate was also determined for each HED by the human factors engi-4 neering consultant and Steering Committee members. Values for these tnree factors were normalized and combined to produce a single assessment score l m which was to indicate the HED's relative need for corrective action. l l l l

The " nature" of the HED was said to have been the basis for determining

a cutoff score. All HEDs with' assessment scores above the cutoff were j

scheduled for corrective action as were all HEDs "which were viewed to have a high consequence." Presumably a high consequence means a high value of the safety factor of an HED's assessment score, regardless of the values for

, l the other factors.

i ,

There are a number of problems associated with the HED assessment l methodology as described in the Summary Report. There is no discussion of 4

how or by whom the cutoff score was determined. It is not clear how the

" nature" of the HEDs was used to determine a cutoff score. Furthermore, how

. is " nature" of the HEDs defined? There is no indication that those HEDs j which previously contributed to crew error were given a high priority.

Some of the data used to compute the frequency factor of the assessment '

' scores were obtained from operator interviews. There is no indication,

{ however, from which interviews these . data were obtained. Still other data were gathered during the walk-throughs of an incomplete set of emergency

! I operating procedures. This raises the question of the completeness of the

] data available for these computations. Finally, the use of some data in '

I computing factors for the assessment scores it based on assumptions that may j or may not be correct (e.g., frequency of use of instruments with a given l HED is highly correlated with the operator's awareness of the HED). l i l In summary. of the 856 HEDs which were identified at Maine Yankee, 221

were deemed in need of corrective action. None of the HEDs and their priorities were listed or described in the Summary Report. Without this 1

information and further description of the HED assessment methodology, it is l

1

12 i

i

,n,,.. , . - , - -.,..-,,<,,-,nn... n,.. n - - - - - - - - - - , , - - - , , - - - - - , - , - . . , , . - - ,.--.~.-----..n-- - ---,,------,~-,-,-,.,n..,

HED assessment and resolution while Section 8 provides schedules for imple-menting HED resolutions.

1. Assessment of HEDs In describing the guidelines the NRC will use to evaluate a plant's assessment of HEDs. NUREG-0800 indicates that the ultimate criterion will be whether the plant assessed the HEDs in terms of their effects on the operator's ability to perform the necessary tasks and resulting consequences of an error on plant safety. These criteria are embodied in NUREG-0800's paragraph 2.5 which states: "To evaluate the significance of design dis-crepancies, the DCRDR team must determine the effect of each HED on operator performance, both alone and in combination with other HEDs;" and "Correc-tive action of each HED should be based on its significance as it affects the safety of the plant."

NUREG-0800 specifies the criteria NRC will use to evaluate the plant's assessment of HEDs. The guidelines indicate that the following should be considered:

1. The relative degree of degradation of operator performance;

2. The effect on plant safety;

3. The possible interactions of HEDs; and

4. Whether HEDs have resulted in previous crew errors.

The fourth guideline is amplified in a later sentence of the same paragraph.

It states that all HEDs that have caused crew error should be considered significant.

As described in the Summary Report, Maine Yankee's HED assessment methodology relied primarily on pl a nt safety as the criterion for determining the need for corrective action of those HEDs identified during the review. Consistent with NUREG-0800, HEDs that were deemed to jeopardize plant safety were scheduled for corrective action. However, thi specific means used by Maine Yankee to determine if an HED jeopardized plant safety I

11

Supplement 1 to NUREG-0737 requires the comparison of control room control and display requirements and characteristics with those derived from

< a prescriptive task analysis. Because of potential inadequacies in the performance of the system function and task analysis (mentioned above), it is not clear whether Maine Yankee performed a comparison of information and control requirements based on an in-depth, prescriptive, and systematic task analysis with its control room inventory. Thus, it is impossible to evalu-ate whether the licensee has satisfied the intent of the inventory require-ment of Supplement 1 to NUREG-0737 until further description of its task analysis procedure and results are provided. The inventory by itself, as '

represented by a full-scale sock-up and equipment list, is complete and satisfactory.

4. Control Room Survey Maine Yankee conducted a control room survey based on the guidance and checklist format in Chapter 6 of NUREG-0700. Each item in Chapter 6 was addressed to determine conformance by either measurement or observation.

Items which were not relevant to the Maine Yankee plant were documented. To

.the extent possible, the mock-up was used for the review. However, those

~

criteria that could not be evaluated on the mock-up were evaluated in the control room.

It appears from both a review of documentation and discussions with Maine Yankee staff that the control room survey was conducted in accordance

with the requirements of Supplement 1 to NUREG-0737 and the guidance of NUREG-0700. The scope of the control room survey was comprehensive and included consideration of all primary control panels. Whether the Remote Shutdown Panel was reviewed is unclear. Although not explicitly identified

! as a requirement in Supplement 1 to NUREG-0737, the NRC staff has recommended that a human engineering evaluation of the Remote Shutdown Capability be included in the scope of the DCROR.

ASSESSENT AND IMPLEENTATION PHASE i

l Maine Yankee's assessment and implementation phase is addressed in I Sections 5 through 8 of the Summary Report. Sections 5 through 7' relate to l

, 10 l I

l

4 I

information and control.needs beyond the safety function level necessary for preparing E0Ps and determining the adequacy of existing instruments and controls. Therefore, further documentation and analysis are required by licensees belonging to the CEOG.

4 More specifically, a licensee must describe the process used to 4

identify parameters and other information and control needs that are not  !

q provided in, or are different from, thos'e specified in the EPGs. They must l also describe how the characteristics of needed information and controls are  !

determined. The task analysis should go to the level of actual operator j actions that are required to perform the E0P steps such that the information i

and control needs for each operator action or subtask are identified. This I is important in that the E0Ps may not encompass all tasks that come into play during emergency operation. However, for each instrument and control i

, used to perform the E0Ps, there should be an auditable record that defines -

f the necessary characteristics of the instrument or control and the bases for that determination.

In summary, the information regarding the licensee's System Function

Review and Task Analysis is still not sufficient to evaluate whether it satisfied the intent of the requirement of NUREG-0737, Supplement 1. Con-l* sistent.with audit team recommendations, the licensee needs to

(1) provide assurance that all display and control interfaces involved in the execution of the upgraded, symptom-based procedures were analyzed; and (2) provide

assurance that the depth of the analysis is adequate to define sequentially necessary functions, tasks / subtasks, information and control capabilities,

and, finally, necessary control and display characteristics. The licensee I should provide assurance that all analyses conducted were systematic, unbiased, and independent of the constraints of the existing control room.

l

3. Control Room Inventory The licensee has prepared a full-scale mock-up from control room

photographs which contains all instruments and controls. In addition, a j list of all instruments and controls has been prepared. Information such as instrument ranges, scaling, manufacturer, etc., were obtained, when j required, from applicable plant documents.

• i l

.- - -.- -. - - - . - .-.- - ..- - - _ - _.