ML20133G205
| ML20133G205 | |
| Person / Time | |
|---|---|
| Site: | 05200003 |
| Issue date: | 01/10/1997 |
| From: | Huffman W NRC (Affiliation Not Assigned) |
| To: | Liparulo N WESTINGHOUSE ELECTRIC COMPANY, DIV OF CBS CORP. |
| References | |
| NUDOCS 9701150221 | |
| Download: ML20133G205 (16) | |
Text
. -. - - - - -. - -
January 10, 1997.
Mr. Nichslas J. Liparulo, Manager Nuclear Safety and Regulatory Analysis Nuclear and Advanced Technology Division Westinghouse Electric Corporation P.O. Box 355 i
Pittsburgh, PA 15230
=
SUBJECT:
COMENTS ON THE AP600 HUMAN FACTORS VERIFICATION AND VALIDATION PLAN
Dear Mr. Liparulo:
The Nuclear Regulatory Commission (NRC) staff has recently completed review of the Human System Interface Design Test Program, Section 18.11, Revision 9, of the AP600 Standard Safety Analysis Report (SSAR). Section 18.11 of the SSAR addresses Element 10 of the Human Factors Engineering Program Review Model (HFEPRM).- The staff has also reviewed other material associated with Sec-tion 18.11 including WCAP-14401, revision 1, " Programmatic Level Description of the AP600 Human Factors Verification and Validation Plan," submitted by Westinghouse letter NSD-NRC-96-4784 dated August 22, 1996. WCAP-14401 is the principal supporting document for the Element 10 SSAR review. A status update of the Element 10 review of SSAR Section 18.11 and supporting WCAPs is.
enclosed with this letter. Although the staff. finds that reviewed material resolves all previously identified open items associated with Element 10, a new question concerning the use of simulators has been recently identified and has been included in:the enclosure as the only remaining open item for Element 10.
If you have any questions regarding this matter, you can contact me at (301) 415-1141.
l Sincerely, criainal signed b W111f'am C. Huffman,y: Project Manager Standardization Project Directorate
. Division of Reactor Program Management.
0ffice of Nuclear Reactor Regulation i
Docket No.'52-003' t
Enclosure:
AP600 DSER Open Item i
Resolution of Element 10 Human Factors Verification and Validation
+
cc w/ enclosure:-
See next page DISTRIBUTION:
Docket File PDST R/F TMartin 1 l PUBLIC DMatthews TRQuay TKenyon BHuffman JSebrosky
)
DJackson 150000 JMoore. 0-15 818 woean, 0-17 G21
{)F ACRS (11)
BBoger, 0-9 E4 JBongarra, 0-19 H15 DOCUMENT NAME: A:EL-10RV1.LTR Ta smashe a espy of this doeusnont,indseem in the boa: "C' = Copy udthout attachment / enclosure
- E' = copy with attachment / enclosure
- N' = No copy 0FFICE PM:PDST:DRPM BC:HHFB:D3CH 4/ D:PDST:DRPM,I l
l NAME WCHuffmant44 C tibdihis)
TRQuay
\\\\0%
DATE 01//o /97 01/w/97 Ol/tV/97 T'
}]{
l D
D 052 3
Mr. Nicholas J. Liparulo Docket No'.52-003 Westinghouse Electric Corporation AP600 I
l cc: Mr. B. A. McIntyre Mr. Ronald Simard, Director Advanced Plant Safety & Licensing Advanced Reactor Programs Westinghouse Electric Corporation Nuclear Energy Institute i
Energy Systems Business Unit 1776 Eye Street, N.W.
~1 P.O. Box 355 Suite 300 Pittsburgh, PA 15230 Washington, DC 20006-3706 Mr. John C. Butler Ms. Lynn Connor i
Advanced Plant Safety & Licensing Doc-Search Associates Westinghouse Electric Corporation Post Office Box 34 Energy Systems Business Unit Cabin John, MD 20818 i
Box 355 l
Pittsburgh, PA 15230 Mr. James E. Quinn, Projects Manager LMR and SBWR Programs Mr. M. D. Beaumont GE Nuclear Energy Nuclear and Advanced Technology Division 175 Curtner Avenue, M/C 165 Westinghouse Electric Corporation San Jose, CA 95125 One Montrose Metro l
11921 Rockville Pike Mr. Robert H. Buchholz Suite 350 GE Nuclear Energy Rockville, MD 20852 175 Curtner Avenue, MC-781 San Jose, CA 95125 j
Mr. Sterling Franks l
U.S. Department of Energy Barton Z. Cowan, Esq.
i NE-50 Eckert Seamans Cherin & Mellott i
19901 Germantown Road 600 Grant Street 42nd Floor
]
Germantown, MD 20874 Pittsburgh, PA 15219 i
Mr. S. M. Modro Mr. Ed Rodwell, Manager Nuclear Systems Analysis Technologies PWR Design Certification Lockheed Idaho Technologies Company Electric Power Research Institute i
Post Office. Box 1625 3412 Hillview Avenue j
Idaho Falls, ID 83415 Palo Alto, CA 94303 Mr. Frank A. Ross Mr. Charles Thompson, Nuclear Engineer U.S. Department of Energy, NE-42 AP600 Certification Office of LWR Safety and Technology NE-50 1
i 19901 Germantown Road 19901 Germantown Road l
Germantown, MD 20874 Germantown, MD 20874 i
i I
i
}
e AP600 DSER Open Item Resolution Element 10 HF Verification and Validation To address Element 10 open items, Westinghouse submitted a Draft Document entitled " Programmatic Level Description of the AP600 Human Factors Verification and Validation Plan" (April 12,1995). The document specified the V&V activities to be performed for the AP600 HFE at a high-level. The staff reviewed the draft and provided comments in a telephone conference on May 8, 1995. As a result of the staff's comments Westinghouse provided a revision to the draft plan on May 10, 1995. Westinghouse published the revised plan as WCAP-14401, Revision 1, Programmatic Level Description of the AP600 Human Factors Verification and Validation Plan (August 1996).
The plan is referenced in the SSAR (Revision 9), Section 18.11. However, the primary focus of the staff's review was on WCAP-14401. Additional Westinghouse support documents, WCAP-14701, revision 0, " Methodology and Results of Defining Issues for the AP600 Human System Interface Design Test Program," and WCAP-14396, revision 1, " Man-in-the-Loop Test Plan," were also reviewed by the staff as part of the evaluation on this element.
The following is an overview of the status of the results of the review for all Element 10 open items.
Open Item (0!TS f. DSER f)
Current Status 18.11.3.1-1:
General V&V Scope Resolved 18.11.3.1-2:
V&V Activities and Sequence Resolved 18.11.3.1-3:
V&V Methodology Source Materials Resolved 18.11.3.2-1: HSI Task Support Verification Resolved 18.11.3.3-1:
HFE Design Verification Methods Resolved 18.11.3.4-1:
Integrated System Validation Methodology Action W 18.11.3.5-1:
Issue Resolution Verification Methods Resolved 18.11.3.6-1:
Final Plant HFE/HSI Design Verification Methodology Resolved Element 10 is being reviewed at a Programmatfc Review level, therefore, detailed evaluations using the HFE PRM acceptance criteria are beyond the scope of the staff review for design certification. At a programmatic level review, the HFE PRM criteria are used to determine whether the Westinghouse program provides a top-level identification of the substance of each criterion which, after design certification, will be developed by Westinghouse into a detailed implementation plan. Westinghouse commitment to the development of such a detailed implementation plan should be described in ITAAC/DAC. The staff will review this plan during post certification review activities.
ITAAC/DAC are also needed for completing the implementation plan and providing the results to the staff for review.
Consistent with this approach, WCAP-14401, Revision 1, indicated that "indi-vidual implementation plans that provide more detailed descriptions of the tests to be performed, and acceptance criteria to be used, will be developed l
Enclosure
for each of the V&V activities specified in this document" (p.1).
The commitment to develop detailed implementation plans was reiterated in Sec-tion 1.3.
Open Item 18.11.3.1-1: General V1V Scone 1.
Criterion: The general scope of V&V should include the following for all applicable facilities as defined in Element I-Human Factors Engineering Program Management:
HSI hardware HSI software Communications Procedures Workstation and console configurations Design of the overall work environment Trained personnel e
The scope of Integrated System Validation may be limited to those applicable facilities required for the evaluation of scenarios described in Item 4 of DSER Section 18.11.4.4 - Integrated System Validation, below.
DSER Evaluation: The general scope of the V&V program plan contained in SSAR Sections 18.5 and 18.8 addresses the identified aspects of the HSI.
Westinghouse's response to RAI 620.82 indicated that local control stations (LCSs) at which critical human actions for abnormal and emergency procedures will be performed will be included in the V&V plan. TSC inclusion in Evalua-tion Issue 16 (described in SSAR Section 18.8.2.3.5.4, " Evaluations for Conformance to HFE Design Guidelines"), and Evaluation Issue 17 (described in SSAR Section 18.8.2.3.5.5, " Evaluations for Validation of Integrated M-HIS,")
need to be clarified for closure of this criterion.
Proposed Resolution:
In WCAP-14401, Revision 1, Westinghouse clarified the scope of the V&V effort. Section 1.2 provides the scope of the V&V tests and includes the PPJi identified scope including the TSC. Westinghouse modified their scope from that provided in their response to RAI 620.82 and indicated that although the current design of the AP600 does not require risk-signifi-cant actions to be taken at LCSs, such actions will be included in V&V should any be identified in future analyses. Based upon this information, this DSER issue is considered resolved and the criterion is satisfied.
STATUS OF QPEN ITEM: Resolved Ooon Item 18.11.3.1-2: V&V Activities and Secuence
- 2. Criterion: The sequence for completion of V&V activities should be as follows:
HSI Task Support Verification HFE Design Verification 2
4 4
Integrated System Validation
{
Human Factors Issue Resolution Verification 4
Final Plant HFE/HSI Design Verification.
DSER Evaluation: Human Factors Issue Resolution, HSI Task Support Verifica-tion, and Final Plant HFE/HSI Design Verification, as described in the HFE PRM, are not discussed in the SSAR. The sequence cannot be addressed. This criterion can not be addressed until the component V&V issues (HSI Task 4
Support Verification, etc.) which follow are addressed.
Proposed Resolution:
In WCAP-14401, Revision 1, Westinghouse clarified the evaluations to be performed as part of the V&V effort. Section 1.1 identifies and defines five evaluation activities: Task Support Verification, HFE Design Verification, Integrated System Validation, Issue Resolution Verification, and l
Final Plant HFE Verification. Figure 1 illustrates the sequence of activities and is consistent with that specified in the HFE PRM criterion.
Based upon this information, this DSER issue is considered resolved and the criterion is j
satisfied.
{
STATUS OF OPEN ITEM: Resolved Open Item 18.11.3.1-3: V1V Methodolony Source Materials 1
{
3.
Criterion: The applicant's V&V effort should be developed using accepted industry standards, guidelines, and practices. A list of documents which may l
be used as guidance is provided in the HFE PRM.
DSER Evaluation: The industry standards and guidelines that will guide the development of the V&V implementation plan are not described in the SSAR.
4 Proposed Resolutfon:
In WCAP-14401, Revision 1, Westinghouse has clarified the technical basis of the V&V effort.
Section 1.3 identifies the industry i
standards, guidelines, and supporting documents that will serve as the basis i
of V&V methodology development. These documents include the IEC and IEEE guidelines as well as the NRC documents that are appropriate to V&V.
l Based upon this information, this DSER issue is considered resolved and the criterion is satisfied.
4 i
STATUS 0F OPEN ITEM: Reso1ved 1
j Open Item 18.11.3.2-1: MSI Task Suonort Verification i
l.
Criterion: All aspects of the HSI (e.g., controls, displays, procedures, and data processing) that are required to accomplish human tasks and actions
[as defined by the task analysis, E0P analysis, and the critical actions of the probabilistic risk assessment / human reliability analysis (PRA/HRA)] should i
be verified as available through the HSI.
k i
i j
DSER Evaluatfon: HSI Task Support Verification is not clearly addressed as part of V&V activities. Westinghouse's response to RAI 620.81 indicated that the design review of displays and controls will confirm that task analysis-identified needs are satisfied at the HSI. However, the timing of such a review and procedures for conducting such reviews as part of vetification has not been identified.
i 2.
Crfterion:
It should be verified that the HSI does not include informa-tion, displays, controls, etc. which do not support operator tasks. This includes non-functional, decorative details such as borders and shadowing on graphical displays.
DSER Evaluation: Westinghouse's response to RAI 620.81 addressed this aspect of verification.
Westinghouse's response indicated that unnecessary (as defined by task analysis) indications and controls will be deleted. At a meetingbetweenWestinghouseandthestaffon12/h3-14/94,thestaffexpressed concern that this decision not be made on the bas's of task analysis alone and that an operational review be performed to verify that deletion of any aspect of the HSI was acceptable. Westinghouse was in agreement and this review should be addressed in the implementation plan.
Proposed Resolutfon:
In WCAP-14401, Revision 1, Westinghouse described the Westinghouse's general approach to HSI Task Support Verification. Section 2 identified the objective and high-level methodology for conducting the
~;
evaluation. The analysis will address the availability of M-MIS features for accomplishing personnel tasks and actions as defined by the task analyses, the E0Ps, and the risk-important human tasks identified by the PRA. This commit-ment satisfactorily addresses Criterion 1.
The plan also indicated that the methodology shall describe how, in each case, the M-MIS design will be verified to ensure that the M-MIS does not include information, controls, and displays that do not support operator tasks. A process for checking such M-MIS features will include an analysis before any information is removed from the M-MIS. This commitment satisfactorily addresses Criterion 2.
Based upon this information, this DSER issue is considered resolved and the criteria are satisfied.
STATUS OF OPEN ITEM: Reso1ved Onen Item 18.11.3.3-1: HFE Desian Verification Methods i
1.
Criterfon: All aspects of the HSI (e.g., controls, displays, procedures, and data processing) should be verified as designed to be appropriate to personnel task requirements and operational considerations as defined by design specifications, and are consistent with accepted HFE guidelines, standards, and principles.
4
OSER Evaluatfon: HFE Design Verification is described in SSAR Sec-tion 18.8.2.3.5.4.1 Evaluation Issue 16: HFE Guidelines.
The acceptance testing aspect to Evaluation Issue 16 addresses the HFE PRM level verifica-tion. The focus of this verification is on (1) evaluating that individual h-MIS components satisfy human engineering criteria, and (2) evaluating that tha integration of M-MIS components satisfy human engineering criteria for work 1
environments. The guidelines are applied to the MCR, remote shutdown station, and other local panels.
(Note the scope item raised in DSER Sec-tion 18.11.3.1, Criterion 1 above regarding the verification of TSC M-MIS components affects this verification.)
Procedures for verification have not been identified and are beyond the scope of design certification. The sources of guidance documents to be used in i
these verifications have not been precisely identified.
SSAR Table 18.5.1 identifies the following documents as being included: NUREG-0700, MIL-STD-1472, ANSI /HFS 100-1988, ASHRAE STD 55-1981, and EPRI-3659. The staff had concern regarding the completeness and appropriateness of these documents for verification of an advanced control room and requested additional information regarding the technical basis of verification guidelines in RAIs 6?P.20 and 620.59. Westinghouse indicated that the documents listed show an illustrative subset of the guidelines to be used. Additional documents will be reviewed for possible inclusion in the list. The actual verification will be based upon six guideline documents addressing: alarms, displays, controls, training, anthropometry, and subsystem integration.
These documents have not been provided to the staff. However, it is not apparent from the document titles that important topics such as procedure HSI design and user-system interaction design (e.g., dialogue format and navigation tools) are addressed by the documents and, therefore, in the HFE Design Verification.
2.
Criterion: Deviations from accepted HFE guidelines, standards, and principles should be acceptably justified based upon a documented rationale such as trade study results, literature based evaluations, demonstrated operational experience, and tests / experiments.
DSER Evaluation: Treatment of deviations is not addressed in the SSAR.
Proposed Resolutfon:
In WCAP-14401, Revision 1, Westinghouse described the 1
general approach to HFE Design Verification.
Section 3 identified the objective and high-level methodology for conducting the evaluation. The analysis will address the verification that all aspects of the M-MIS are consistent with accepted HFE guidelines, standards, and principles. The verification will utilize AP600-specific guidance documents and will cover alarms, displays, controls, data processing, navigation, computerized proce-dures, workstation and console configurations, and anthropometric consider-4 ations and their integration. The document identified an illustrative subset of the documents that will be used in the development of the AP600-specific guidance.
It included the most recent control room design guidance including IEC 964 and NUREG/0700 (Revision 1). This commitment satisfactorily addressed the staff's DSER concerns with regard to Criterion 1.
i i
5 1
4 The plan also identified the process through which guidelines deviations will be addressed and their technical basis documented. This commitment satisfac-torily addressed Criterion 2.
4 Based upon this information, this DSER issue is considered resolved and the criteria are satisfied.
STATUS OF OPEN ITEM: Res01ved Open Item 18.11.3.4-1:
Intearated System Validation Methodolony 1.
Criterion: The methodology for integrated system validation should address:
4 General objectives Per:onnel performance issues to be addressed (e.g., crew coordination)
Test methodology and procedures 1
Test participants (operators to participate in the test program)
Test conditions (including plant conditions, operating sequences, and accident scenarios)
HSI description Performance measures Data analysis OSER Evaluation: Technical review of this item is beyond the scope of the design certification review since it is the framework for the V&V implementa-tion plan.
Proposed Resolution:
In WCAP-14401, Revision 1, Westinghouse described the general approach to Integrated System Validation.
Section 4 identified the objective and high-level methodology for conducting the evaluation.
Section 4.1 identified the aspects to the methodology that will be addressed in the implementation plan.
Each of the topics identified in the PRM was included.
In addition, the plan addressed the process by which results will be used to evaluate potential design changes and where made their subsequent verifica-tion.
This satisfactorily addressed Criterion 1.
2.
Criterion: Validation should be performed by evaluating dynamic task performance using tools which are appropriate to the accomplishment of this objective. The primary tool for this purpose is a simulator, i.e., a facility that physically represents the HSI configuration and which dynamically represents the operating characteristics and responses of the plant design in real time. The requirement to validate performance at plant HSIs outside the CR will be dependent on the app 11 cant's design. Human actions at non-CR facilities such as remote shutdown panels and LCSs may be evaluated using mock-ups, prototypes, or similar tools.
)
6 i
l DSER Eva7uatfon: SSAR Section 18.8.2.3.5.5.1 and Westinghouse's response to RAI 620.18 describe the tools to be employed for validation testing.
Westing-house will use a "near full-scope, high fidelity simulator consisting of integrated M-MIS components and a high-fidelity dynamic simulation of plant behavior." As indicated previously in the DSER evaluation of Criterion 1 in DSER Section 18.8.3.1, the role of the TSC needs clarification.
Proposed Resolutfon:
In WCAP-14401, Revision 1, Westinghouse described the i
general approach to Integrated System Validation. Section 4.2 addressed the tools for conducting validation. A "near full-scope" simulator will be used.
"Near" means that features of the simulation that are not relevant to the tests being performed may not be high-fidelity.
Personnel actions that are performed at non-control room facilities, such as remote shutdown panels and the TSC may be evaluated using static mock-ups or prototypes.
As a result of reviewing SSAR, Revision 9, and several supporting WCAPs, the staff has identified the need for further clarification from Westinghouse on their use of the simulator as an evaluation tool for the AP600 HSI design.
Specifically, WCAP-14401, Revision 1, Section 4.0, describes Westinghouse's approach for addressing Integrated System Validation. Westinghouse indicates that " integrated system validation will be performed using an AP600-specific, near full-scope, high-fidelity simulator of the AP600 control room that is similar to a trainina simulator. However, Figure 1.1 of WCAP-14401, Revi-sion 1, identifies that Integrated System Validation will utilize an AP600-specific, near full-scope, high fidelity, trainina simulator.
In WCAP-14396, i
Revision 1, Man-In-The-Loop Test Plan Description, Section 3.0, Formal V&V of Final HSI Design, Westinghouse indicates that formal HFE/HSI design V&V will be performed when an AP600 plant has been purchased and will use an AP600 dynamic, high fidelity trainina simulator.
In Westinghouse's September 15, 4
1992, letter to the Chairman (ET-NRC-92-3748), in addressing Item F (Role of I
the Operator In A Passive Plant Control Room), Westinghouse stated that a high-fidelity, near full scope control room prototype (eauivalent to a trainina simulator) is included near the end of the [ man-in-the-loop testing]
program to perform certain verification and validation tests. Westinghouse should clearly describe: 1) the use of each simulator type, (near full-scope, high fidelity simulator that is similar to a training simulator; near full-3 scope, high fidelity training simulator; training simulator); 2) the differ-i ences that exist among the simulator types; and, 3) the guidance /information sources that might be used to support their development (e.g., ANSI 3.5, RG 1.149; EPRI NP-6701; IAEA-TE000-685, etc.).
As was indicated in the discussion of Open Item 18.11.3.1-1: General V&V Scope above, the staff recognizes that at present the AP600 design does not require 1
risk-significant actions to be taken from LCSs, therefore they are not j
included in the scope of V&V.
Further as indicated in that discussion, this is acceptable to the staff since Westinghouse will include such LCSs in V&V evaluations should the further detailed design of the plant require a risk-
)
important action to be performed at a LCS. Given this interpretation, the staff's DSER concerns with regard to Criterion 2 are addressed.
i l
7 1
This commitment satisfactorily addressed Criterion 2, however, this is an open item until the staff's questions on the use of simulators are addressed in a revision to the SSAR or an appropriate, docketed, secondary reference.
3.
Criterion: The integrated system validation DSER Evaluations should address:
Adequacy of entire HSI configuration for achievement of HFE program goal s, Confim allocation of function and the structure of tasks assigned to personnel, Adequacy of staffing and the HS! to support staff to accomplish their
- tasks, Adequacy of procedures, Confirm the dynamic aspects of the HSI for task accomplishment, and DSER Evaluation and demonstration of error tolerance to human and system failures.
OSER Evaluatfon: SSAR Section 18.8.2.3.5.5.1 indicated that the general question being addressed is the support for operator performance during normal, abnormal, and emergency conditions provided by the integration of M-MIS components in the MCR. The purpose of the evaluations is to determine whether the M-MIS, as designed and implemented, supports safe and efficient operation of the plant for the conditions addressed by the design mission.
This approach is consistent with the HFE PRM criterion; however, the specific types of evaluations are not identified in the SSAR. Specification of evaluations should be addressed in the implementation plan.
Proposed Resolution:
In WCAP-14401, Revision 1, Westinghouse described their general approach to Integrated System Validation.
Section 4.3 identified the objectives of Integrated System Validation. The implementation plan will specifically address each of the objectives identified in the HFE PRM.
This satisfactorily addressed Criterion 3.
4.
Criterion: All critical human actions as defined by a task analysis and PRA/HRA should be tested and found to be adequately supporud in the design, including the performance of critical actions outside the control room. The design of tests and evaluations to be performed as part of HFE V&V activities should 'specifically examine these actions.
DSER Evaluatfon:
In their response to RAI 620.51 (Revision 2), Westinghouse identifled WCAPs 9817 and 12601 as describing "the scope and process for verification of the M-MIS to ensure that all critical human actions as defined by the task analysis and PRA have been adequately supported in the design, and that the V&V program explicitly addresses these issues." These WCAPs have not yet been received for review.
Proposed Resolutfon:
In WCAP-14401, Revision 1, Westinghouse described the general approach to Integrated System Validation. Section 4.4 identified the specific commitment to validate the performance of risk-important tasks.
8
{
l l
1
4 These tasks were defined as (1) important and representative tasks defined in task analysis, (2) risk important tasks defined by the PRA threshold criteria, and design-basis and beyond-design-basis accident scenarios covered by the E0Ps.
This satisfactorily addressed Criterion 4.
5.
Crfterion: Regulatory Guide (RG) 1.33, Appendix A contains several i
categories of activities that should be covered by procedures.
The validation should evaluate selected evolutions based upon procedures developed to address i
this Guide. The DSER Evaluation should include appropriate procedures in each l
relevant category, i.e.:
Administrative Procedures General Plant Operating Procedures o
j Procedures for Startup, Operation, and Shutdown of Safety-Related Systems Procedures for Abnormal, Offnormal and Alarm Conditions Procedures for Combating Emergencies and Other Significant Events Procedures for Control of Radioactivity Procedures for Control of Measuring and Test Equipment and for Surveil-lance Tests, Procedures, and Calibration Procedures for Performing Maintenance Chemistry and Radiochemical Control Procedures DSER Evaluation: Not addressed in the SSAR.
i Proposed Resolutfon:
In discussions of this PRM criterion, Westinghouse requested clarification of whether each category of procedures indicated in the PRM criterion is to be addressed by validation. The staff indicated that RG 1.33 categories were included in the PRM because they encompass " typical safety-related activities that should be covered by written procedures."
Thus, all of the above categories should be represented in the scenario sampling process. However, it is recognized that not all categories need to receive equal emphasis and some categories (e.g., administrative procedures and procedures for performing maintenance) may be best evaluated as an adjunct to other tests.
Administrative procedures are important to safe plant operation, however, they may not need to be tested as completely as E0Ps.
Instead, selected situations governed by such procedures should be reflected in validation scenarios to ensure that the AP600 CR design in conjunction with such procedures can achieve their intended functions without interfering with plant operations.
9
l Thus for example, situations involving equipment control (e.g., locking and tagging of equipment), shift and relief turnover, or maintenance of minimum shift complement and call-in of personnel, could be incorporated into selected i
test scenarios or validated separately.
Procedures for Performing Maintenance are least amenable to validation of the i
type covered by this PRM criterion. While the staff considers the design for maintenance an important aspect of plant design and one which is addressed by the HFE program, it does not typically involve validation of an integrated system. The staff does think it is appropriate to validate maintenance that i
is to be performed in the NCR while the plant is being operated.
This validation should show that it can be accomplished without interfering with 4
operator tasks that are necessary for monitoring and controlling the plant.
Thus in this restricted context, Procedures for Performing Maintenance should be included as a small part of validation tests.
As is indicated in RG 1.33, the procedures may be combined, separated, or deleted to conform to the applicant's procedures plan. The same approach is applicable to integrated system validation. The main goal of integrated system validation is evaluate the performance of the integrated system in
" operational" contexts, and not to validate procedures or any other single aspect of the design. Reference in the PRM to the procedure categories is to provide an aid to defining the range of operational contexts that are appro-4 j
priate to the integrated system performance.
Westinghouse included a discussion of their treatment of Regulation Guide 1.33 procedures in WCAP-14401. Section 4.5 indicated that Westinghouse will i
include test scenarios the create situations governed by sample procedures from selected RG 1.33 procedures to ensure that the performance of plant j
operations.
I This satisfactorily addressed Criterion 5.
6.
Criterfon: Dynamic Evaluations should evaluate HSI under a range of operational conditions and upsets, and should include:
J Normal plant evolutions, e.g., start-up, full power, and shutdown operations; i
Instrument failures, e.g., safety-related system logic & control (SSLC) unit, fault tolerant controller (NSSS), local " field unit" for multiplexer (MUX) system, break in MUX line; HSI equipment and processing failure, e.g., loss of VDUs, loss of data processing, loss of large overview display; Transients, e.g., turbine trip, loss of offsite power, station black-out, loss of all feed water, loss of service water, loss of power to selected buses /CR power supplies, and SRV transients;
[
l 10 4
9
Accidents, e.g., main steam line break, positive reactivity addition, control rod insertion at power, control rod ejection, anticipated transient without scram (ATWS), and various-sized loss of coolant accidents (LOCAs); and Reactor shutdown and cooldown from remote shutdown panel.
DSER Evaluation: RAI 620.60 requested information regarding conditions scenario types such as instrument failures, HSI equipment and processing i
failure, and accidents.
Westinghouse has indicated that plant conditions such
{
as those identified in the criterion will be addressed in validation.
Scenario selection will be defined in terms of cognitive demands.
When the cognitive selection criteria are mapped onto specific test scenarios, the resulting set of scenarios will include the types of events listed. However, at the present level of description provided in the SSAR, it is not possible to determine that the proposed approach will result in the scenario diversity specified in the HFE PRM. Additional information regarding the identifica-tions of test scenarios for Evaluation Issue 17 should be included in the 4
implementation plan.
Proposed Resolutfon:
In WCAP-14401, Revision 1, Westinghouse described their general approach to Integrated System Validation.
Section 4.6 discussed the selection of test scenarios. Test scenarios will be defined using a multi-dimensional set of criteria. The dimensions are identified and include all of the types of scenarios included in the HFE PRM.
In addition, Westinghouse has identified design features that are specific to AP600 such as ADS, situations that are cognitively challenging to the crew such as complicated situation assessment under conflicting plant state information, and scenarios that would
]
enable validation of key HRA assumptions.
.r l
This satisfactorily addressed Criterion 6.
7.
Criterfon: The validation scenarios should be realistic. Selected ones should include environmental conditions such as noise and distractions which may effect human performance in an actual NPP.
For actions outside the control room, the performance impacts of potentially harsh environments (i.e.,
high radiation) which require additional time should be realistically simulat-ed (i.e., time to don protective clothing and access hot areas).
DSER Evaluation: Not addressed in the SSAR.
Proposed Resolution:
In WCAP-14401, Revision 1, Westinghouse described their general approach to Integrated System Validation.
Section 4.7 addressed how the scenarios selected for validation will be made realistic. Considerations regarding the incorporation of environmental conditions, communication demands, number of personnel in the control room were identified in the program description.
This satisfactorily addressed Criterion 7.
11
\\
l l
8.
Crfterion:
Performance measures for dynamic evaluations should be adequate to test the achievement of all objectives, design goals, and perfor-l l
mance requirements and should include at a minimum:
System performance measures relevant to plant safety Crew primary task performance (e.g., task times, procedure violations)
Crew errors Situation awareness Workload Crew communications and coordination Dynamic anthropometry DSER Evaluations Physical positioning and interactions.
DSER Evaluatfon: SSAR Section 18.8.2.3.5.5.1 and SSAR Table 18.5-2 (sheet 8) identify task completion time and task completion success as the performance measures for validation.
In addition, decision tracing will be used to evaluate participant decisions and actions.
Following scenarios, participants will be debriefed to assess their understanding of plant conditions and how
{
features of the M-MIS contributed to their performance.
In RAI 620.84 the staff requested information concerning the measurement of situation awareness and workload.
In Westinghouse's response they indicated that workload will be assessed in validation studies. However, while situa-tion awareness is a major consideration in concept tests, it will not "be a primary focus" in validation.
Situation awareness would be assessed only indirectly through observation of task performance. Situation awareness should be given similar consideration as workload. Westinghouse indicates that workload measures are most useful when complete integrated operator tasks are being performed. The same logic applies to situation awareness. Accurate situation awareness may be more difficult to establish when complete integrat-ed operator tasks are being performed. At such a time the operator's workload may be higher and the situations encountered more complex.
In fact, workload and situation awareness are closely linked. When workload goes up, operators cognitively cope by employing information processing heuristics and task management strategies.
Both can impact the operator's ability to form situation awareness.
Proposed Resolution:
In WCAP-14401, Revision 1, Westinghouse described the Westinghouse's general approach to Integrated System Validation.
Section 4.8 discussed performance measurement. The aspects to integrated system perfor-mance identified in the PRM were included. Westinghouse indicated that the process by which objective acceptance criteria for each measure will be defined in the implementation plan.
This satisfactorily addressed Criterion 8.
Based upon this information, this DSER issue is considered resolved and the criteria are satisfied.
STATUS OF OPE # ITE#: A:: tion W (Criterion 2 only)
Resolved for all other Criterion 12
Open Item 18.11.3.5-1:
Issue Resolution Verification Methods 1.
Criterion: All issues documented in the Human Factors Issue Tracking System of Element I should be verified as adequately addressed.
DSER Evaluetion:
In RAI 620.80 the staff requested information concerning issue resolution verification. Westinghouse indicated that the issues were tracked using a " human factors checklist" and their closure is identified in system design documentation which is subject to design reviews.
In their response to RAI 620.51 (Revision 2), Westinghouse identified WCAP-12601 as the document describing the process for closure of open design change proposals.
However, the staff's review of the HFE Issues Tracking system indicated that the issues tracking system was an open issue (see Section 18.2.3.4 of this report). Until resolved, the acceptability of using the approach described in Westinghouse's response to RAI 620.80 cannot be determined.
This item is related to Open Item 18.2.3.4-1:
HFE Issues Tracking System.
2.
Criterion:
Issues that could not be resolved until a plant is built should be specifically identified and incorporated into the Final Plant HFE/HSI Design Verification.
DSER Evaluatfon: The treatment of resolution of issues which remain until the plant is built is not addressed in the SSAR.
Proposed Resolutfon In WCAP-14401, Revision 1, Westinghouse described their general approach to Issue Resolution Verification. Section 5 provided a commitment to develop a procedure to ensure that all issue documented in the HFE issue tracking system are verified to be completely addressed in the final M-MIS.
This commitment satisfactorily addresses the staff's DSER concerns with regard i
to Criterion 1.
The program description further stated that the implementation plan will describe a procedure for identifying and tracking HFE issues that cannot be i
resolved until a plant is built. This procedure will address how verification of these issues ill be incorporated into the process for final plant HFE verification. This commitment satisfactorily addressed Criterion 2.
Based upon this information, this DSER issue is considered resolved and the criteria are satisfied.
ST.4TUS OF OPEN ITEM: Reso1ved l
13
l 1
Onen Item 18.11.3.6-1: Final Plant HFE/HSI Desian Verification Methadelony l
1.
Criterion:
Following design process V&V activities, a design description should be developed which describes the detailed design and its performance criteria.
I DSER Evaluation: Final Plant HFE/HSI Design Verification is not addressed in the SSAR.
2.
Criterion: Aspects of the design which were not addressed in design process V&V should be evaluated using an appropriate V&V method. Aspects of 2
the design addressed by this criterion may include design characteristics such as new or modified displays for plant specific design features and features j
that cannot be evaluated in a simulator such as control room lighting and noise.
4 DSER Evaluation:
Final Plant HFE/HSI Design Verification is not addressed in the SSAR.
J 3.
Criterion: The in-plant HFE should conform to the design that resulted from the HFE design process and V&V activities.
I DSER Evaluation:
Final Plant HFE/HSI Design Verification is not addressed in the SSAR.
4 j
Proposed Resolutfon:
In WCAP-14401, Revision 1 Westinghouse described the general approach to Final Plant HFE/HSI Design Verification. Section 6
]
provided a commitment to develop a methodology for verifying that the in-plant HFE conforms to the M-MIS design that results for the HFE design process and V&V activities. The M-MIS was defined in the final functional requirements and design description.
Conformance of the actual system to this description is verified during factory acceptance tests and site acceptance tests. The d
implementation plan will specify the verifications to be performed.
This commitment satisfactorily addressed Criteria 1 and 3.
i The program description indicated that the implementation plan will include 3'
procedures for identifying and evaluating aspects of the M-MIS that were not addressed during prior V&V activities.
This commitment satisfactorily addressed Criterion 2.
Based upon this information, this DSER issue is considered resolved and the criteria are satisfied.
STATUS OF QPEN ITEM: Resolved 14
__