Regulatory Guide 1.171
| ML003740108 | |
| Person / Time | |
|---|---|
| Issue date: | 09/30/1997 |
| From: | Office of Nuclear Regulatory Research |
| To: | |
| References | |
| DG-1057 RG-1.171 | |
| Download: ML003740108 (8) | |
U.S. NUCLEAR REGULATORY
COMMISSION
REGULATORY
GUI OFFICE OF NUCLEAR REGULATORY
RESEARCH REGULATORY
GUIDE 1.171 (Draft was DG-1057) SOFTWARE UNIT TESTING FOR DIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMS OF NUCLEAR POWER PLANTS
A. INTRODUCTION
In 10 CFR Part 50, "Domestic Licensing of Pro duction and Utilization Facilities," paragraph
55a(a)(1)
requires, in part, 1 that systems and components be de signed, tested, and inspected to quality standards com mensurate with the safety function to be performed.
Criterion
1, "Quality Standards and Records," of Ap pendix A, "General Design Criteria for Nuclear Power Plants," to 10 CFR Part 50 requires, in part, 1 that a qual ity assurance program be established and implemented in order to provide adequate assurance that systems and components important to safety will satisfactorily per form their safety functions.
Appendix B, "Quality As surance Criteria for Nuclear Power Plants and Fuel Re processing Plants," to 10 CFR Part 50 describes criteria that a quality assurance program for systems and com ponents that prevent or mitigate the consequences of postulated accidents must meet. In particular, besides the systems and components that directly prevent or mitigate the consequences of postulated accidents, the criteria of Appendix B also apply to all activities affect ing the safety-related functions of such systems and components as designing, purchasing, installing, test ing, operating, maintaining, or modifying.
A specific l1n this regulatory guide, many of the regulations have been paraphrased;
see 10 CFR Part 50 for the full text.requirement is contained in 10 CFR 50.55a(h), which requires that reactor protection systems satisfy the cri teria of IEEE Std 279-1971, "Criteria for Protection Systems for Nuclear Power Generating Stations." 2 Paragraph
4.3 of IEEE Std 279-19713 states that quali ty of components is to be achieved through the specifi cation of requirements known to promote high quality, such as requirements for design, inspection, and test. Many of the criteria in Appendix B to 10 CFR Part 50 contain requirements closely related to testing activities.
Criterion I, "Organization," requires the es tablishment and execution of a quality assurance pro gram. Criterion H, "Quality Assurance Program," re quires, in part, that the program take into account the need for special controls, processes, test equipment, tools, and skills to attain the required quality, as well as the need for verification of quality by inspection and test. Criterion III, "Design Control," requires, in part, that measures be established for verifying and checking the adequacy of design, such as by the performance of a 2 Revision I of Regulatory Guide 1.153, "Criteria for Safety Systems," en dorses IEEE Std 603-1991,"Criteria for Safety Systems for Nuclear Pow er Generating Stations," as a method acceptable to the NRC staff for satis fying the NRC's regulations with respect to the design, reliability, qualifi cation, and testability of the power, instrumentation, and control portions of the safety systems of nuclear power plants. 3 IEEE publications may be obtained from the IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854.7.USNRCREGULATORYGUIDES
The guides we Issued in the following ten broad divisions:
Reglatory Guides are Issued to descibe and make avlable tothe public such Informs lion as methods acceptable to the NRC staf for Implementing specific pans of the Com- 1. Power Reactors 6. Products mission's regulations, techniques usedbythestaff inevaluating specific problems orpos- 2Z Research and Test Reactors 7. Transportation tulated accdentsa and data needed by the NRC staff In Its review of ap:icationrs forper- 3& Fuels and Materials Facilities
8. Occupations!
Health mits and licensea.
Regulatory guides are not sstitutes for regulations, and compiance
4. Environmental and Siting 9. Antitust and Financial Review with them Is not required.
Methods and solutions different from those set out in theguides
5. Matrials and Plant Protection
10. General will be acceptable If they provide a basis for the findings requisite to the Issuance or con linuance of a permit or license by the Commission.
Single copies of regulatory guides may be obtained free of chrge bywrlfing te Printing.
This guide was lesu after consideration of comments received from thre public. Com- Graphics anid Distribution Branch. Office of Administrtion, U.S. Nuclear Regulatory Com ments andsuggestions for inprovements Inthese guides wencosurged at all Imes, and mission, Washington, DC 2055-0001;
or by fox at (301)415-5272 gue will be revised, as appropriate, to accommodate comments and to reflect new in= on or aperience.
Issued guides may also bepurchased!
from me* National Technical Information Service on Whitten comments may be submitted to te Rules Review and Directives Branch, DFIPS, a standing order basis. Details on this service may be obtained by writing NTIS, 5285 Port ADM, U.S. Nuclear Regulatory Commission, Washington, DC 2055-0001.
Royal Road, Springfield, VA 2216
1. DE September
1997 suitable testing program, and that design control measures be applied to items such as the delineation of acceptance criteria for inspections and tests. Criterion V, "Instructions, Procedures, and Drawings," requires activities affecting quality to be prescribed by docu mented instructions, procedures, or drawings of a type appropriate to the circumstances and that these activi ties be accomplished in accordance with these instruc tions, procedures, or drawings.
Criterion V further re quires that instructions, procedures, and drawings include appropriate quantitative or qualitative accep tance criteria for determining that important activities have been satisfactorily accomplished.
Criterion XI, "Test Control," requires establishment of a test pro gram to ensure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorpo rate the requirements and acceptance limits contained in applicable design documents.
Test procedures must include provisions for ensuring that all prerequisites for the given test have been met, that adequate test instru mentation is available and used, and that the test is per formed under suitable environmental conditions.
Crite rion XI also requires that test results be documented and evaluated to assure that test requirements have been sat isfied. Finally, Criteria VI, "Document Control," and XVII, "Quality Assurance Records," provide for the control of the issuance of documents, including changes thereto, that prescribe all activities affecting quality and provide for the maintenance of sufficient records to furnish evidence of activities affecting quali ty. The latter requires test records to identify the inspec tor or data recorder, the type of observation, the results, the acceptability of the results, and the action taken in connection with any deficiencies noted. This regulatory guide endorses ANSI/IEEE Std 1008-1987, "IEEE Standard for Software Unit Test ing," 3 with the exceptions stated in the Regulatory Position.
IEEE Std 1008-1987 describes a method ac ceptable to the NRC staff for complying with parts of the NRC's regulations for promoting high functional reliability and design quality in software used in safety systems.4 In particular, the method is consistent with the previously cited General Design Criteria and the criteria for quality assurance programs of Appendix B as they apply to software unit testing. The criteria of Appendices A and B apply to systems and related quali 4 The term "safety systems" is synonymous with "safety-related systems." The General Design Criteria cover systems, structures, and components "important to safety." The scope of this regulatory guide is, however, lim ited to "safety systems," which are a subset of "systems important to safety..ty assurance processes, and if those systems include software, the requirements extend to the software ele ments. In general, information provided by regulatory guides is reflected in the Standard Review Plan (NUREG-0800).
The Office of Nuclear Reactor Regu lation uses the Standard Review Plan to review applica tions to construct and operate nuclear power plants. This regulatory guide will apply to the revised Chapter 7 of that document.
The information collections contained in this regu latory guide are covered by the requirements of 10 CFR Part 50, which were approved by the Office of Manage ment and Budget, approval number 3150-0011.
The NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information un less it displays a currently valid OMB control number.
B. DISCUSSION
The use of industry consensus standards is part of an overall approach to meeting the requirements of 10 CFR Part 50 when developing safety systems for nuclear power plants. Compliance with standards does not guarantee that regulatory requirements will be met. However, compliance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assur ance processes used to design safety systems. These practices are based on past experience and represent in dustry consensus on approaches used for development of such systems.
Software incorporated into instrumentation and control systems covered by Appendix B will be referred to in this regulatory guide as safety system software.
For safety system software, software testing is an im portant part of the effort to achieve compliance with the NRC's requirements.
Software engineering practices rely, in part, on software testing to meet general quality and reliability requirements consistent with Criteria 1 and 21 of Appendix A to 10 CFR Part 50, as well as Cri teria I, II, III, V, VI, XI, and XVII of Appendix B. The consensus standard, IEEE Std 1008-1987 (reaffirmed in 1993), defines a method for planning, preparing for, conducting, and evaluating software unit testing. The method described is consistent with the previously cited regulatory requirements as they apply to safety system software.
Current practice for the development of software for high-integrity applications includes the use of a software life cycle process that incorporates software testing activities, e.g., IEEE Std 1074-1991, "IEEE Standard for Developing Software Life Cycle 1.171-2 ,
Processes." 3 Software testing, including software unit testing, is a key element in software verification and validation activities, as indicated by IEEE Std 1012-1986, "IEEE Standard for Software Verification and Validation Plans," 3 and IEEE Std 7-4.3.2-1993, "Standard Criteria for Digital Computers in Safety Sys tems of Nuclear Power Generating Stations." A com mon approach to software testing [NUREG/CR-6101, "Software Reliability and Safety in Nuclear Reactor Protection Systems" (November
1993); NUREG/ CR-6263, "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis and Re search Needs" (June 1995)]5 utilizes a three-level test program to help ensure quality in a complex software product or complex set of cooperating software prod ucts, i.e., unit-level testing, integration-level testing, and system-level testing such as system validation tests or acceptance tests. IEEE Std 1008-1987 delineates an approach to the unit testing of software that is based on the assumption of a larger context established by verifi cation and validation (V&V) planning as well as general planning for the full range of testing activities to be applied. Therefore, software unit testing per formed in accordance with IEEE Std 1008-1987 should be consistent with planning information estab lished in V&V plans and higher-level software test plans, although that planning information is not within the scope of IEEE Std 1008-1987.
C. REGULATORY
POSITION The requirements in ANSI/IEEE Std 1008-1987, "IEEE Standard for Software Unit Testing," provide an approach acceptable to the NRC staff for meeting the requirements of 10 CFR Part 50 as they apply to the unit testing of safety system software, subject to the provi sions listed below. The appendices to IEEE Std 1008-1987 are not endorsed by this regulatory guide except as noted below. Appendix A to this standard pro vides guidance regarding the implementation of the software unit testing approach, and Appendix B to the standard provides context regarding software engineer ing information and testing assumptions that underlie the software unit testing approach.
To meet the requirements of 10 CFR 50.55a(h)
and Appendix A to 10 CFR Part 50 as assured by complying with the criteria of Appendix B to 10 CFR Part 50 ap 5 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC 20402-9328 (telephone
(202)512-2249);
or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161. Copies are I available for inspection or copying for a fee from the NRC Public Docu ment Room at 2120 LStreet NW., Washington, DC; the PDR's mailing ad dress is Mail Stop LL-6, Washington, DC 20555-0001;
telephone
(202)634-3273;
fax (202)634-3343.
plied to the unit testing of safety system software, the following exceptions are necessary and will be consid ered by the NRC staff in the review of submittals from licensees and applicants. (In this section, the cited crite ria are in Appendix B to 10 CFR Part 50 unless other wise noted.) 1. SOFTWARE TESTING DOCUMENTATION
Criterion XI, "Test Control," requires that a test program be established to ensure that all testing re quired to demonstrate that systems and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorporate requirements and acceptance limits contained in applicable design documents.
Criterion I, "Organization," Criterion II, "Quality Assurance Pro gram," Criterion III, "Design Control," Criterion V, "Instructions, Procedures, and Drawings," Criterion VI, "Document Control," and Criterion XVIi, "Quality Assurance Records," contain requirements bearing on information associated with testing. IEEE Std 1008-1987, in section 1.1, mandates the use of the Test Design Specification and the Test Summary Report de fined by ANSI/IEEE Std 829-1983, "IEEE Standard for Software Test Documentation." In addition, IEEE Std 1008-1987 either incorporates additional informa tion into these two documents or indicates the need for additional documents.
Regardless of whether these two documentation formats are used, the documentation used to support software unit testing (either documen tation used directly in the software unit testing activity or documentation of the overall testing effort) must in clude information necessary to meet regulatory re quirements as applied to software test documentation.
As a minimum, this information includes:
"* Qualifications, duties, responsibilities, and skills required of persons and organizations assigned to testing activities, " Environmental conditions and special controls, equipment, tools, and instrumentation needed for the accomplishment of testing, " Test instructions and procedures incorporating the requirements and acceptance limits in applicable design documents, " Test prerequisites and the criteria for meeting them, "* Test items and the approach taken by the testing program, "* Test logs, test data, and test results, "* Acceptance criteria, 1.171-3 Test records indicating the identity of the tester, the type of observation, the results and acceptability, and the action taken in connection with any deficiencies.
Any of the above information items that are not present in the documentation selected to support soft ware unit testing must be incorporated as additional items. 2. TEST PROGRAM Criterion XI, "Test Control," requires establish ment of a test program to ensure that all testing required to demonstrate that structures, systems, and compo nents will perform satisfactorily in service is identified and performed in accordance with written test proce dures that incorporate the requirements and acceptance limits contained in applicable design documents.
The two aspects of test coverage that are particularly impor tant for the unit testing of safety system software are coverage of requirements and coverage of the internal structure of the code. 2.1 Coverage of Requirements For safety system software, those requirements identified as essential to the safety determination
6 must be tested. Section 3.2.2(5) of IEEE Std 1008-1987 sug gests consideration of expected use of the unit in the de termination of features to be tested. All features and as sociated procedures, states, state transitions, and associated data characteristics essential to the safety de termination must be included in the testing.
2.2 Coverage of Internal Structure Section 3.1.2(2) of IEEE Std 1008-1987 specifies statement coverage (covering each source language statement with a test case) as a criterion for measuring the completeness of the software unit testing activity.
Statement coverage is a very weak criterion for meas uring test completeness
[See Beizer 7 and NUREG/ CR-6263 8]. Therefore, the staff does not endorse state ment coverage as a sufficient coverage criterion for software unit testing. For safety system software, the unit test coverage criteria to be employed should be identified and justified.
6 Regulatory Guide 1.172, "Software Requirements Specifications for Dig ital Computer Software Used in Safety Systems of Nuclear Power Plants," endorses IEEE Std 830-1993, "IEEE Recommended Practice for Soft ware Requirements Specifications." 7 Boris Beizer, Software Testing Techniques, Van Nostrand Reinhold, 1990. 8S. Seth et al., "High Integrity Software for Nuclear Power Plants: Candi date Guidelines, Technical Basis and Research Needs," NUREG/ CR-6263, June 1995.3. TEST PROGRAM RECORDS Criteria VI, "Document Control," and XVII, "Quality Assurance Records," as well as 10 CFR 21.51, require the control and retention of documents and records affecting quality. In addition, Criterion III, "Design Control," requires that design changes be sub ject to design control measures commensurate with those applied to the original design. Preservation of testing products is discussed in section 3.8.2(4) of IEEE Std 1008-1987.
Since design control measures must be applied to acceptance criteria for tests and since some software testing materials are frequently re-used and evolve during the course of software development and software maintenance (for example, regression test materials), such materials should be configuration items under change control of a software configuration management system.9 Additional information on this topic is provided in section A6 of Appendix A to IEEE Std 1008-1987.
4. INDEPENDENCE
IN SOFFWARE VERIFICATION
Criterion III, "Design Control," imposes an inde pendence requirement for the verification and checking of the adequacy of the design, requiring that those per sons who verify and check be different from those who accomplish the design. Therefore, independence is an additional requirement for software unit testing. Either those persons who establish the requirements-based elements for a software unit test must be different from those who designed or coded the software, or there must be independent review of the establishment of the requirements-based elements.
The guidance in section A7 of Appendix Ato IEEE Std 1008-1987 provides ac ceptable ways to meet this requirement for software unit testing. These independent persons must be suffi ciently competent in software engineering to ensure that software unit testing is adequately implemented.
5. OTHER STANDARDS
Section 1.3 of IEEE Std 1008-1987 references ANSI/IEEE Std 729-1983, "IEEE Standard Glossary of Software Engineering Terminology," and ANSI/ IEEE Std 829-1983, "IEEE Standard for Software Test Documentation." These referenced standards should be treated individually.
If a referenced standard has been incorporated sep arately into the NRC's regulations, licensees and appli cants must comply with that standard as set forth in the 9 Regulatory Guide 1.169 endorses IEEE Std 828-1990, "IEEE Standard for Software Configuration Management Plans," and IEEE Std 1042-1987, "IEEE Guide to Software Configuration Management," to provide guidance for general software configuration management plans and their implementation.
1.171-4 K
regulation.
If the referenced standard has been endorsed in a regulatory guide, the standard constitutes a method acceptable to the NRC staff of meeting a regulatory re> quirement as described in the regulatory guide. If a ref erenced standard has been neither incorporated into the NRC's regulations nor endorsed in a regulatory guide, licensees and applicants may consider and use the in formation in the referenced standard, if appropriately justified, consistent with current regulatory practice.
D. IMPLEMENTATION
The purpose of this section is to provide informa tion to applicants and licensees regarding the NRC staff's plans for using this regulatory guide. No backfit-ting is intended or approved in connection with the is suance of this proposed guide. Except in those cases in which an applicant pro poses an acceptable alternative method for complying with the specified portions of the NRC's regulations, the methods described in this guide will be used in the evaluation of submittals in connection with applica tions for construction permits and operating licenses.
This guide will also be used to evaluate submittals from operating reactor licensees that propose system modifi cations voluntarily initiated by the licensee if there is a clear nexus between the proposed modifications and this guidance.
BIBLIOGRAPHY
Beizer, Boris, Software Testing Techniques, Van Nos trand Reinhold, 1990. Hecht, H., A.T. Tai, K.S. Tso, "Class 1E Digital Sys tems Studies," NUREG/CR-6113, USNRC, October 1993.1 Institute of Electrical and Electronics Engineers, "Stan dard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations," IEEE Std 7-4.3.2, 1993.1 Lawrence, J.D., "Software Reliability and Safety in Nuclear Reactor Protection Systems," NUREG/ CR-6101 (UCRL-ID-117524, Lawrence Livermore National Laboratory), USNRC, November 1993.1 1Copies may be purchased at current rates from the U.S. Government Prin ting Offuie, P.O. Box 37082, Washington, DC 20402-9328 (telephone
(202)512-2249);
or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161. Copies are availabl" for inspection or copying for a fee from the NRC Public Docu went Room at 2120 L Street NW., Washington, DC; the PDR's mailing ad dress is Mail Stop LL-6, Washington, DC 20555-0001;
telephone
(202)634-3273;
fax (202)634-3343.
Lawrence, J.D., and G.G. Preckshot, "Design Factors for Safety-Critical Software," NUREG/CR-6294, USNRC, December 1994.1 Seth, S., et al., "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis and Research Needs," NUREG/CR-6263, USNRC, June 1995.1 USNRC, "Criteria for Digital Computers in Safety Systems of Nuclear Power Plants," Regulatory Guide 1.152, Revision 1, January 1996.2 USNRC, "Standard Review Plan," NUREG-0800, February 1984.1 2 Single copies of regulatory guides maybe obtained free ofcharge by writ ing the Office of Administration, Printing, Graphics and Distribution Branch, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001;
or by fax at (301)415-5272.
Copies are available for in spection or copying for a fee from the NRC Public Document Room at 2120 L Street NW, Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001;
telephone
(202)634-3273;
fax (202)634-3343.
1.171-6 1-.
REGULATORY
ANALYSIS A separate regulatory analysis was not prepared for this regulatory guide. The regulatory analysis prepared for Draft Regulatory Guide DG-1057, "Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," provides the regulatory basis for this guide. A copy of the regulatory analysis is available for inspection and copying for a fee at the NRC Public Document Room, 2120 L Street NW., Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001;
phone (202)634-3273;
fax (202)634-3343.
Federal Recycling Program 1.171-7 UNITED STATES NUCLEAR REGULATORY
COMMISSION
WASHINGTON, DC 20555-0001
2 FIRST CLASS MAIL / POSTAGE AND FEES PAIb USNRC '/ PERMIT NO. G-67 OFFICIAL BUSINESS PENALTY FOR PRIVATE USE, $300 co 0 ~0 -4 0 I .. a. z cr" "4'l