Regulatory Guide 1.171: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
Line 1: Line 1:
{{Adams
{{Adams
| number = ML13004A375
| number = ML003740108
| issue date = 07/31/2013
| issue date = 09/30/1997
| title = Software Unit Testing for Digitial Computer Software Used in Safety Systems of Nuclear Power Plants
| title = (Draft Was DG-1057) Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants
| author name = Sturzebecher K J
| author name =  
| author affiliation = NRC/RES/DE
| author affiliation = NRC/RES
| addressee name =  
| addressee name =  
| addressee affiliation =  
| addressee affiliation =  
| docket =  
| docket =  
| license number =  
| license number =  
| contact person = Orr M P
| contact person =  
| case reference number = DG-1208
| case reference number = DG-1057
| document report number = RG-1.171, Rev. 1
| document report number = RG-1.171
| package number = ML12354A534
| document type = Regulatory Guide
| document type = Regulatory Guide
| page count = 10
| page count = 8
}}
}}
{{#Wiki_filter:U.S. NUCLEAR REGULATORY COMMISSION
{{#Wiki_filter:U.S. NUCLEAR REGULATORY  
July 2013  Revision 1 REGULATORY GUIDE
COMMISSION
  OFFICE OF NUCLEAR REGULATORY RESEARCH
REGULATORY  
Technical Lead Karl Sturzebecher Written suggestions regarding this guide or development of new guides may be submitted through the NRC's public Web site under the Regulatory Guides document collection of the NRC Library at http://www.nrc.gov/reading
GUI OFFICE OF NUCLEAR REGULATORY  
-rm/doc-collections/reg
RESEARCH REGULATORY
-guides/contactus.html.    Electronic copies of this guide and other recently issued guides are available through the NRC public Web site under the Regulatory Guides document collection of the NRC Library at http://www.nrc.gov/reading
GUIDE 1.171 (Draft was DG-1057) SOFTWARE UNIT TESTING FOR DIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMS OF NUCLEAR POWER PLANTS  
-rm/doc-collections/
and through the NRC Agencywide Documents Access and Management System (ADAMS) at http://www.nrc.gov/reading
-rm/adams.html, under Accession No. ML13004A375.  The regulatory analysis may be found in ADAMS under Accession No. ML103120752 and the staff responses to the public comments on DG
-1208 may be found in ADAMS under Accession No. ML13004A370.      REGULATORY GUIDE 1.1
7 1 (Draft was issued as D G-120 8, dated August 2012)
  SOFTWARE UNIT TESTING FOR DIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMS OF
NUCLEAR POWER PLANTS


==A. INTRODUCTION==
==A. INTRODUCTION==
Purpose    This regulatory guide (RG) describes a method that the staff of the U.S.
In 10 CFR Part 50, "Domestic Licensing of Pro duction and Utilization Facilities," paragraph
55a(a)(1)  
requires, in part, 1 that systems and components be de signed, tested, and inspected to quality standards com mensurate with the safety function to be performed.


Nuclear Regulatory Commission (NRC) considers acceptable for use in complying with NRC regulations with respect to the software unit testing of digital computer software used in the safety systems of nuclear power plants.
Criterion
1, "Quality Standards and Records," of Ap pendix A, "General Design Criteria for Nuclear Power Plants," to 10 CFR Part 50 requires, in part, 1 that a qual ity assurance program be established and implemented in order to provide adequate assurance that systems and components important to safety will satisfactorily per form their safety functions.


Applicable Rules and Regulations The regulatory framework that the NRC has established for nuclear power plants consists of a number of regulations and supporting guidelines applicable to the software unit testing of digital computer softwar
Appendix B, "Quality As surance Criteria for Nuclear Power Plants and Fuel Re processing Plants," to 10 CFR Part 50 describes criteria that a quality assurance program for systems and com ponents that prevent or mitigate the consequences of postulated accidents must meet. In particular, besides the systems and components that directly prevent or mitigate the consequences of postulated accidents, the criteria of Appendix B also apply to all activities affect ing the safety-related functions of such systems and components as designing, purchasing, installing, test ing, operating, maintaining, or modifying.


====e. Title====
A specific l1n this regulatory guide, many of the regulations have been paraphrased;
10, of the Code of Federal Regulations
see 10 CFR Part 50 for the full text.requirement is contained in 10 CFR 50.55a(h), which requires that reactor protection systems satisfy the cri teria of IEEE Std 279-1971, "Criteria for Protection Systems for Nuclear Power Generating Stations." 2 Paragraph
, Part 50, "Domestic Licensing of Production and Utilization Facilities"
4.3 of IEEE Std 279-19713 states that quali ty of components is to be achieved through the specifi cation of requirements known to promote high quality, such as requirements for design, inspection, and test.  Many of the criteria in Appendix B to 10 CFR Part 50 contain requirements closely related to testing activities.
(10 CFR Part 50) (Ref. 1), Appendix A, "General Design Criteria for Nuclear Power Plants," General Design Criterion (GDC)
1, "Quality Standards and Records," requires, in part, that quality standards be established and implemented to provide adequate assurance that systems and components important to safety will satisfactorily perform their safety function


====s.  GDC====
Criterion I, "Organization," requires the es tablishment and execution of a quality assurance pro gram. Criterion H, "Quality Assurance Program," re quires, in part, that the program take into account the need for special controls, processes, test equipment, tools, and skills to attain the required quality, as well as the need for verification of quality by inspection and test. Criterion III, "Design Control," requires, in part, that measures be established for verifying and checking the adequacy of design, such as by the performance of a 2 Revision I of Regulatory Guide 1.153, "Criteria for Safety Systems," en dorses IEEE Std 603-1991,"Criteria for Safety Systems for Nuclear Pow er Generating Stations," as a method acceptable to the NRC staff for satis fying the NRC's regulations with respect to the design, reliability, qualifi cation, and testability of the power, instrumentation, and control portions of the safety systems of nuclear power plants.  3 IEEE publications may be obtained from the IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854.7.USNRCREGULATORYGUIDES
21, "Protection System Reliability and Testability," requires, in part, that the protection system be designed for high functional reliability. Appendix B, "Quality Assurance Criteria for Nuclear Power Plants and Fuel Reprocessing Plants," to 10 CFR Part 50 describes criteria that a quality assurance program for systems and components that prevent or mitigate the consequences of postulated accidents must meet. In particular, in addition to the systems and components that directly prevent or mitigate the consequences of postulated accidents, Appendix B criteria also apply to all activities affecting the safety
The guides we Issued in the following ten broad divisions:
-related functions of those systems and components; these activities includ e designing, purchasing , installing, testing , operatin g, maintaining , repairing or modifying.    In 10 CFR 50.55a(a)(1) requires, in part, that systems and components be designed, fabricated, erected, tested, and inspected to quality standards commensurate with the safety function to be performe
Reglatory Guides are Issued to descibe and make avlable tothe public such Informs lion as methods acceptable to the NRC staf for Implementing specific pans of the Com- 1. Power Reactors 6. Products mission's regulations, techniques usedbythestaff inevaluating specific problems orpos- 2Z Research and Test Reactors 7. Transportation tulated accdentsa and data needed by the NRC staff In Its review of ap:icationrs forper- 3& Fuels and Materials Facilities
8. Occupations!
Health mits and licensea.


====d. The regulation in====
Regulatory guides are not sstitutes for regulations, and compiance
10 CFR 50.55a(h) requires that reactor protection systems satisfy the criteria in Institute of Electrical and Electronics Engineers (IEEE) Standard (Std.)
4. Environmental and Siting 9. Antitust and Financial Review with them Is not required.
603-1991, "IEEE Standard Criteria for Safety Systems for Nuclear Power Generating Stations," including a correction sheet dated January 30, 1995 (Ref. 2), or in IEEE Std.


279-1971, "Criteria for Protection Systems for Nuclear Power RG 1.1 7 1, Rev. 1, Page
Methods and solutions different from those set out in theguides
2 Generating Stations" (Ref.
5. Matrials and Plant Protection  
10. General will be acceptable If they provide a basis for the findings requisite to the Issuance or con linuance of a permit or license by the Commission.


3).  These criteria shall be part of the evaluation of the recognized quality codes and standards selected for their applicability, adequacy, and sufficiency and shall be supplemented or modified as needed to assure a quality product and that it will perform the required safety function.  The guidance on the safety systems equipment employing digital computers, and programs or firmware requires quality standards be used for testing software units.
Single copies of regulatory guides may be obtained free of chrge bywrlfing te Printing.


This RG endorses American National Standards Institute (ANSI)/IEEE Std.
This guide was lesu after consideration of comments received from thre public. Com- Graphics anid Distribution Branch. Office of Administrtion, U.S. Nuclear Regulatory Com ments andsuggestions for inprovements Inthese guides wencosurged at all Imes, and mission, Washington, DC 2055-0001;
or by fox at (301)415-5272 gue will be revised, as appropriate, to accommodate comments and to reflect new in= on or aperience.


1008-1987, "IEEE Standard for Software Unit Testing" (Ref.
Issued guides may also bepurchased!
from me* National Technical Information Service on Whitten comments may be submitted to te Rules Review and Directives Branch, DFIPS, a standing order basis. Details on this service may be obtained by writing NTIS, 5285 Port ADM, U.S. Nuclear Regulatory Commission, Washington, DC 2055-0001.


4), with the clarifications and exceptions as described in Section C, "Staff Regulatory Guidance
Royal Road, Springfield, VA 2216
."  ANSI/IEEE Std. 1008-1987, which was reaffirmed in 2002, describes a method acceptable to the NRC staff for complying with NRC regulations for promoting high functional reliability and design quality in the software used in safety systems.


1   In particular, the method is consistent with the previously cited GDC in Appendix A to 10 CFR Part 50 and the criteria for quality assurance programs in Appendix B to 10 CFR Part 50 as they apply to software unit testing. The criteria in Appendices A and B to 10 CFR Part 50 apply to systems and related quality assurance processes, and the requirements extend to the software elements if those systems include software.
===1. DE September ===
1997 suitable testing program, and that design control measures be applied to items such as the delineation of acceptance criteria for inspections and tests. Criterion V, "Instructions, Procedures, and Drawings," requires activities affecting quality to be prescribed by docu mented instructions, procedures, or drawings of a type appropriate to the circumstances and that these activi ties be accomplished in accordance with these instruc tions, procedures, or drawings.


Purpose of Regulatory Guides    The NRC issues RGs to describe methods that the staff considers acceptable for use in implementing specific parts of NRC regulations, to explain techniques that the staff uses in evaluating specific problems or postulated accidents, and to provide guidance to applicants.  However RGs are not substitutes for regulations and compliance with them is not required.  The information provided by this RG is also in the Standard Review Plan, NUREG
Criterion V further re quires that instructions, procedures, and drawings include appropriate quantitative or qualitative accep tance criteria for determining that important activities have been satisfactorily accomplished.
-0800, "Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants:
  LWR Edition," Chapter 7, "Instrumentation and Controls," (Ref. 5).  The NRC staff uses the NRC Standard Review Plan to review 10 CFR Part 50 and 10 CFR Part 52, "Licenses, Certifications, and Approvals for Nuclear Power Plants," (Ref. 6) license applications.


Paperwork Reduction Act This RG contains information collection requirements covered by 10 CFR Part 50 and 10 CFR Part 52 that the Office of Management and Budget (OMB) approved under OMB control numbers 3150
Criterion XI, "Test Control," requires establishment of a test pro gram to ensure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorpo rate the requirements and acceptance limits contained in applicable design documents.
-0011 and 3150
-0151, respectively.  The NRC may neither conduct nor sponsor, and a person is not required to respond to, an information collection request or requirement unless the requesting document displays a currently valid OMB control number.


==B. DISCUSSION==
Test procedures must include provisions for ensuring that all prerequisites for the given test have been met, that adequate test instru mentation is available and used, and that the test is per formed under suitable environmental conditions.
Reason for Revision Both the original version of this RG and this revision endorse ANSI/IEEE Std. 1008
-1987.  Subsequently, associated software RGs and various related software standards have been developed or updated and this revision of RG
1.171 is being updated to be consistent with these standards and related guidance.  The applicant or licensee should consider the hierarchy guidance of these different RGs and standards that relate to the software development process and unit testing. For example, RG 1.170,                                       


1 The term "safety systems" is synonymous with "safety
Crite rion XI also requires that test results be documented and evaluated to assure that test requirements have been sat isfied. Finally, Criteria VI, "Document Control," and XVII, "Quality Assurance Records," provide for the control of the issuance of documents, including changes thereto, that prescribe all activities affecting quality and provide for the maintenance of sufficient records to furnish evidence of activities affecting quali ty. The latter requires test records to identify the inspec tor or data recorder, the type of observation, the results, the acceptability of the results, and the action taken in connection with any deficiencies noted. This regulatory guide endorses ANSI/IEEE  
-related systems." The scope of the GDC includes systems, structures, and components "important to safety." However, the scope of this regulatory guide is limited to "safety systems," which are a subset of "systems important to safety."  Although not specifically scoped to include non
Std 1008-1987, "IEEE Standard for Software Unit Test ing," 3 with the exceptions stated in the Regulatory Position.
-safety-related but "important to safety systems" this regulatory guide provides methods that the staff finds appropriate for the design, development and implementation of all important to safety systems. The NRC may apply this guidance in licensing reviews of non
-safety but important to safety digital software and may tailor it to account for the safety significance of the system software
.   
RG 1.1 7 1, Rev. 1, Page
3 "Software Test Documentation for Digital Computer Software Used in Safety Systems of Nuclear Power Plants" (Ref. 7), endorses IEEE Std. 829
-2008, "IEEE Standard for Software and System Test Documentation" (Ref. 8), and provides an approach that the NRC staff considers acceptable for meeting the requirements of 10 CFR Part 50 as they apply to the test documentation, including unit test documentation, of safety system software.  The endorsed consensus standard, ANSI/IEEE Std.


1008-1987, defines a method for planning, preparing, conducting, and evaluating software unit testing that is consistent with the previously cited regulatory requirements as they apply to safety system software.
IEEE Std 1008-1987 describes a method ac ceptable to the NRC staff for complying with parts of the NRC's regulations for promoting high functional reliability and design quality in software used in safety systems.4 In particular, the method is consistent with the previously cited General Design Criteria and the criteria for quality assurance programs of Appendix B as they apply to software unit testing. The criteria of Appendices A and B apply to systems and related quali 4 The term "safety systems" is synonymous with "safety-related systems." The General Design Criteria cover systems, structures, and components "important to safety." The scope of this regulatory guide is, however, lim ited to "safety systems," which are a subset of "systems important to safety..ty assurance processes, and if those systems include software, the requirements extend to the software ele ments.  In general, information provided by regulatory guides is reflected in the Standard Review Plan (NUREG-0800).
The Office of Nuclear Reactor Regu lation uses the Standard Review Plan to review applica tions to construct and operate nuclear power plants.  This regulatory guide will apply to the revised Chapter 7 of that document.


Background The use of industry consensus standards, such as IEEE standards, is part of an overall approach to meet the requirements of 10
The information collections contained in this regu latory guide are covered by the requirements of 10 CFR Part 50, which were approved by the Office of Manage ment and Budget, approval number 3150-0011.
CFR Part 50 when developing safety systems for nuclear power plants.  Compliance with these standards does not guarantee that regulatory requirements will be met.  However, compliance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assurance processes used to design safety systems.  These practices are based on past experience and represent industry consensus on approaches used for the development of such systems.


This RG refers to software incorporated into the instrumentation and control systems covered by Appendix B to 10 CFR Part 50 as "safety system software."  For safety system software, software testing is an important part of the effort to comply with NRC regulations.  Software engineering practices rely, in part, on software testing to meet general quality and reliability requirements consistent with GDC
The NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information un less it displays a currently valid OMB control number.
1 and 21 of Appendix A to 10 CFR Pa rt 50, as well as Criteria I, II, III, V, VI, XI, and XVII of Appendix B.    Several criteria in Appendix B to 10 CFR Part 50 contain requirements closely related to testing activities.  These listed criteria are only part of and not the entire requirement
:  Criterion I, "Organization," requires, in part, the establishment and execution of a quality assurance program.


Criterion II, "Quality Assurance Program," requires, in part, that the program take into account the need for (1)
==B. DISCUSSION==
special controls, processes, test equipment, tools, and skills necessary to attain the required quality and (2) the verification of quality through inspections and tests.
The use of industry consensus standards is part of an overall approach to meeting the requirements of 10 CFR Part 50 when developing safety systems for nuclear power plants. Compliance with standards does not guarantee that regulatory requirements will be met.  However, compliance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assur ance processes used to design safety systems. These practices are based on past experience and represent in dustry consensus on approaches used for development of such systems.
 
Criterion III, "Design Control," requires, in part, that measures be established for verifying and checking the adequacy of the design (e.g., through the performance of a suitable testing program) and that design control measures be applied to items such as the delineation of acceptance criteria for inspections and tests.
 
Criterion V, "Instructions, Procedures, and Drawings," requires, in part, activities affecting quality be prescribed by documented instructions, procedures, or drawings of a type appropriate to the circumstances and that these activities be accomplished in accordance with these instructions, procedures, or drawings.  Criterion V further requires that instructions, procedures, and drawings include appropriate quantitative or qualitative acceptance criteria for determining that important activities have been satisfactorily accomplished.
 
RG 1.1 7 1, Rev. 1, Page
4 Criterion VI, "Document Control," requires, in part, that all documents that prescribe activities affecting quality, such as instructions, procedures, and drawings, be subject to controls that ensure that documents, including changes, are reviewed for adequacy and approved for release by authorized personnel.
 
Criterion XI, "Test Control," requires, in part, establishment of a test program to assure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorporate the requirements and acceptance limits contained in applicable design documents.  Test procedures must include provisions for ensuring that all prerequisites for the given test have been met, that adequate test instrumentation is available and used, and that the test is performed under suitable environmental conditions.  Criterion XI also requires that test results be documented and evaluated to ensure that test requirements have been satisfied.
 
Criterion XVII, "Quality Assurance Records," requires, in part, that sufficient records be maintained so that data that are closely associated with the qualifications of personnel, procedures, and equipment are identifiable and retrievable.  Test records must identify the inspector or data recorder, the type of observation made, the results, the acceptability of the results, and the action taken in connection with any noted deficiencies.
 
Related Guidance Current practice for the development of software for safety
-related applications includes the use of a software life
-cycle process that incorporates software testing activities (e.g., IEEE
Std. 1074-2006, "IEEE Standard for Developing a Software Life Cycle Process" (Ref.
 
9), as endorsed by RG 1.173, "Developing Software Life Cycle Processes for Digital Computer Software Used in Safety Systems of Nuclear Power Plants" (Ref.
 
10)).  Software testing, including software unit testing, is a key element in software verification and validation (V&V) activities, as indicated by IEEE
Std. 1012-2004, "IEEE Standard for Software Verification and Validation" (Ref.


11), and IEEE
Software incorporated into instrumentation and control systems covered by Appendix B will be referred to in this regulatory guide as safety system software.
Std. 7-4.3.2-2003, "IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations"
(Ref. 12).  Software testing consists of several levels.  NUREG/CR
-6101, "Software Reliability and Safety in Nuclear Reactor Protection Systems," issued November
1993 (Ref.


13), and NUREG/CR
For safety system software, software testing is an im portant part of the effort to achieve compliance with the NRC's requirements.
-6263, "High Integrity Software for Nuclear Power Plants:  Candidate Guidelines, Technical Basis, and Research Needs," issued June
1995 (Ref.


14), provide a common approach to software testing.  This approach includes a three
Software engineering practices rely, in part, on software testing to meet general quality and reliability requirements consistent with Criteria 1 and 21 of Appendix A to 10 CFR Part 50, as well as Cri teria I, II, III, V, VI, XI, and XVII of Appendix B.  The consensus standard, IEEE Std 1008-1987 (reaffirmed in 1993), defines a method for planning, preparing for, conducting, and evaluating software unit testing. The method described is consistent with the previously cited regulatory requirements as they apply to safety system software.
-level test program to help ensure quality in a complex software product or a complex set of cooperating software products (i.e., unit
-level testing, integration
-level testing and system
-level testing such as system validation tests or acceptance tests).  ANSI/IEEE
Std. 1008-1987 delineates an approach to the unit testing of software that assumes a larger context established by V&V planning and general planning for the application of the full range of testing activities.  This context may be defined, for example, in IEEE
Std. 1012-2004, as endorsed by RG 1.168, "Verification, Validation, Reviews, and Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants" (Ref.


15). Therefore, software unit testing that licensees perform in accordance with ANSI/IEEE Std.
Current practice for the development of software for high-integrity applications includes the use of a software life cycle process that incorporates software testing activities, e.g., IEEE Std 1074-1991, "IEEE Standard for Developing Software Life Cycle 1.171-2 ,
Processes." 3 Software testing, including software unit testing, is a key element in software verification and validation activities, as indicated by IEEE Std 1012-1986, "IEEE Standard for Software Verification and Validation Plans," 3 and IEEE Std 7-4.3.2-1993, "Standard Criteria for Digital Computers in Safety Sys tems of Nuclear Power Generating Stations." A com mon approach to software testing [NUREG/CR-6101, "Software Reliability and Safety in Nuclear Reactor Protection Systems" (November
1993); NUREG/ CR-6263, "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis and Re search Needs" (June 1995)]5 utilizes a three-level test program to help ensure quality in a complex software product or complex set of cooperating software prod ucts, i.e., unit-level testing, integration-level testing, and system-level testing such as system validation tests or acceptance tests. IEEE Std 1008-1987 delineates an approach to the unit testing of software that is based on the assumption of a larger context established by verifi cation and validation (V&V) planning as well as general planning for the full range of testing activities to be applied. Therefore, software unit testing per formed in accordance with IEEE Std 1008-1987 should be consistent with planning information estab lished in V&V plans and higher-level software test plans, although that planning information is not within the scope of IEEE Std 1008-1987.


1008-1987 should be consistent with the planning information established in V&V plans and higher level software test plans, although that planning information is not within the scope of ANSI/IEEE Std.
C. REGULATORY
POSITION The requirements in ANSI/IEEE
Std 1008-1987, "IEEE Standard for Software Unit Testing," provide an approach acceptable to the NRC staff for meeting the requirements of 10 CFR Part 50 as they apply to the unit testing of safety system software, subject to the provi sions listed below. The appendices to IEEE Std 1008-1987 are not endorsed by this regulatory guide except as noted below. Appendix A to this standard pro vides guidance regarding the implementation of the software unit testing approach, and Appendix B to the standard provides context regarding software engineer ing information and testing assumptions that underlie the software unit testing approach.


1008-1987. This RG is based on standards and describes methods acceptable for any safety system software, and discusses the required V&V activities. The applicant or licensee determines how the required activities will be implemented.
To meet the requirements of 10 CFR 50.55a(h)
and Appendix A to 10 CFR Part 50 as assured by complying with the criteria of Appendix B to 10 CFR Part 50 ap 5 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC 20402-9328 (telephone
(202)512-2249);
or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161. Copies are I available for inspection or copying for a fee from the NRC Public Docu ment Room at 2120 LStreet NW., Washington, DC; the PDR's mailing ad dress is Mail Stop LL-6, Washington, DC 20555-0001;
telephone
(202)634-3273;
fax (202)634-3343.


RG 1.1 7 1, Rev. 1, Page
plied to the unit testing of safety system software, the following exceptions are necessary and will be consid ered by the NRC staff in the review of submittals from licensees and applicants. (In this section, the cited crite ria are in Appendix B to 10 CFR Part 50 unless other wise noted.) 1. SOFTWARE TESTING DOCUMENTATION
5 Harmonization with International Standards The International Atomic Energy Agency (IAEA) has established a series of safety guides and standards constituting a high level of safety for protecting people and the environment.  IAEA safety guides are international standards to help users striving to achieve high levels of safety.  Pertinent to this RG, IAEA Safety Guide NS
Criterion XI, "Test Control," requires that a test program be established to ensure that all testing re quired to demonstrate that systems and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorporate requirements and acceptance limits contained in applicable design documents.
-G-1.1, "Software for Computer Based Systems Important to Safety in Nuclear Power Plants" issued September 2000 (Ref. 16) discusses the importance of unit testing for computer software used in safety related systems.  This RG incorporates similar unit testing recommendations and is consistent with the basic principles provided in IAEA Safety Guide NS
-G-1.1.    Documents Discussed in Staff Regulatory Guidance This RG endorses, in part, the use of one or more codes or standards developed by external organizations, and other third party guidance documents.  These codes, standards and third party guidance documents may contain references to other codes, standards or third party guidance documents ("secondary references").  If a secondary reference has itself been incorporated by reference into NRC regulations as a requirement, then licensees and applicants must comply with that standard as set forth in the regulation.  If the secondary reference has been endorsed in a RG as an acceptable approach for meeting an NRC requirement, then the standard constitutes a method acceptable to the NRC staff for meeting that regulatory requirement as described in the specific RG.  If the secondary reference has neither been incorporated by reference into NRC regulations nor endorsed in a RG, then the secondary reference is neither a legally
-binding requirement nor a "generic" NRC approval as an acceptable approach for meeting an NRC requirement.  However, licensees and applicants may consider and use the information in the secondary reference, if appropriately justified and consistent with current regulatory practice, consistent with applicable NRC requirements.      C.  STAFF REGULATORY GUIDANCE
  The requirements in ANSI/IEEE
Std. 1008-1987 provide an acceptable approach for meeting NRC regulatory requirements on the unit testing of safety system software with the exceptions and additions listed in these regulatory positions. In this section of the guide, the cited criterion refers to Appendix B to 10 CFR Part 50 unless otherwise noted. This RG does not endorse the appendices to ANSI/IEEE Std. 1008-1987, except as noted below.


1. Software Testing Documentation Section 1.1 of ANSI/IEEE
Criterion I, "Organization," Criterion II, "Quality Assurance Pro gram," Criterion III, "Design Control," Criterion V, "Instructions, Procedures, and Drawings," Criterion VI, "Document Control," and Criterion XVIi, "Quality Assurance Records," contain requirements bearing on information associated with testing. IEEE Std 1008-1987, in section 1.1, mandates the use of the Test Design Specification and the Test Summary Report de fined by ANSI/IEEE  
Std. 1008-1987 mandates the use of the test design specifications and the test summary report documents, which can be found in IEEE
Std 829-1983, "IEEE Standard for Software Test Documentation." In addition, IEEE Std 1008-1987 either incorporates additional informa tion into these two documents or indicates the need for additional documents.
Std. 829-2008, Clauses 10, "Level Test Design" and 17, "Master Test Report"
respectively.  In addition, ANSI/IEEE
Std. 1008-1987 either incorporates additional information into these two documents or indicates the need for additional documentation.  Regardless of whether the licensee uses these two documentation formats, its documentation to support software unit testing (either documentation used directly in the software unit testing activity or documentation of the overall testing effort) should include information necessary to meet regulatory requirements as applied to software test documentation. As a minimum, this information should include the following:
  a. qualifications, duties, responsibilities, and skills required of persons and organizations assigned to testing activities;


RG 1.1 7 1, Rev. 1, Page
Regardless of whether these two documentation formats are used, the documentation used to support software unit testing (either documen tation used directly in the software unit testing activity or documentation of the overall testing effort) must in clude information necessary to meet regulatory re quirements as applied to software test documentation.
6 b. special conditions and controls, equipment, tools, and instrumentation needed for the accomplishment of testing;


c. test instructions and procedures that incorporate the requirements and acceptance limits in applicable design documents;
As a minimum, this information includes:
  d. test prerequisites and the criteria for meeting these requirements and acceptance limits;
"* Qualifications, duties, responsibilities, and skills required of persons and organizations assigned to testing activities, " Environmental conditions and special controls, equipment, tools, and instrumentation needed for the accomplishment of testing, " Test instructions and procedures incorporating the requirements and acceptance limits in applicable design documents, " Test prerequisites and the criteria for meeting them, "* Test items and the approach taken by the testing program, "* Test logs, test data, and test results, "* Acceptance criteria, 1.171-3 Test records indicating the identity of the tester, the type of observation, the results and acceptability, and the action taken in connection with any deficiencies.
  e. test items and the approach taken by the testing program;


f. test logs, test data, and test results;
Any of the above information items that are not present in the documentation selected to support soft ware unit testing must be incorporated as additional items.  2. TEST PROGRAM Criterion XI, "Test Control," requires establish ment of a test program to ensure that all testing required to demonstrate that structures, systems, and compo nents will perform satisfactorily in service is identified and performed in accordance with written test proce dures that incorporate the requirements and acceptance limits contained in applicable design documents.
  g. acceptance criteria; and h. test records that indicate the identity of the tester, the type of observation made, the results and acceptability, and the action taken in connection with any deficiencie s.  The licensee should incorporate any information regarding the items listed above that are not in the documentation selected to support software unit testing as additional items.


2. Test Program The coverage of requirements and the internal structure of the code are two particularly important aspects of test coverage necessary for the unit testing of safety system software, as follows:
The two aspects of test coverage that are particularly impor tant for the unit testing of safety system software are coverage of requirements and coverage of the internal structure of the code.  2.1 Coverage of Requirements For safety system software, those requirements identified as essential to the safety determination
  a. Coverage of Requirements. The testing should include all features and associated procedures, states, state transitions, and associated data characteristics essential to the safety determination.
6 must be tested. Section 3.2.2(5) of IEEE Std 1008-1987 sug gests consideration of expected use of the unit in the de termination of features to be tested. All features and as sociated procedures, states, state transitions, and associated data characteristics essential to the safety de termination must be included in the testing.


b. Coverage of Module Structur
2.2 Coverage of Internal Structure Section 3.1.2(2) of IEEE Std 1008-1987 specifies statement coverage (covering each source language statement with a test case) as a criterion for measuring the completeness of the software unit testing activity.


====e. Section====
Statement coverage is a very weak criterion for meas uring test completeness  
3.1.2(2) of ANSI/IEEE
[See Beizer 7 and NUREG/ CR-6263 8]. Therefore, the staff does not endorse state ment coverage as a sufficient coverage criterion for software unit testing. For safety system software, the unit test coverage criteria to be employed should be identified and justified.
Std. 1008-1987 specifies statement coverage (covering each source language statement with a test case) as a criterion for measuring the completeness of software unit testing.  The staff believes that statement coverage is an insufficient criterion for measuring test completeness (Ref. 14 and Ref. 1
7). Therefore, the staff does not endorse statement coverage as a sufficient criterion for software unit testing. For safety system software, the licensee should identify and justify the unit testing coverage criteria that it will use.


3. Test Program Records Criteria VI and XVII and 10
6 Regulatory Guide 1.172, "Software Requirements Specifications for Dig ital Computer Software Used in Safety Systems of Nuclear Power Plants," endorses IEEE Std 830-1993, "IEEE Recommended Practice for Soft ware Requirements Specifications." 7 Boris Beizer, Software Testing Techniques, Van Nostrand Reinhold, 1990.  8S. Seth et al., "High Integrity Software for Nuclear Power Plants: Candi date Guidelines, Technical Basis and Research Needs," NUREG/ CR-6263, June 1995.3. TEST PROGRAM RECORDS Criteria VI, "Document Control," and XVII, "Quality Assurance Records," as well as 10 CFR 21.51, require the control and retention of documents and records affecting quality. In addition, Criterion III, "Design Control," requires that design changes be sub ject to design control measures commensurate with those applied to the original design. Preservation of testing products is discussed in section 3.8.2(4) of IEEE Std 1008-1987.
CFR 21.51, "Maintenance and Inspection of Records" (Ref.


1 8), require the control and retention of documents and records that affect quality.  In addition, Criterion III requires the licensee to subject design changes to design control measures commensurate with those that it applied to the original desig
Since design control measures must be applied to acceptance criteria for tests and since some software testing materials are frequently re-used and evolve during the course of software development and software maintenance (for example, regression test materials), such materials should be configuration items under change control of a software configuration management system.9 Additional information on this topic is provided in section A6 of Appendix A to IEEE Std 1008-1987.


====n. Section====
===4. INDEPENDENCE ===
3.8.2(4) of ANSI/IEEE
IN SOFFWARE VERIFICATION
Std. 1008-1987 discusses the preservation of testing products.  Included with the testing products preservation are the test iterations caused by any task, procedure and design criteria variations or deviations from the original IEEE Std. 829-2008, Clauses 8 and 9, "Master Test Plan" and "Level Test Plans."  Since the design control measures are required for testing acceptance criteria and because some software testing materials are frequently reused and evolve during the course of software development and software maintenance (e.g., regression test materials), such materials should be configuration items under the change control of a software configuration management system.
Criterion III, "Design Control," imposes an inde pendence requirement for the verification and checking of the adequacy of the design, requiring that those per sons who verify and check be different from those who accomplish the design. Therefore, independence is an additional requirement for software unit testing. Either those persons who establish the requirements-based elements for a software unit test must be different from those who designed or coded the software, or there must be independent review of the establishment of the requirements-based elements.


RG 1.1 7 1, Rev. 1, Page
The guidance in section A7 of Appendix Ato IEEE Std 1008-1987 provides ac ceptable ways to meet this requirement for software unit testing. These independent persons must be suffi ciently competent in software engineering to ensure that software unit testing is adequately implemented.
7 4. Independence Software Verification Criterion III imposes an independence requirement for the verification and checking of the adequacy of the design.  IEEE Std. 1008
-1987 does not include a requirement for independent software verification.  The RG 1.168 provides additional guidance on testing independence.


5. References to ANSI/IEEE
===5. OTHER STANDARDS ===
Std. 829-1983    ANSI/IEEE Std. 1008-1987 includes references to ANSI/IEEE
Section 1.3 of IEEE Std 1008-1987 references ANSI/IEEE  
Std. 829-1983; however, ANSI/IEEE Std. 829-1983 has been revised since the publication of ANSI/IEEE
Std 729-1983, "IEEE Standard Glossary of Software Engineering Terminology," and ANSI/ IEEE Std 829-1983, "IEEE Standard for Software Test Documentation." These referenced standards should be treated individually.
Std.1008-1987.  With the new ANSI/IEEE Std. 829
-2008 revision there are added levels of test documentation for the licensee and applicant to consider, which also includes unit test documentation.  Thus IEEE Std. 829
-2008, which is endorsed by RG 1.170, should be used.


6. Annexes  ANSI/IEEE Std. 1008-1987 contains the following informative appendixes listed below.  These appendixes are listed here as sources of information; they have not received regulatory endorsement unless otherwise noted:
If a referenced standard has been incorporated sep arately into the NRC's regulations, licensees and appli cants must comply with that standard as set forth in the 9 Regulatory Guide 1.169 endorses IEEE Std 828-1990, "IEEE Standard for Software Configuration Management Plans," and IEEE Std 1042-1987, "IEEE Guide to Software Configuration Management," to provide guidance for general software configuration management plans and their implementation.
  Appendix A, "Implementation and Usage Guidelines," provides some additional guidance on using the standard.  Although this is a useful introduction to several topics, the NRC does not endorse the appendix because it does not provide sufficient guidance on how to perform specific activities.


Appendix B, "Concepts and Assumptions," contains a variety of topics that relate unit testing to software engineering in general and that discuss testing assumptions. This appendix is helpful but out of date; therefore, the NRC does not endorse it.    Appendix C, "Sources for Techniques and Tools," lists documents that relate to unit testing.  This list is out of date; therefore, the NRC does not endorse it.
1.171-4 K
regulation.


Appendix D, "General References," lists a basic set of references on software testing. This list is out of date; therefore, the NRC does not endorse it.
If the referenced standard has been endorsed in a regulatory guide, the standard constitutes a method acceptable to the NRC staff of meeting a regulatory re> quirement as described in the regulatory guide. If a ref erenced standard has been neither incorporated into the NRC's regulations nor endorsed in a regulatory guide, licensees and applicants may consider and use the in formation in the referenced standard, if appropriately justified, consistent with current regulatory practice.


==D. IMPLEMENTATION==
==D. IMPLEMENTATION==
The purpose of this section is to provide information on how applicants and licensees
The purpose of this section is to provide informa tion to applicants and licensees regarding the NRC staff's plans for using this regulatory guide. No backfit-ting is intended or approved in connection with the is suance of this proposed guide.  Except in those cases in which an applicant pro poses an acceptable alternative method for complying with the specified portions of the NRC's regulations, the methods described in this guide will be used in the evaluation of submittals in connection with applica tions for construction permits and operating licenses.
2  may use this guide and information about NRC
plans for using this RG. In addition, it describes how the staff complies with 10 CFR 50.109, "Backfitting" and any applicable finality provisions in 10 CFR Part 52, "Licenses, Certifications, and Approvals for Nuclear Power Plants."
Use by Applicants and Licensees Applicants and licensees may voluntarily
3                                       
 
2  In this section, "licensees" refers to licensees of nuclear power plants under 10 CFR Parts 50 and 52; and the term "applicants" refers to applicants for licenses and permits for (or relating to) nuclear power plants under 10 CFR Parts
50 and 52, and applicants for standard design approvals and standard design certifications under 10 CFR Part 52.
 
use the guidance in this document to demonstrate compliance with the underlying NRC regulations.  Methods or solutions that differ from those described
 
RG 1.1 7 1, Rev. 1, Page
8 in this RG may be deemed acceptable if they provide sufficient basis and information for the staff to verify that the proposed alternative demonstrates compliance with the appropriate NRC regulations.  Current licensees may continue to use guidance the NRC found acceptable in the past to comply with the identified regulations, as long as their current licensing basis remains unchanged.
 
Licensees may use the information in this RG for actions which do not require NRC review and approval such as changes to a facility design under 10 CFR 50.59, "Changes, Tests, and Experiments."  Licensees may use the information in this RG or applicable parts to resolve regulatory or inspection issues.    Additionally, an existing applicant may be required to adhere to new rules, orders, or guidance if 10 CFR 50.109(a)(3) applies.
 
If a licensee believes that the NRC either is using this RG or requesting or requiring the licensee to implement the methods or processes in this RG in a manner inconsistent with the discussion in this implementation section, then the licensee may file a backfit appeal with the NRC in accordance with the guidance in NUREG
-1409, "Backfitting Guidelines," (Ref.
 
1 9) and the NRC Management Directive 8.4, "Management of Facility
-Specific Backfitting and Information Collection" (Ref.
 
20).      Use by NRC Staff During regulatory discussions on plant
-specific operational issues, the staff may discuss with licensees various actions consistent with staff positions in this RG, as one acceptable means of meeting the underlying NRC regulatory requirement.  Such discussions would not ordinarily be considered backfitting, even if prior versions of this RG are part of the licensing basis of the facility.  However, unless this RG is part of the licensing basis for a facility, the staff may not represent to the licensee that the licensee's failure to comply with the positions in this RG constitutes a violation.
 
If an existing licensee voluntarily seeks a license amendment or change and (1) the staff's consideration of the request involves a regulatory issue directly relevant to this new or revised RG, and (2) the specific subject matter of this RG is an essential consideration in the staff's determination of the acceptability of the licensee's request, then the staff may request that the licensee either follow the guidance in this RG or provide an equivalent alternative process that demonstrates compliance with the underlying NRC regulatory requirements.  This action is not considered backfitting as defined in 10 CFR
50.109(a)(1) or a violation of any of the issue finality provisions in 10 CFR Part 52.
 
The NRC staff does not intend or approve any imposition or backfitting of the guidance in this RG.  The staff does not expect any existing licensee to use or commit to using the guidance in this RG , unless the licensee makes a change to its licensing basis.  The staff does not expect or plan to request licensees to voluntarily adopt this RG to resolve a generic regulatory issue.  The staff does not expect or plan to initiate NRC regulatory action that would require the use of this RG.  Examples of such unplanned NRC regulatory actions include issuance of an order requiring the use of the RG, requests for information under 10 CFR 50.54(f) as to whether a licensee intends to commit to use of this RG , generic communication, or promulgation of a rule requiring the use of this RG without further backfit consideration.
 
3  In this section, "voluntary" and "voluntarily" mean that the licensee is seeking the action of its own accord, without the force of a legally binding requirement or an NRC representation of further licensing or enforcement action.
 
RG 1.1 7 1, Rev. 1, Page
9 REFERENCES
4    1. U.S. Code of Federal Regulations (CFR) "Domestic Licensing of Production and Utilization Facilities, Part 50, Chapter 1, Title 10, "Energy." 
  2. Institute of Electrical and Electronic Engineers (IEEE), Std. 603
-1991, "IEEE Standard Criteria for Safety Systems for Nuclear Power Generating Stations," Piscataway, NJ, 1991 (including a correction sheet dated January 30, 1995).
5 3.  IEEE, Std. 279
-1971, "Criteria for Protection Systems for Nuclear Power Generating Stations," Piscataway, NJ, 1971.
 
4. IEEE Std. 1008
-1987, "IEEE Standard for Software Unit of Testing," Piscataway, NJ, 1987.
 
5. U. S. Nuclear Regulatory Commission (NRC), NUREG
-0800, "Standard Review Plan for the Review of Safety Analysis Reports for Nuclear Power Plants," Chapter 7, "Instrumentation and Controls," Washington, DC. (http://www.nrc.gov/reading
-rm/doc-collections/nuregs/staff/sr0800/ch7/
)  6. CFR, "Licenses, Certifications, and Approvals for Nuclear Power Plants," Part 52, Chapter 1, Title 10, "Energy."    7. NRC, Regulatory Guide (RG) 1.170, "Software Test Documentation for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," Washington, DC.
 
8. IEEE Std. 829
-2008, "IEEE Standard for Software Test Documentation," Piscataway, NJ, 2008.
 
9. IEEE Std. 1074
-2006, "IEEE Standard for Developing a Software Life Cycle Process," Piscataway, NJ, 2006.
 
10. NRC, RG 1.173, "Developing Software Life Cycle Processes for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," Washington, DC.
 
11. IEEE Std. 1012-2004, "IEEE Standard for Software Verification and Validation," Piscataway, NJ, 2004.  12. IEEE Std. 7
-4.3.2-2003, "IEEE Standard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations," Piscataway, NJ, 2003.
 
1 3. NRC, NUREG/CR
-6101, "Software Reliability and Safety in Nuclear Reactor Protection Systems," Washington, DC, November 1993. (ADAMS Accession No. ML072750055)
  14. NRC, NUREG/CR
-6263, "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis, and Research Needs," Washington, DC, June 1995.  (ADAMS Accession Nos. ML063470590
and ML063470593)


4  Publicly available NRC published documents are available electronically through the Electronic Reading Room on the NRC's public Web site at: http://www.nrc.gov/reading
This guide will also be used to evaluate submittals from operating reactor licensees that propose system modifi cations voluntarily initiated by the licensee if there is a clear nexus between the proposed modifications and this guidance.
-rm/doc-collections/.  The documents can also be viewed online or printed for a fee in the NRC's Public Document Room (PDR) at 11555 Rockville Pike, Rockville, MD; the mailing address is USNRC PDR, Washington, DC 20555; telephone 301
-415-4737 or (800) 397
-4209; fax (301) 415
-3548; and e-mail pdr.resource@nrc.gov
.  5  Copies of Institute of Electrical and Electronics Engineers (IEEE) documents may be purchased from the Institute of Electrical and Electronics Engineers Service Center, 445 Hoes Lane, PO Box 1331, Piscataway, NJ 08855 or through the IEEE's public Web site at http://www.ieee.org/publications_standards/index.htm


====l.      ====
BIBLIOGRAPHY
RG 1.1 7 1, Rev. 1, Page
Beizer, Boris, Software Testing Techniques, Van Nos trand Reinhold, 1990. Hecht, H., A.T. Tai, K.S. Tso, "Class 1E Digital Sys tems Studies," NUREG/CR-6113, USNRC, October 1993.1 Institute of Electrical and Electronics Engineers, "Stan dard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations," IEEE Std 7-4.3.2, 1993.1 Lawrence, J.D., "Software Reliability and Safety in Nuclear Reactor Protection Systems," NUREG/ CR-6101 (UCRL-ID-117524, Lawrence Livermore National Laboratory), USNRC, November 1993.1 1Copies may be purchased at current rates from the U.S. Government Prin ting Offuie, P.O. Box 37082, Washington, DC 20402-9328 (telephone
10 15. NRC, RG 1.168, "Verification, Validation, Reviews, and Audits for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," Washington, DC.
(202)512-2249);
or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161. Copies are availabl" for inspection or copying for a fee from the NRC Public Docu went Room at 2120 L Street NW., Washington, DC; the PDR's mailing ad dress is Mail Stop LL-6, Washington, DC 20555-0001;
telephone
(202)634-3273;
fax (202)634-3343.


16. International Atomic Energy Agency (IAEA) Safety Guide NS
Lawrence, J.D., and G.G. Preckshot, "Design Factors for Safety-Critical Software," NUREG/CR-6294, USNRC, December 1994.1 Seth, S., et al., "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis and Research Needs," NUREG/CR-6263, USNRC, June 1995.1 USNRC, "Criteria for Digital Computers in Safety Systems of Nuclear Power Plants," Regulatory Guide 1.152, Revision 1, January 1996.2 USNRC, "Standard Review Plan," NUREG-0800, February 1984.1 2 Single copies of regulatory guides maybe obtained free ofcharge by writ ing the Office of Administration, Printing, Graphics and Distribution Branch, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001;
-G-1.1, "Software for Computer Based Systems Important to Safety in Nuclear Power Plants" issued September 2000, Vienna, Austria, 2000.
or by fax at (301)415-5272.


6 17.    Beizer, B., Software Testing Techniques, Van Nostrand Reinhold, New York, NY, 1990.
Copies are available for in spection or copying for a fee from the NRC Public Document Room at 2120 L Street NW, Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001;
telephone
(202)634-3273;
fax (202)634-3343.


7 18. CFR, "Maintenance and Inspection of Records"
1.171-6 1-.  
Part 21.51, Chapter 1, Title 10, "Energy." 
REGULATORY
19. NRC, NUREG
ANALYSIS A separate regulatory analysis was not prepared for this regulatory guide. The regulatory analysis prepared for Draft Regulatory Guide DG-1057, "Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," provides the regulatory basis for this guide. A copy of the regulatory analysis is available for inspection and copying for a fee at the NRC Public Document Room, 2120 L Street NW., Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001;
-1409, "Backfitting Guidelines," Washington, DC. (ADAMS Accession No. ML032230247) 
phone (202)634-3273;
20. NRC, Management Directive 8.4, "Management of Facility
fax (202)634-3343.
-Specific Backfitting and Information Collection," Washington DC. (ADAMS Accession No. ML050110156) 


6  Copies of International Atomic Energy Agency (IAEA) documents may be obtained through their Web site: WWW.IAEA.Org/
Federal Recycling Program 1.171-7 UNITED STATES NUCLEAR REGULATORY
or by writing the International Atomic Energy Agency P.O. Box 100 Wagramer Strasse 5, A
COMMISSION
-1400 Vienna, Austria.  Telephone (+431) 2600
WASHINGTON, DC 20555-0001
-0, Fax (+431) 2600
2 FIRST CLASS MAIL / POSTAGE AND FEES PAIb USNRC '/ PERMIT NO. G-67 OFFICIAL BUSINESS PENALTY FOR PRIVATE USE, $300 co 0 ~0 -4 0 I ..  a. z cr" "4'l}}
-7, or E-Mail at Official.Mail@IAEA.Org    7 Boris Beizer, Software Testing Techniques, June 1990, ISBN
-10: 1850328803, ISBN
-13: 978-1850328803 can be purchased at many book stores and online locations, including the following Web site: http://www.amazon.com/Software
-Testing-Techniques
-Boris-Beizer/dp/1850328803/ref=sr_1_1?s=books&ie=UTF8&qid=1288897895&sr=1
-1.}}


{{RG-Nav}}
{{RG-Nav}}

Revision as of 17:57, 31 August 2018

(Draft Was DG-1057) Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants
ML003740108
Person / Time
Issue date: 09/30/1997
From:
Office of Nuclear Regulatory Research
To:
References
DG-1057 RG-1.171
Download: ML003740108 (8)
v • d • e


U.S. NUCLEAR REGULATORY

COMMISSION

REGULATORY

GUI OFFICE OF NUCLEAR REGULATORY

RESEARCH REGULATORY

GUIDE 1.171 (Draft was DG-1057) SOFTWARE UNIT TESTING FOR DIGITAL COMPUTER SOFTWARE USED IN SAFETY SYSTEMS OF NUCLEAR POWER PLANTS

A. INTRODUCTION

In 10 CFR Part 50, "Domestic Licensing of Pro duction and Utilization Facilities," paragraph

55a(a)(1)

requires, in part, 1 that systems and components be de signed, tested, and inspected to quality standards com mensurate with the safety function to be performed.

Criterion

1, "Quality Standards and Records," of Ap pendix A, "General Design Criteria for Nuclear Power Plants," to 10 CFR Part 50 requires, in part, 1 that a qual ity assurance program be established and implemented in order to provide adequate assurance that systems and components important to safety will satisfactorily per form their safety functions.

Appendix B, "Quality As surance Criteria for Nuclear Power Plants and Fuel Re processing Plants," to 10 CFR Part 50 describes criteria that a quality assurance program for systems and com ponents that prevent or mitigate the consequences of postulated accidents must meet. In particular, besides the systems and components that directly prevent or mitigate the consequences of postulated accidents, the criteria of Appendix B also apply to all activities affect ing the safety-related functions of such systems and components as designing, purchasing, installing, test ing, operating, maintaining, or modifying.

A specific l1n this regulatory guide, many of the regulations have been paraphrased;

see 10 CFR Part 50 for the full text.requirement is contained in 10 CFR 50.55a(h), which requires that reactor protection systems satisfy the cri teria of IEEE Std 279-1971, "Criteria for Protection Systems for Nuclear Power Generating Stations." 2 Paragraph

4.3 of IEEE Std 279-19713 states that quali ty of components is to be achieved through the specifi cation of requirements known to promote high quality, such as requirements for design, inspection, and test. Many of the criteria in Appendix B to 10 CFR Part 50 contain requirements closely related to testing activities.

Criterion I, "Organization," requires the es tablishment and execution of a quality assurance pro gram. Criterion H, "Quality Assurance Program," re quires, in part, that the program take into account the need for special controls, processes, test equipment, tools, and skills to attain the required quality, as well as the need for verification of quality by inspection and test. Criterion III, "Design Control," requires, in part, that measures be established for verifying and checking the adequacy of design, such as by the performance of a 2 Revision I of Regulatory Guide 1.153, "Criteria for Safety Systems," en dorses IEEE Std 603-1991,"Criteria for Safety Systems for Nuclear Pow er Generating Stations," as a method acceptable to the NRC staff for satis fying the NRC's regulations with respect to the design, reliability, qualifi cation, and testability of the power, instrumentation, and control portions of the safety systems of nuclear power plants. 3 IEEE publications may be obtained from the IEEE Service Center, 445 Hoes Lane, Piscataway, NJ 08854.7.USNRCREGULATORYGUIDES

The guides we Issued in the following ten broad divisions:

Reglatory Guides are Issued to descibe and make avlable tothe public such Informs lion as methods acceptable to the NRC staf for Implementing specific pans of the Com- 1. Power Reactors 6. Products mission's regulations, techniques usedbythestaff inevaluating specific problems orpos- 2Z Research and Test Reactors 7. Transportation tulated accdentsa and data needed by the NRC staff In Its review of ap:icationrs forper- 3& Fuels and Materials Facilities

8. Occupations!

Health mits and licensea.

Regulatory guides are not sstitutes for regulations, and compiance

4. Environmental and Siting 9. Antitust and Financial Review with them Is not required.

Methods and solutions different from those set out in theguides

5. Matrials and Plant Protection

10. General will be acceptable If they provide a basis for the findings requisite to the Issuance or con linuance of a permit or license by the Commission.

Single copies of regulatory guides may be obtained free of chrge bywrlfing te Printing.

This guide was lesu after consideration of comments received from thre public. Com- Graphics anid Distribution Branch. Office of Administrtion, U.S. Nuclear Regulatory Com ments andsuggestions for inprovements Inthese guides wencosurged at all Imes, and mission, Washington, DC 2055-0001;

or by fox at (301)415-5272 gue will be revised, as appropriate, to accommodate comments and to reflect new in= on or aperience.

Issued guides may also bepurchased!

from me* National Technical Information Service on Whitten comments may be submitted to te Rules Review and Directives Branch, DFIPS, a standing order basis. Details on this service may be obtained by writing NTIS, 5285 Port ADM, U.S. Nuclear Regulatory Commission, Washington, DC 2055-0001.

Royal Road, Springfield, VA 2216

1. DE September

1997 suitable testing program, and that design control measures be applied to items such as the delineation of acceptance criteria for inspections and tests. Criterion V, "Instructions, Procedures, and Drawings," requires activities affecting quality to be prescribed by docu mented instructions, procedures, or drawings of a type appropriate to the circumstances and that these activi ties be accomplished in accordance with these instruc tions, procedures, or drawings.

Criterion V further re quires that instructions, procedures, and drawings include appropriate quantitative or qualitative accep tance criteria for determining that important activities have been satisfactorily accomplished.

Criterion XI, "Test Control," requires establishment of a test pro gram to ensure that all testing required to demonstrate that structures, systems, and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorpo rate the requirements and acceptance limits contained in applicable design documents.

Test procedures must include provisions for ensuring that all prerequisites for the given test have been met, that adequate test instru mentation is available and used, and that the test is per formed under suitable environmental conditions.

Crite rion XI also requires that test results be documented and evaluated to assure that test requirements have been sat isfied. Finally, Criteria VI, "Document Control," and XVII, "Quality Assurance Records," provide for the control of the issuance of documents, including changes thereto, that prescribe all activities affecting quality and provide for the maintenance of sufficient records to furnish evidence of activities affecting quali ty. The latter requires test records to identify the inspec tor or data recorder, the type of observation, the results, the acceptability of the results, and the action taken in connection with any deficiencies noted. This regulatory guide endorses ANSI/IEEE Std 1008-1987, "IEEE Standard for Software Unit Test ing," 3 with the exceptions stated in the Regulatory Position.

IEEE Std 1008-1987 describes a method ac ceptable to the NRC staff for complying with parts of the NRC's regulations for promoting high functional reliability and design quality in software used in safety systems.4 In particular, the method is consistent with the previously cited General Design Criteria and the criteria for quality assurance programs of Appendix B as they apply to software unit testing. The criteria of Appendices A and B apply to systems and related quali 4 The term "safety systems" is synonymous with "safety-related systems." The General Design Criteria cover systems, structures, and components "important to safety." The scope of this regulatory guide is, however, lim ited to "safety systems," which are a subset of "systems important to safety..ty assurance processes, and if those systems include software, the requirements extend to the software ele ments. In general, information provided by regulatory guides is reflected in the Standard Review Plan (NUREG-0800).

The Office of Nuclear Reactor Regu lation uses the Standard Review Plan to review applica tions to construct and operate nuclear power plants. This regulatory guide will apply to the revised Chapter 7 of that document.

The information collections contained in this regu latory guide are covered by the requirements of 10 CFR Part 50, which were approved by the Office of Manage ment and Budget, approval number 3150-0011.

The NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information un less it displays a currently valid OMB control number.

B. DISCUSSION

The use of industry consensus standards is part of an overall approach to meeting the requirements of 10 CFR Part 50 when developing safety systems for nuclear power plants. Compliance with standards does not guarantee that regulatory requirements will be met. However, compliance does ensure that practices accepted within various technical communities will be incorporated into the development and quality assur ance processes used to design safety systems. These practices are based on past experience and represent in dustry consensus on approaches used for development of such systems.

Software incorporated into instrumentation and control systems covered by Appendix B will be referred to in this regulatory guide as safety system software.

For safety system software, software testing is an im portant part of the effort to achieve compliance with the NRC's requirements.

Software engineering practices rely, in part, on software testing to meet general quality and reliability requirements consistent with Criteria 1 and 21 of Appendix A to 10 CFR Part 50, as well as Cri teria I, II, III, V, VI, XI, and XVII of Appendix B. The consensus standard, IEEE Std 1008-1987 (reaffirmed in 1993), defines a method for planning, preparing for, conducting, and evaluating software unit testing. The method described is consistent with the previously cited regulatory requirements as they apply to safety system software.

Current practice for the development of software for high-integrity applications includes the use of a software life cycle process that incorporates software testing activities, e.g., IEEE Std 1074-1991, "IEEE Standard for Developing Software Life Cycle 1.171-2 ,

Processes." 3 Software testing, including software unit testing, is a key element in software verification and validation activities, as indicated by IEEE Std 1012-1986, "IEEE Standard for Software Verification and Validation Plans," 3 and IEEE Std 7-4.3.2-1993, "Standard Criteria for Digital Computers in Safety Sys tems of Nuclear Power Generating Stations." A com mon approach to software testing [NUREG/CR-6101, "Software Reliability and Safety in Nuclear Reactor Protection Systems" (November

1993); NUREG/ CR-6263, "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis and Re search Needs" (June 1995)]5 utilizes a three-level test program to help ensure quality in a complex software product or complex set of cooperating software prod ucts, i.e., unit-level testing, integration-level testing, and system-level testing such as system validation tests or acceptance tests. IEEE Std 1008-1987 delineates an approach to the unit testing of software that is based on the assumption of a larger context established by verifi cation and validation (V&V) planning as well as general planning for the full range of testing activities to be applied. Therefore, software unit testing per formed in accordance with IEEE Std 1008-1987 should be consistent with planning information estab lished in V&V plans and higher-level software test plans, although that planning information is not within the scope of IEEE Std 1008-1987.

C. REGULATORY

POSITION The requirements in ANSI/IEEE Std 1008-1987, "IEEE Standard for Software Unit Testing," provide an approach acceptable to the NRC staff for meeting the requirements of 10 CFR Part 50 as they apply to the unit testing of safety system software, subject to the provi sions listed below. The appendices to IEEE Std 1008-1987 are not endorsed by this regulatory guide except as noted below. Appendix A to this standard pro vides guidance regarding the implementation of the software unit testing approach, and Appendix B to the standard provides context regarding software engineer ing information and testing assumptions that underlie the software unit testing approach.

To meet the requirements of 10 CFR 50.55a(h)

and Appendix A to 10 CFR Part 50 as assured by complying with the criteria of Appendix B to 10 CFR Part 50 ap 5 Copies are available at current rates from the U.S. Government Printing Office, P.O. Box 37082, Washington, DC 20402-9328 (telephone

(202)512-2249);

or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161. Copies are I available for inspection or copying for a fee from the NRC Public Docu ment Room at 2120 LStreet NW., Washington, DC; the PDR's mailing ad dress is Mail Stop LL-6, Washington, DC 20555-0001;

telephone

(202)634-3273;

fax (202)634-3343.

plied to the unit testing of safety system software, the following exceptions are necessary and will be consid ered by the NRC staff in the review of submittals from licensees and applicants. (In this section, the cited crite ria are in Appendix B to 10 CFR Part 50 unless other wise noted.) 1. SOFTWARE TESTING DOCUMENTATION

Criterion XI, "Test Control," requires that a test program be established to ensure that all testing re quired to demonstrate that systems and components will perform satisfactorily in service is identified and performed in accordance with written test procedures that incorporate requirements and acceptance limits contained in applicable design documents.

Criterion I, "Organization," Criterion II, "Quality Assurance Pro gram," Criterion III, "Design Control," Criterion V, "Instructions, Procedures, and Drawings," Criterion VI, "Document Control," and Criterion XVIi, "Quality Assurance Records," contain requirements bearing on information associated with testing. IEEE Std 1008-1987, in section 1.1, mandates the use of the Test Design Specification and the Test Summary Report de fined by ANSI/IEEE Std 829-1983, "IEEE Standard for Software Test Documentation." In addition, IEEE Std 1008-1987 either incorporates additional informa tion into these two documents or indicates the need for additional documents.

Regardless of whether these two documentation formats are used, the documentation used to support software unit testing (either documen tation used directly in the software unit testing activity or documentation of the overall testing effort) must in clude information necessary to meet regulatory re quirements as applied to software test documentation.

As a minimum, this information includes:

"* Qualifications, duties, responsibilities, and skills required of persons and organizations assigned to testing activities, " Environmental conditions and special controls, equipment, tools, and instrumentation needed for the accomplishment of testing, " Test instructions and procedures incorporating the requirements and acceptance limits in applicable design documents, " Test prerequisites and the criteria for meeting them, "* Test items and the approach taken by the testing program, "* Test logs, test data, and test results, "* Acceptance criteria, 1.171-3 Test records indicating the identity of the tester, the type of observation, the results and acceptability, and the action taken in connection with any deficiencies.

Any of the above information items that are not present in the documentation selected to support soft ware unit testing must be incorporated as additional items. 2. TEST PROGRAM Criterion XI, "Test Control," requires establish ment of a test program to ensure that all testing required to demonstrate that structures, systems, and compo nents will perform satisfactorily in service is identified and performed in accordance with written test proce dures that incorporate the requirements and acceptance limits contained in applicable design documents.

The two aspects of test coverage that are particularly impor tant for the unit testing of safety system software are coverage of requirements and coverage of the internal structure of the code. 2.1 Coverage of Requirements For safety system software, those requirements identified as essential to the safety determination

6 must be tested. Section 3.2.2(5) of IEEE Std 1008-1987 sug gests consideration of expected use of the unit in the de termination of features to be tested. All features and as sociated procedures, states, state transitions, and associated data characteristics essential to the safety de termination must be included in the testing.

2.2 Coverage of Internal Structure Section 3.1.2(2) of IEEE Std 1008-1987 specifies statement coverage (covering each source language statement with a test case) as a criterion for measuring the completeness of the software unit testing activity.

Statement coverage is a very weak criterion for meas uring test completeness

[See Beizer 7 and NUREG/ CR-6263 8]. Therefore, the staff does not endorse state ment coverage as a sufficient coverage criterion for software unit testing. For safety system software, the unit test coverage criteria to be employed should be identified and justified.

6 Regulatory Guide 1.172, "Software Requirements Specifications for Dig ital Computer Software Used in Safety Systems of Nuclear Power Plants," endorses IEEE Std 830-1993, "IEEE Recommended Practice for Soft ware Requirements Specifications." 7 Boris Beizer, Software Testing Techniques, Van Nostrand Reinhold, 1990. 8S. Seth et al., "High Integrity Software for Nuclear Power Plants: Candi date Guidelines, Technical Basis and Research Needs," NUREG/ CR-6263, June 1995.3. TEST PROGRAM RECORDS Criteria VI, "Document Control," and XVII, "Quality Assurance Records," as well as 10 CFR 21.51, require the control and retention of documents and records affecting quality. In addition, Criterion III, "Design Control," requires that design changes be sub ject to design control measures commensurate with those applied to the original design. Preservation of testing products is discussed in section 3.8.2(4) of IEEE Std 1008-1987.

Since design control measures must be applied to acceptance criteria for tests and since some software testing materials are frequently re-used and evolve during the course of software development and software maintenance (for example, regression test materials), such materials should be configuration items under change control of a software configuration management system.9 Additional information on this topic is provided in section A6 of Appendix A to IEEE Std 1008-1987.

4. INDEPENDENCE

IN SOFFWARE VERIFICATION

Criterion III, "Design Control," imposes an inde pendence requirement for the verification and checking of the adequacy of the design, requiring that those per sons who verify and check be different from those who accomplish the design. Therefore, independence is an additional requirement for software unit testing. Either those persons who establish the requirements-based elements for a software unit test must be different from those who designed or coded the software, or there must be independent review of the establishment of the requirements-based elements.

The guidance in section A7 of Appendix Ato IEEE Std 1008-1987 provides ac ceptable ways to meet this requirement for software unit testing. These independent persons must be suffi ciently competent in software engineering to ensure that software unit testing is adequately implemented.

5. OTHER STANDARDS

Section 1.3 of IEEE Std 1008-1987 references ANSI/IEEE Std 729-1983, "IEEE Standard Glossary of Software Engineering Terminology," and ANSI/ IEEE Std 829-1983, "IEEE Standard for Software Test Documentation." These referenced standards should be treated individually.

If a referenced standard has been incorporated sep arately into the NRC's regulations, licensees and appli cants must comply with that standard as set forth in the 9 Regulatory Guide 1.169 endorses IEEE Std 828-1990, "IEEE Standard for Software Configuration Management Plans," and IEEE Std 1042-1987, "IEEE Guide to Software Configuration Management," to provide guidance for general software configuration management plans and their implementation.

1.171-4 K

regulation.

If the referenced standard has been endorsed in a regulatory guide, the standard constitutes a method acceptable to the NRC staff of meeting a regulatory re> quirement as described in the regulatory guide. If a ref erenced standard has been neither incorporated into the NRC's regulations nor endorsed in a regulatory guide, licensees and applicants may consider and use the in formation in the referenced standard, if appropriately justified, consistent with current regulatory practice.

D. IMPLEMENTATION

The purpose of this section is to provide informa tion to applicants and licensees regarding the NRC staff's plans for using this regulatory guide. No backfit-ting is intended or approved in connection with the is suance of this proposed guide. Except in those cases in which an applicant pro poses an acceptable alternative method for complying with the specified portions of the NRC's regulations, the methods described in this guide will be used in the evaluation of submittals in connection with applica tions for construction permits and operating licenses.

This guide will also be used to evaluate submittals from operating reactor licensees that propose system modifi cations voluntarily initiated by the licensee if there is a clear nexus between the proposed modifications and this guidance.

BIBLIOGRAPHY

Beizer, Boris, Software Testing Techniques, Van Nos trand Reinhold, 1990. Hecht, H., A.T. Tai, K.S. Tso, "Class 1E Digital Sys tems Studies," NUREG/CR-6113, USNRC, October 1993.1 Institute of Electrical and Electronics Engineers, "Stan dard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations," IEEE Std 7-4.3.2, 1993.1 Lawrence, J.D., "Software Reliability and Safety in Nuclear Reactor Protection Systems," NUREG/ CR-6101 (UCRL-ID-117524, Lawrence Livermore National Laboratory), USNRC, November 1993.1 1Copies may be purchased at current rates from the U.S. Government Prin ting Offuie, P.O. Box 37082, Washington, DC 20402-9328 (telephone

(202)512-2249);

or from the National Technical Information Service by writing NTIS at 5285 Port Royal Road, Springfield, VA 22161. Copies are availabl" for inspection or copying for a fee from the NRC Public Docu went Room at 2120 L Street NW., Washington, DC; the PDR's mailing ad dress is Mail Stop LL-6, Washington, DC 20555-0001;

telephone

(202)634-3273;

fax (202)634-3343.

Lawrence, J.D., and G.G. Preckshot, "Design Factors for Safety-Critical Software," NUREG/CR-6294, USNRC, December 1994.1 Seth, S., et al., "High Integrity Software for Nuclear Power Plants: Candidate Guidelines, Technical Basis and Research Needs," NUREG/CR-6263, USNRC, June 1995.1 USNRC, "Criteria for Digital Computers in Safety Systems of Nuclear Power Plants," Regulatory Guide 1.152, Revision 1, January 1996.2 USNRC, "Standard Review Plan," NUREG-0800, February 1984.1 2 Single copies of regulatory guides maybe obtained free ofcharge by writ ing the Office of Administration, Printing, Graphics and Distribution Branch, U.S. Nuclear Regulatory Commission, Washington, DC 20555-0001;

or by fax at (301)415-5272.

Copies are available for in spection or copying for a fee from the NRC Public Document Room at 2120 L Street NW, Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001;

telephone

(202)634-3273;

fax (202)634-3343.

1.171-6 1-.

REGULATORY

ANALYSIS A separate regulatory analysis was not prepared for this regulatory guide. The regulatory analysis prepared for Draft Regulatory Guide DG-1057, "Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," provides the regulatory basis for this guide. A copy of the regulatory analysis is available for inspection and copying for a fee at the NRC Public Document Room, 2120 L Street NW., Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001;

phone (202)634-3273;

fax (202)634-3343.

Federal Recycling Program 1.171-7 UNITED STATES NUCLEAR REGULATORY

COMMISSION

WASHINGTON, DC 20555-0001

2 FIRST CLASS MAIL / POSTAGE AND FEES PAIb USNRC '/ PERMIT NO. G-67 OFFICIAL BUSINESS PENALTY FOR PRIVATE USE, $300 co 0 ~0 -4 0 I .. a. z cr" "4'l


Retrieved from "https://kanterella.com/w/index.php?title=Regulatory_Guide_1.171&oldid=1013406"