ML20198F762

Reg Guide 01.171, Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants

ML20198F762
Person / Time
Issue date:	09/30/1997
From:	NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES)
To:
References
TASK-*****, TASK-RE REGGD-01.171, REGGD-1.171, NUDOCS 9801120147
Download: ML20198F762 (8)

Text

- _ _ _ __ -

l om U.S. NUCLEAR REGULATORY COMMISSION September 1997 o eww U

%*****OFFICE ) OF NUCLEAR REGU REGULATORY GUIDE 1.171 (Draft was OG-1057)

SOFTWARE UNIT TESTING FOR DIGITAL COMPUTER SOFTWARE ,

USED IN SAFETY SYSTEMS OF NUCLEAR POWER PLANTS A, INTRODUCTION requirement is contained in 10 CFR 50.55a(h), which requires that reactor protection systems satisfy the cri-In 10 CFR Part 50," Domestic Licensing of Pr

• teria of IEEE Std 279-1971, " Criteria for Protection duction and Utilization Facilities," paragraph 55a(a)(1) Systems for Nuclear Power Generating Stations,"2 requires, in part,I hat t systems and components be de- paragraph 4.3 ofIEEE Std 279-1971 3states that quali-signed, tested, and inspected to quality standards com- ty of components is to be achieved through the specifi-mensurate with the safety function to be performed. cation of requirements known to promote high quality, Criterion 1," Quality Standards and Records," of Ap- such as requirements for design, inspection, and test, pendix A," General Design Criteria for Nuclear Power Plants," to 10 CFR Part 50 requires, in part,I that a qual. Many of the criteria in Appendix B to 10 CFR Part 50 contain requirements closely related to testing ity assurance program be established and implemented O activities, Criterion I, " Organization," requires the es-in order to provide adequate assurance that systems and tablishment and execution of a quality assurance pro-comp (ments impo. tant to safety will satisfactorily per.

form their safety functions. Appendix B," Quality As- gram, Criterion 11, "Ouality Assuranct Program," re-surance Criteria for Nuclear Power Plants and Fuel Re. quires, in part, that the program take into account the processing Plants," to 10 CFR Part 50 describes criteria need for special controls, processes, test equipment, tools, and skills to attain the required quality, as well as that a quality assurance program for system's and com.

p(ments that prevent or mitigate the consequences of the need for verification of quality by inspection and postulated accidents must meet. In particular, besidr.s test. Criterion 111 " Design Control," requires, in part, that measures be established for verifying and checking the systems and components that directly prevent or mitigate the consequences of postulated accidents, the the adequacy of design, such as by the performance of a criteria of Appendix 3 also apply to all activities affect- 2Remion i of Regulatory cuide i.153 -criteria for safety systems, en ing the safety related functions of such systems and dorses ll EE std 603-1991,"Cnteria for saresy systems for Nuclest row.

cr omrating stan ns," as a method acceptable t the NRCstaff forsatis components as designing, purchasing, installing, test- fying the NRC's regulanons with respect to the design, rehabihty, e quahf; mg, operating, maintaining, or modifying, A specif,c i canon, and testabihty of the power, instrumentation, and control po,tions of the safely systems of nuclear power plants.

Iln this regulatory guide, many of the regulat,ons have been paraphrased, 3 1 ELE pubheations may be obtained from the IEEE service Center,445 see 10 CFR Pa.130 for the full teit. Ihm LJinc, Piscatawsy, NJ 08854.

UiNRC REGllATORY 00 tdt 3 The guKjet ow itsued m the fo%Weg Den tWOed diuSKm8 nega cory ou oes are seuses to oe.ance and make oweta io the outwee suen a,torme bon as metreds acceptatHe to the NRC stan for swpdammeng specinc peria of the Co'* 1 Fewer Rasctors 6 Prooucts meseon a regulanons technniues weed try the staP in eveushng specAc probsams or pN 2 Research md Test Reactors 7 Transportanon Listed ecenteries and Oata neeoed try the PeAC staff an es re+ew at app 6 canons for per. 3 Fuses and Matenela Fac*een 6 Occupahonal Hee!!n StE47.qu SYanoNYN*[ntYo'm NN[oe gh andN ction $

p 7 prma e in nos esma te io ce anuance or con

,A The guo wasi euert ener co%,devanon of comme,in recoved wom the auche com. Nr.c= '

ouYtNcYa or Nem u bR a CN

! i  ;'"*e.'"=re'le=f" ar""'t""J==","0* :3d

"-**'"-a==*'~"'*=

. ,en - oe ,, t m - ano m e e,. m o m issued.guioes

..t.nngo,o. me,-eso ne pure ost. aseo-tem m ens the Nanonei Technical peam.eeeovwre ,~m ir4ormar.cn m ,1 service on Aou u s Nucw r Reguatory commaemr weshmaton. oc 20%5-0001 noye nose. spanon o va 22$61

~

9001120147 970930 ,

PDR REGGD 'kphrh u lf ,,{. } ,l, p' ( {

01.171 R PDR ' ..t, g;4 4

1

}

l suitable testing program, and that design control ty assurance processes, and if those systems include measures be applied to items such as the delineation of software, the requirements extend to the software ele-acceptance criteria for inspections and tests. Criterion ments.

V," Instructions, Procedures, and Drawings " requires in general, information provided by regulatary activit! , affecting quality to be prescribed by docu" guides is reflected in the Standard Review Plan menteo instructions, procedures, or d awings of a type (NUREG-0800).The Office of Nuclear Reactor Regu-appropriate to the circumstances and . hat these activi- lation uses the Standard Review Plan to review applica-ties be accomplished in accordance wid these instruc- tions to construct and operate nuclear power plants.

tions, procedures, or drawings. Criterion V further re- This regulatory guide will apply to the revised Chapter quires that instructions, procedures, ano dra c;.igs 7 of that document.

include appropriate quantitative or qualitative accep-The information collections contained in this regu-tance criteria for determining that important activities latory guide are covered by the requirements of 10 CFR have been satisfactorily accomplished. Criterion XI, Part 50, which were approved by the Office of Manage-

" Test Centrol," requires establishment of a test pro-gram to ensure that all testing required to demonstrate ment and Budget, approval number 3150-0011.The NRC may not conduct or sponsor, and a person is not that structures, systems, and components will perform satisfactorily in service is identified and performed in required to respond to, a collection of information un-less it displays a currently valid OMB control numbei.

accordance with written test procedures that incorpo-rate the requirements and acceptance limits contained 11, DISCUSSION in applicable design documents. Test procedures must The use of industry consensus standards is part of include provisions for ensuring that all prerequisites for an overall approach to meeting the requirements of the given test have been met, that adequate test instru-10 CFR Part 50 when developing safety systems for mentation is available and used, and that the test is per-nuclear power plants. Compliance with standards does formed under su.i table environmental conditions. Crite-not guarantee that regulatory requirements will be met.

rion XI also requires that test results be documented and h ver, compliance does ensure that practices evaluated to assure that test requirements have been sat-accepted within various technical communities will be asfied. hnally, Critena VI, " Document Control,' and .

mcorporated into the development and quality assur-XVil, " Quality Assurance Records,,, provide for the

. ance processes used to design safety systems. These control of. the issuance of documents, meluding pra es are ase n p st expedence and represent m. -

changes thereto, that prescribe all activities ffecting

. dustry consensus on approaches used for development quality and provide for the maintenance of sufficient records to turnish evidence of activities affecting quali- "I ""' ' Y *'# *'

ty. The latter requires test records to identify the inspec. Software incorporated into instrumentation anct tot or data recorder, the type of observation, the results, control systems covered by Appendix B will be referred the acceptability of t,.: results. nd the action taken in to in this regulatory guide as safety system software.

connection with any deficiencies noted, For safety system software, software testing is an im-portant part of the effort to achieve compliance with the

.Du.s regulatory guide endorses ANSI /IEEE Stu NRC's requirements. Software engineering practices 1008-1987, "lEEE Standard for Sottware Unit Test-rely,.m part, on software testing to meet general quality ing,, 3 with the exceptions stated .in the Regulatory and reliability requirements consistent with Criteria 1 Position. IEEE Std 1008-1987 desenbes a method ac-and 21 of Appendix A to 10 CFR Part 50, as well as Cri-ceptable to the NRC staff for complymg with parts of teria 1,11, III, V, VI, XI, and XVil of Appendix B.

the NRC s regulatioas for promoting high functional reliability and design quality in software used in safety The consensus standard, IEEE Std 1008-1987 systems.4 in particular, the methcd is consistent with (reaffirmed in 1993), defines a method for planning, the previously cited General Design Criteria and the preparing for, conducting, and evaluating software unit criteria for quality assurance programs v

• ndix B testing. The method described is consistent with the as thcy apply to software unit testing. ,ieria of previously cited regulatory requirements as they apply Appendice< A and B apply to systems and metated quali. to safety system software.

Current practice for the development risoftware meierm saren ostems is synonymouswith"sarcty.rclated systems, for high integrity applications includes ti.e use of a ne nenersmesign entena cour systems, siructure , and componena software life cycle process that incorpor tes software importan io s.,ren The scopcorihis regubiory guide ts however.hm.

tied to "sately .yuems, which are a subset or systems important to wk8 activities, e.8., IEEE Std 1074- 1991, "lEEE saren? Standard for Developing Software Life Cycle 1.171 - 2

Processes."3 Software testing, including software unit plied to the unit testing of safety system software, the testing, is a key element in software verification and following exceptions are necessary and will be consid- t validation activities, as indicated by lEEE Std ered by the NRC staffin the review of submittals from O)

( 1012-1986,"lEEE Standard for Software Verification and Validation Plans,"3 and IEEE Std 7-4.3.2-1993, licensees and applicants. (In this section, the cited crite-ria are in Appendix B to 10 CFR Part 50 unless other-

" Standard Criteria for Digital Computers in Safety Sys- wise noted.)

tems of Nuclear Power Generating Stations." A com-mon approach to software testing [NUREG/CR-6101, 1. SOITWARE TESTING DOCUMENTATION "SoftwaN Reliability and Safety in Nuclear Rcactor Criterion XI, " Test Control," requires that a test Protection Systems" (November 1993); NUREG/ program be established to ensure that all testing re-CR-6263,"lligh Integrity Software for Nuclear Power quired to demonstrate that systems and components PMts: Candidate Guidelines, Technical Basis and Re- will perform satisfactorily in service is identified and search Needs"(June 1995)}5 utilizes a three-level test performed in accordance with written test procedures program to help ensure quality in a complex software that incorporate requirements and acceptance limits product or complex set of cooperating software prod- contained in applicable design documents. Criterion I, s ucts, i.e., unit level testing, integiation level testing, " Organization," Criterion 11. " Quality Assurance Pro-and system-level testing such as system validation tests gram," Criterion lit, " Design Control," Criterion V, or acceptance tests. IEEE Std 1008-1987 delineates an " Instructions, Procedures, and Drawings," Criterion approach to the unit testing of software that is based on VI," Document Control," and Criterion XVil," Quality the assumption of a larger context estah'ished by vesifi- Assurance Records," contain requirements bearing on cation had validation (V&V) phnning as well as nformation associated with testing. IEEE Std general planning for the full range of testing activities 1008-1987,in section 1.1, mandates the use of the Test to be applied. Therefore, software unit testing per. Design Specification and the Test Summary Report de-formed in accordance with IEEE Std 1008-1987 fined by ANSI /IEEE Std 829-1983, "IEEE Standard should be consistent with planning information estab- for Software Test Documentation." In addition, IEEE s lished in V&V plans and higher-level software test Std 1008-1987 either incorporates additional informa-plans, although that planning information is not within tion into these two documents or indicates the need for

[V) the scope of IEEE Std 1008-1987, additional documents. Rege s of whether these two C REGULATORY POSITION documentation formats are ud, the documentation used to support software unit testing (either documen-The requirements in ANSI /IEEE Std 1008-1987, tation used directly in the software unit testing activity "lEEE Standard for Software Unit Testing," provide an or documentation of the overall testing effort) must in-approach acceptable to the NRC staff for meeting the clude information necessary to meet regulatory re-requirements of 10 CFR Part 50 as they apply to the unit quirements as applied to software test documentation.

testing of safety system software, subject to the provi- As a minimum, this information includes:

sions listed below. The appendices to IEEE Std .

• Qualifications, duties, responsibilities, and skills 1008-1987 are not endorsed by this regulatory guide required of persons and organizations assigned to

.xcept as noted below. Appendix A to this standard pro-testing activities, vides guidance regarding the implementation of the software unit testing approach, and Appendix B to the

• Environmental conditions and special controls, standard provides context regarding software engineer- equipment, tools, and instrumentaticn needed for ing information and testing assumptions that underlie the accomplishment of testing, the software unit testing approach.

• Test instructions and procedures incorporating fhe To meet the requirements of 10 CFR 50.55a(h) and requirements and acceptance limits in applicable Appendix A 510 CFR Part 50 as assured by complying design documents, with tb -nteria of Appendix B to 10 CFR Part 50 ap-

• Test prerequisites and the criterra for meeting them, 5

Copics are avahble al cuncnt rates from the U.s. Government Pnntmg omce. ro. Ikn 37082. washmgten. DC 20402-9328 (telephone

• Test items at:d the approach taken by the testing h

t j (202)$12-2249). or from the National Techmcal Information service by w nting NTIS at !285 Por1 Roy al Roed, Spongricid. VA 22161. Copics a'e r pmgram,

(/ avatlable for inspection or copying for a fee from the NRC Public Docu-ment Room at 2120 L street Nw., washmgton. DC; the PDR's maihng .d.

• Test logs, test data, and test results, dress is Mail stop LL-6. ,Vashington. DC 20$55-0001; telephonc , ,

(20:304-3m famo2)634-3m .

Acceptance criteria, 1.171 - 3

1

• Test records indicating the identity of the tester, the 3. TEST PROGRAM RECORDS type of observation, the results and acceptability, Criteria VI, " Document Control," and XVII, and the action taken in connection with any " Quality Assurance Records," as well as 10 CFR 21.51, deficiencies, require the control and retention of documents and Any of the above information items that are not records affecting quality. In addition, Criterion 111, present in the documentation selected to support soft. " Design Control," requires that design changes be sub-ware unit testing must be incorporated as additional ject to design control measures commensurate with items. those applied to the original design. Preservation of testing products is discussed in section 3.8.2(4) of

2. TEST PROGRAM IEEE Std 1008-1987. Since design control measures Criterion XI, " Test Control," requires establish- must be applied to acceptance criteria for tests and since ment of a test program to ensere that all testing required some software testing materials are frequently re-used to demonstrate that structures, systems, and compo. and evolve during the course of software development nents will perform satisfactorily in service is identified and software maintenance (for example, regression test and performed in accordance with written test proce- materials), such materials should be configuration dures that incorporate the requirements and acceptance iteras under change control of a software configuration limits contained in applicable design documents. The management system? Additional information on this two aspects of test coverage that are particularly irapor. topic is provided in section A6 of Appendix A to IEEE tant for the unit testing of safety system software are Std 1008-1987, coverage of requirements and coverage of the internal 4. INDEPENDENCE IN SOFTWARE structure of the code. VERIFICATION 2.1 Coverage of Requirements Criterion 111. " Design Control," imposes an inde.

pendence requirement for the verification and checking For safety system software, those requirements identified as essential to the safety determination6 must oW adequacy ch Mgn, requng dat hw per-sons who verify and check be different from those who be tested. Section 3.2.2(5) of IEEE Std 1008-1987 sug-accomplish the design. Therefore, independence is an J gests consideration of expected use of the unit in the de-additional requirement for software unit testing. Either termination of features to be tested. All features and as-those persons who establish the requirements-based sociated procedures, states, state transitions, and associated data characteristics essential to the safety de- elements for a software unit test must be different from those who designed or coded the software, or there termination must be included in the testing.

must be independent review of the establishment of the 2.2 Caerage of Internal Structure requirements-based elements. The guidance in section Section 3.1.2(2) of IEEE Std 1008-1987 specifies A7 of Appendix AtoIEEE Std 1008-1987 provides ac-statement coverage (covering each source language ceptable ways to meet this requirement for software statement with a test case) as a criterion for measuring unit testing. These independent persons must be suffi-the completeness of the software unit testing activity. ciently competent in software engineering to ensure that software unit testing is adequately implemented.

Statement coverage is a very weak criterion for meas.

uring test completeness [See Beizer 7 and NUREG/ 5. OTilER STANDARDS CR-6263 ].8 Therefore, the staff does not endorse state-

. Section 1.3 of IEEE Std 1008-1987 references ment coverage as a sufficient coverage criterion for ANSI /IEEE Std 729-1983,"lEEE Standard Glossary software unit testing. For safety system software, the Software Engineering Term,nology," i and ANSl/

unit test coverage criteria to be employed should be IEEE Std 829-1983,"lEEE Standard for Software Test identified and justified.

Documentation " These referenced standards should be treated individually.

If a referenced standard has been incorporated sep-

+ Regulatory outae i.i?2. son are Reyuirements seceifications for n'8- arately into the NRC's regulations, licensees and appli-ital Computer sohw are Used in safety sy stems of Nuclear Power Plants." .

endones IEFE Std 830-1993. "lEEE Recommended Practice for Soft. Cants must comply w.th i that standard ns set forth in the ware Requirements speciricatter.s."

7noris Deuct,Sopware Testmg Techmques, %n Nostrand Reinhoid,1990. ' Regulatory cuide 1.169 endones IEEE Std 828-1990. "lEEE Standard for sohware "onfiguration Management Plans," and IEEE Wi

'S. Seth et al.,"thgh Integrity Schware for Nudear Power Plants: Candi- 1042-1987." LEE Guide to software Configuration Management," to date ouulelines, Technical Basis and Research Needs." NUREGl prmide guidance for general software configuration management plans CR-6263, June 1995 and the6r implementation.

1.171 - 4

regnlation. If the referenced rtandard hasbeenendorsed ting is intended or approved in connection with the is-in a regulatory guide, the stendard constitutes a method suance of this proposed guide.

acceptable to the NRC staff of meeting a regulatory re-Q,O quirement as described in the regulatory guide. If a ref- Except in those cases in which an applicant pro-erenced standard has been neither incorporated into the poses an acceptable alternative method for complying l NRC's regulations nor endorsed in a regulatory guide, with the specified portions of the NRC's regulations, licensees and applicants may consider and use the in- the methods described in this guide will be used in the formation in the referenced standard, if appropriately evaluation of submittals in connection with applica-justified, consistent with current regulatory practice, tions for construction permits and operating licenses.

D, IMPLEMENTATION This guide will also be used to evaluate submittals from operating reactor licensees that propose system modiD-The purpose of this section 5 to provide informa- cations voluntarily initiated by the licensee if there is a lion to applicants and licensees regarding the NRC clear nexus between the proposed modifications and staff's plans for using this regulatory guide. No backfit- this guidance.

l v

.%/

t BIBLIOGRAPilY Beizer, Boris, Software Testing Techniques, Van Nos- Lawrence, J.D., and G.G. Preckshot, " Design Factors trand Reinhold,1990. for Safety Critical Software," NUREG/CR-6294, USNRC, December 1994.1 liecht,11., A.T. Tai, K.S. Tso, " Class IE Digital Sys-terns Studies," NUREG/CR-6113, USNRC, October Seth, S., et al., "lligh Integrity Software for Nuclear 1993.1 Power Plants: Candidate Guidelines, Technical Basis and Research Needs," NUREG/CR-6263, USNRC, Institute of Electrical and Electronics Engineers,"Stan-June 1995.1 dard Criteria for Digital Computers in Safety Systems of Nuclear Power Generating Stations," IEEE Std USNRC, " Criteria for Digital Computers in Safety 7-4.3.2, 1993,3 Systems of Nuclear Power Plants," Regulatory Guide 1.152, Revision 1, January 1996.2 Lawrence, J.D., " Software Reliability and Safety in Nuclear Reactor Protection Systems," NUREG/ USNRC, " Standard Review Plan," NUREG-0800, CR4101 (UCRL-ID-ll7524, Lawrence Livermore February 1984.1 National Laboratory), USNRC, November 1993.1 Single copies of regulatory guides may be obtained free of charge by wnt-1Copics may be purcheed at current rates from the U.S. oovernment Prin.

ting Office, PO. Don 37082, Washington, DC 20402-9328 (telephone ing the Office of Administranon, Pnnting. Graphics and Distribution (2021512-2249) or from the Nanonal TechnicalInformation Semce by D'anch. U.S. Nuclear Regulatory Commission. Washington. DC 20555-0001; or by fax at (301)415-5272 Copics are available for in-writing NTis et 5285 Port Royal Road.Spnngrictd. VA 22161, Copies are av allable for inspection or copying for a fee from the NRC Pubhe Docu. spection or copymg for a fee from the NRC Pubhc Document Room at 2120 L Street NW. Washington, DC; the PDR's maihng address is Mail ment Room at 2120 l. Street NW., Washmgton. DC; the PDR's maihng ad.

stop t.L-6, Washington, DC 20555-0001; telephone (202)634-3273; f as dreu is Mail $ top I.14. Wuhington. DC 20555-0001; telephone (202)634-3273, fax (202)634-3343 (202)634- 3343.

O O

1.171 - 6

REGULATORY ANALYSIS rh

( ) A separate regulatory analysis was not prepared for this regulatory guide.The

() regulatory analysis prepared for Draft Regulatory Guide DG-1057," Software Unit Testing for Digital Computer Software Used in Safety Systems of Nuclear Power Plants," provides the regulatory basis for this guide. A copy of be regulatory analysis is available for inspection and copying for a fee at the NRC Public Document Room,2120 L Street NW., Washington, DC; the PDR's mailing address is Mail Stop LL-6, Washington, DC 20555-0001; phone (202)634-3273; fax (202)634-3343.

r Printed a recycled paper rh Federal Recycling Program 1.171 - 7

l 1 '

l\l 1

N U

C WL e

P E E

N A

S A A HR N R U L I T

YO F GE N FF T GI OC R A OUT LE PL ,N AD RB I

D T S VU AS TI CO RA T ENE 2 Y T US 5 C E 0

SS

.E 5O S 5

3 5

0

- M 0

0 0M 0I 1 S S

I O

N e

P O

S P TIF E AR R GS M U ET e

I TS AC NN RD NLA S

O. CFS E G EM 4

7 SM P L A

I D

s

. 0$!!E o ar .5ia *=

, llfljj