ML20044C202
| ML20044C202 | |
| Person / Time | |
|---|---|
| Issue date: | 02/28/1993 |
| From: | NRC OFFICE OF INFORMATION RESOURCES MANAGEMENT (IRM) |
| To: | |
| References | |
| NUREG-BR-0167, NUREG-BR-167, NUDOCS 9303190127 | |
| Download: ML20044C202 (64) | |
Text
.
!)
= NUREG/BR-0.167 m.-
.~
hNb'
^q '_
- N$'"'?
y h
+
2.
y
~~
(
f i
'v b7 4 4.
t c
v' 3
(
s f
t 7.
<l.
' N
,.p,,
7
'.'d $
n n
%~
v
^
i,~-
kiin
. y j.#fe#J s
., ~ -
-4 g;
-;g
-Q.-
t g [.r y 4. : - c'e s
ad i:
4I.
Sonware QualityAssulan$32s6:
- 3. mm
... an, Aje> p.s 4
4 j:n e e
ok3 y,
.- e s,g IVogra,m and Guidelinesp' 1
4 k
+.:N%
,)
+
71 my c.,yw <i-i*
'.5,.
1 d
x i
t i '
.;.2 ;.
g-:l,
~l
)
v
.i
< <.2 y
~
m.
j 4
1 v
w y
?o
~
1 D
L-1
)
I a
i e
L e.
)
? U.S.LNuclear Rhgulat6ry" Commission 1
l 9303190127 930228 PDR NUREG BR-Ol67 R PDR
.i
I
~
- +
O 4
~.
s:.4.
i i
- l f
~
-i R i
- f
- .
. s
(
f
)
< j !. c 5
3
. p. ?
[]f' y s;'j '
'rr i
).
s-e SoftwareQualityAssursnch aaa
'hgram and Gdidelines s
e l
i 1
February 1993" i
l
)
i I
Division of Information Support Services Office of Information Reso'urces Management U.S. Nuclear' Regulatory Commission -
Washington, DC 20555.:
s ABSTRACT This document offers guidance to both NRC organiza-shown under references, and therefore meets the current tions and NRC contractors in the development and main-industry standards for the operational levels of software
+
tenance of software for use try the NRC staff.
described herein.This document may provide guidance but does not address the complex issue of software quality This document is based on various industry standards, for software used in n3 clear plants.
iii
n CONTENTS Page.
Abstract................
iii Acknowl edgem ents......................................................................
. ix 1
Int rod uc tion...................................................................
I 1.1 Purpose..................................................................
I 1.2 Scope and Applicability......................................................
I 13 The NRC Software Development / Sustaining Engineering Er vironm ent.......................................
I 1.4 Use of This Docum ent......................................................
2 1.5 Organization of his Document...............................
2 1.6 Maintenance of his Document.....................
2 1.7 Style Used in This Document...................................................
2 2
n e Sof tware L.ife Cycle............................................................
.5 2.1 Concepts and Definitions....................................................
5 2.2 Requirements Definition......................................................
5 23 Design....................................................-.............
5 2.4 I m pl ementation...................................................... -......
5 2.5 Qualification Testing.........................................................
5
- 2.6 Installation and Acceptance...................................................
6 2.7 Operations and Sustaining Engineering..........................................
6' 2.8 Retirement and Archiving............................................'.........
6 3
Verification and Validation............................................................
7 3.1 Concepts and Definitions......................................................
7 3.2 Verification and Validation Activities............................................
7 3.2.1 Verification and Validation Planning Activities...............................
7 3.2.2 Formal Life Cycle Reviews and Audits......................................
8 3.23 Formal Peer Inspections..................................................
10 3.2.4 Testing.................................................................
10 33 Techniq u es and Tools........................................................
11 4
. Documentation and Deliverables.....................................................
13 4.1 Concepts and Definitions.....................................................
13 4.2 Software Proj ect Plan.........................................................
13 '
43 Software Requirements Documentation..........................................
13 4.4 Software Design Documentation 13 4.5 Software Implementation Documentation........................................
13 4.6 Software Verification and Validation Documentation..............................
13 4.7 User Docum entation.......................................................
14 4.8 Oth er Docum en tation........................................................
14 4.9 D eliverabl es..............................................................
14 4.9.1 Documentation Deliverables............................................
14 4.9.2 Soft ware Deliverabl es.....................................................
14 4.10 Techniqu es and Tools.......................................................
15 v
u a
,-r ij
. Page
, 5-
' Project Managem ent..................................................................
17-
)
'5.1 Concepts and Definitions '......................................................
.17 f
5.2
- Project Planning and Organizing................................................
.17.
5.2.1 - Required Inputs to the Contract.................................... -........
17 4
4 5.2.2 Estimatin g '............................................................. :
s 5.23 Methodology, Standards, and Procedures...................................
18!
5.2.4 The Software Project Plan...............................................
-18:
53 Project Tracking and Oversight.................................................
19 5.4 Su pplier Control......................................... 2..................
20 5.5 M etrics....................................................................
. 20 ?
[
5.6 Security...............................................................,....
20L 5.7 Training...................................................................
20' 5.8 Risk Managem ent...........................................................
20:
5.9 Techniqu es and Tools.........................................................
21-l 6
Configuration Management...........................................................
23 t
6.1 Co.icepts and Definitions......................................................
~ 23
. 6.2 Baselines.....................................................................
23 63 Chan ge Con trol.............................................................
24:
6.4 Status of Baselines and Changes................................................
24 :
6.5 Software Development Library................................................,
'24 6.6 Software. Access. and M edia Control............................................
124 Co' figuration Audit s ;...........,..............................................
24 6.7 n
r 6.8 Techniqu es and Tools............................................ w............
25 7
Nonconformance Reporting and Corrective Action......................................... 7.1 Concepts and Definitions.......................................................
27 7.2 Activities.................................... 2..............................
27 7.2.1 Nonconformance Detection and Reporting.................................
~27, t
7.2.2 Impact Assessment and Corrective Action...................................
28-7.23 Tracking and Management Reports.........................................
28.
'73 Interrelationships.......................................................... s..
L28 1
7.4 Techniqu es and Tools.........................................................
28i j
8
. Quality Assessment and Improvement...................................................
29 8.1 Concepts and Definitions................................. ;....................
29 8.2 Responsibility for Quality Assessment and Improvement...........................
29
}
83 Documentation for Quality Assessment and Improvement..........................
29 :
8.4 Q uality Assessments.................................... 2...............
. 29 :
8.5 Quality Records Collection. Maintenance, and Retention.............................
29:
8.6~
Quality Improvement.................................. 2........ r..............
- 29 ~
8.7 Techniqu es and Tools.......... 2..............................................
29 9
' Software Developed Before Issuance of This Document.................................... -
/33
. Appendix A Sample Software Project Management Plan.................................
35 l
i Appendix B Glossary............. 2.......................... ;............ 2......
55 Appendix C ' ' Reference Docum ents.................................................,
57.
~
}
TABLES Page 1 Summary of Typical Ilfe Cycle Activities and Documents.................................
3 3-1 Verification and Validation Activities by Major I.ife Cycle Activity..........................
8 3-2 Formal life Cycle Reviews and Audits..................................................
9 8-1 Assessments of Products and Processes Used in Software Development..
30 FIGURE 5-1 Table of Contents for a Software Project Plan........................................... -.
19
'h vii
..i.
F ACKNOWLEDGEMENTS i
' he Office of Information Resources Management process were: Emily Robinson, Wil Madison and John wishes to acknowledge the contributions of NRC staff and Voglewede, IRM; Frank Coffman and Ixo Beltracchi,'
contractors in developing this document. He final tex-RES: Jack Sprauf and John llockley, NMSS; Jim Stewart, tual content was organized and written by Frank J.
Ralph Caruso, and Tony Mendiola, NRR; Steve Arndt, Douglas, SOFIRAN, Inc. NRC staff who participated in AEOD; and Mark Stella. ACRS.
reviews and discussion briefings during the development r
5 I
i%
a y
L-1 ' INTRODUCTION 1.1 ' Purpose '
1.3 The NRC Software > Development /
Sustaining Engineenng It is the purpose and intent of this document to offer guid-Environment ~
ance to both NRC organizations and NRC contractors in the development and maintenance of software for use by There are three types of organizations involved with NRC the NRC staff.
software: the regulated industry, NRC contractors, and '
NRC staff. nese guidelin es do not apply to the regulated -
1.2 Scope and Applicability industry. NRC contractors develop and maintain two genu eral types of application software: 1) technical / scientific s Software quality assurance is the planned and systematic and 2) administrative! management information systems; pattern of all actions necessary to provide adequate confi.
(MIS). Minimal software development and maintenance
,t dence that a software product conforms to established are done directly by the NRC staff.
]
tecimical requirements.Thus, the scope of software qual-ity assurance includes both management and technical as-De roles of NRC staff and NRC contractors in software 1 pects of software development and maintenance. ncre-development and maintenance can be divided into three -
fore, this document provides guidelines for: the software categories: sponsors, developers, and users. A sponsor is 1
~
life cycle; verification and validation activities; documen-the NRC organization that sponsors and manages the,
tation and deliverables; project management; configura-software development / maintenance effort. The sponsor g
-tion management, nonconformance reporting and cor-acts as the acquirer or buyer for the user, A developer is -
rective action; and quality assessment and improvement.
the organization, usually a contractor, that develops or,
maintains the software. A' user is the organization whoi Three levels of software are defined to make clear the :
utilizes the software product produced by the developer L l
wide variety of software used by the NRC.He three lev-Re user is mvolved in defmmg regmrements and should -
els are:
be made a partner during the development effort to help ;
l ensure that the product being built will meet the user's 1.
Level 1 Software-Technical application software used in a safety decision by the NRC (an example De authority for categorizing the software to be devele a'
would be R121AP5) oped (either Level 1, Level 2, or Level 3) resides with the sponsor.The user's concurrence with the categorization 1
- 2. ' Level 2 Software-Technical or non-technical ap-should be sought. The Information Resources Manage-j
. plication software not used in a safety decision by the ment QRM) organization is avalable for consultation.
NRC (an example would be an agency financial soft- -
during the categonzation process.--
ware system)
- l The development and maintenance of software is a" project, i.e., it has definitive start and end dates and a 3.
Level 3 Software-Technical or non-technical ap.
product (s)is delivered upon completion ~.Iloth the spon '
plication software not used in a safety decision and.
sor and the developer assign responsibility for the suc- -
having local or limited use by the NRC (examples cessful completion of the project to a project manager,-
t
'I would include a macro for Lotus 1-2-3)
- The IRM Officeis responsible for the coordination of the ;
l NRC software quality assurance (SQA) initiative embod-The guidelmes in this document apply to Level 1 and ied in this document, IRM is responsible for maintaining'.
Level 2 software only; they do not apply to Ixvel 3 soft-this document.
ware or any other software.
The IRM organization is also responsible for the estab -
The~ degree of applicability of these guidelines will de-lishment and coordination of the NRC SOA Working:
pend on the level of software being developed,its pur-Group. The objectives of this working group are to:
pose and use, ~and a managerial judgment of the cost-
'~
1 effectiveness of each software quality activity. Most L f acilitate communications (e.g., successes, lessons
{
projects should incorporate venfication and validation,-
learned)about software development and mainte-configuration management, and documentation control nance among the NRC organizations involved sith -
- activities.
acquiring and using software
~
3 i
L
^-
2.
Facilitate technology transferof Ihe best practices in Section 8 addresses quality assessment and improve-e the management and technical aspects of software ment development and maintenance Section 9 discusses application of the guidelines in ume to schware beloped hefom isso 3.
Provide a focal point for improvements to the guide-ance o e domment lines in this document Table 1-1 shows an overview of a typical software life cy-1A Use of This Document cle. It is meant as a quick-look reference. it contains only.
majoractivities performed and documentation produced.
He SQA Working Group is chaired by IRM/DISS and It is not meant to be complete listing of all activities per-has members from each major office involved with soft-formed or documents produced.
ware development and maintenance: ACRS,AEOD, IRM, NRR, NMSS, and RES.
1.6 Maintenance of This Document This document will be used by sponsor project managers The maintenance of this document is the responsibility of as a guide in developing inputs (e.g., the statement of the IRM organization. Changes to the document will be work) to the request for proposal for software to be devel.
issued as change pages as required. When the numberof oped, and by developer project managers as a planning change pages is deemed to be excessive, a new version of ~
tool. It can also be used as a software quality assurance the document will be published.
reference by sponsors and developers.
Suggestions for improvement are solicited from the en-He guidelines in this document are not intended to be tire NRC software community: sponsors, developers, and users.
applied rigidly. Rey should be used within the context of NRC pohey and Federal standards, as applicable to the project at hand, as well as with cost-effective manage-1.7 Style Used in This Document ment and engineering judgment based on past experi-ence.
Most of this document is writt en using verbs in the indica-tive mood.The indicative mood is the standard mood of This document applies to all software currently in use, be.
verbs, for example-ing developed, or planned for future development. Own-ers of software developed prior to the publication of this The software life cycle defined in this sectionprovides the document should read Section 9 in particular, Future basis for planning and implementing a software develop-software plans should implement all major elements of ment or maintenance project.
this document, but the extent ofimplementation must be decided as pan of the planning process by the sponsor and He life cycle consists of the following major activities:
developer, requirem ents definition. design, implementation, qualifi-cation testing, installation and acceptance, operations 1.5 Organization of This Document and sustaininE engineering, and retirement and archiving.
In addition to this introductory section, When the m. tent is to communicate explicitly a suggestion o
Section 2 defines the software life cycle.
or guideline to the sponsor or developer, we have chosen to use the imperative mood of the verb.nree examples follow *'
o Section 3 discusses verification and validation activi-ties.
Define the requirements so that they are correct, comp!cte, verifiabic, consistent, and technically fea-r o
Section 4 identifies deliverables, m.cluding required cible.
documentation.
o Section 5 addresses project management.
Perform planning activities for verification and vali-dation in parallel with requirements definition ac-tivities.
o Section 6 addresses ccmfiguration management.
Section 7 discusses nonconformance reporting and Require the developer's approach to quality assess-corrective action.
ment and improvement to be documented.
2
III liil!ll!I,l lit IIillt lI!
i
,} ibililli,b t hll!
1 l
!iii j
j li ! !illli: I l1l!!!!
Iliii j
[
II, IIlllIrlil0hltIllii illd!
Illi,ild!Ill1III!iii l
t 11!!
ph ill!!Idilililli 1111 I!Il 1Ji dil 3
j i
Il lilldfili ll,llIlll!!!!!!!ii j
5 j
lI}[
h I
1 3
o-L I
)
i-1,114 I}ll
!,ildji!!!ii j
3 l}U!lll1 llll tilIIII Ju
]
1111 dill 11111
~
~ ~ --
1 i
i j
n a
2 THE SOFTWARE LIFE CYCLE' q
2.1 Concepts and Definitions stmints, attributes, and external interfaces. ne require-a ments form the basis for the sofIware plans, products, and The software life cycle defined in this section provides the activities.
basis for planning and implementing a software develop-y ment or maintenance project. He life cycle consists of Ensure that the documented requirements define the re-
~ !
~
the following major activities:
sponse of the software to anticipated classes ofinput data (including erroneous data) and provide the information -
1.
Requirements definition and detail necessary to design the software (e.g., mathe-d
~
matical models, equations, data requirements)..
2.
Design Define the requirements so that they are correct, com-plete, verifiable. consistent, and t echnically feasible. Per-3.
Implementation form planning activities for verification and validation in :
j parallel with requirements definition activities.
4.
Qualification testing b
Because requirements inevitably change as a project 5.
Installation and acceptance evolves, manage the requirements throughout the devel-opment and mamtenance efforts m accordance with well- '
defined change control procedures (See Section 6).
]
6.
Operations and sustaining engineering 2.3 Design 7.
Retirement and archiving ne design process is the set of activities that results in j
Each major activityleads to specific products that can be the development, documentation,'and review of a soft-5 measured, evaluated, approved, and controlled. No strict ware design that meets the requirements defmed in the q
chronological constraints exist between major activities.
software requirements documentation.'
?!
The major activities may overlap in time and may be ap-a plied iteratively or recursively.
As the design evolves, events (e.g., additional m. sight into -
]
problem areas) may necessitate the modification of the Each major activityis accompanied by verification actions requirements documentationi Manage changes to re-that ensure that the products and processes of the major quirements documentation m accordance -with well-'
activity meet the requirements for those products and =
defined change control procedures.
processes. Verification actions are discussed in Section 3, and the documentation and software deliverables of the 2.4 : Implementation -
software life cyc!c are discussed in Section 4.
d Th e implem entation process is the set of activities that re ~
j
- The software life cycle presented here must be:
sults in software that has.
1.
Tailored to fit the scope of each development / main-1.
Been constructed in accordance with the design-tenance effort documentation and coding standards' 2.
Used within the context of NRC policy and Federal 2.
Undergone informal unit a' d integration testing _
n standards, as applicable to the project at hand, as well as with cost-effective management and engi-
- As the software is implemented, events (e.g., additional.
neering judgment based on past expenence nsight into data flow patterns) may necessitate the modi '
. fication of the design, requirements, and/or verification Some projects will not encounter all major activities. '
and validation documentation. Manage changes to docu- -
- {
~
. mentation in accordance with well-defined change ccm--
l 2.2 = Requirements Definition:
tml pmcedures.
j The requirements definition process is the set of activities 2.5 Qualification Testing that results in the specification, documentation, and re-view of the requirements that the software pmduct must The quahfication testing process is the set of activities as-
. satisfy, including functionality, performance, design con -
sociated with:
l
's
'5-
]
1.
Formally testing the implemented software, using his stage of the life cycle usually concludes with the user 1est cases defined in the verification and validation accepting the software for operational use.De responsi-documentation, against the baselined requirements bility for the sustaining engineering and maintenance of defined in the software requirements documenta-the software may be assigned to an organization different tion from the sponsor and/or the developer of the software.
2.
Reviewing and analyzing the test resuhs to ensure 2.7 Operations and Sustaining that the implemented software meets requirements Engineering and that the software produces correct results forall test cases executed Opemtion of the software is conducted by the user in ac-cordance with the operation and usage instructions in the user's documentation. Sustaining engineering is set of To evaluate technical adequacy, the software test results software engineering and software maintenance activities can be compared to results from alternative methods, needed to such as:
1.
Retain the software's initial functionality and design 1.
Analysis without computer assistance integrity (software engineering) 2.
Other validated computer programs 2.
Remove latent errors (corrective maintenance) 3.
Experiments and tests 3.
Respond to new or revised requirements (perfective maintenance) 4.
Standard problems with known solutions 4.
Adapt the software to changes in the operating envi-r ament (adapth mahname) 5.
Confirmed published data and correlations Perform sustaining engineering activities in a traceable, 2.6 Installation and Acceptance planned, and orderly manner based on:
Section 3.2.4 discusses qualification testing in more de-1.
The major life-cycle activities described in Sections -
tail.
2.1 through 2.6 i
installation aethities include one or more of the follow-2.
The verification and validation activities described 58 in Section 3 1.
Installing hardware 3.
Updating the required documentation and software deliverables as described in Section 4 2.
Installing the developed / maintained software 4.
The project management activities described in Sec-3.
Integrating the developed / maintained software with tion 5 other components (e.g., oth er software components, hardware, data) 5.
He configuration management activities described in Section 6 4.
Reformatting or creating data bases 6.
He nonconformance reporting and corrective ac-5.
Verifying that all components have been included tion activities described in Section 7 7.
De software quality assessment and improvement Acceptance activities include:
aethities described in Section 8 1.
Execution of acceptance tests (which typically con-sist of some of the qualification test cases plus addi.
2.8 Ret.irement and Arch..iving tional test cases)
Retirement means the support for a software product is terminated, and the routine use of the software is 2.
Documentation of the acceptance of the software by prevented, ne software and its documentation are ar-the sponsor chived.
1 6
i
n
~
i 1
3 VERIFICATION AND VALIDATION-
]
~-
3.1 - Concepts and Definitions Subject the validation of modifications to previously vali-dated software to selective regression testing.The objec 1
. Verification is the process of ensuring that the products tives of regression testing are to:
and processes of each major activity of the life cycle meet
. the standards for the products and the objectives of that 1.
Detect possible errors introduced during the modifi-5 majoractivity. Validation is the process of demonstrating cation process '
that the as-built software meets its requirements. Testing i
is the process of detecting errors and verifying perform-ance. Testing typically includes unit,' integration, qualifi-2.
Ensure that the modifications have not caused unin-cation, and axeptance testmg.
tended adverse effects :
Independent ' verification and validation (IV&V)is verifi-cation and validation by an organization that is both tech-3.
Validate that the' modified software still meets
~
nically and managerially separatti from the organization specified requirements responsible for developing the software. Sponsors and us-3 ers of Level 1 software should together decide if the ex-pense of a separate IV&V contractor is warranted for theirproject.
M Verification and Validation A~ tivi c
g
- ties 7
Examples of verification activities include:
A This section discusses verification and validation plannirig ;
.i 1.
Formal majorlife cycle reviews and audits (e.g., Pre-activities (Section 3.2.1); formal life cycle reviews and j
liminary Design Review) audits (Section 3.2.2); peer inspections (Section 3.2.3):
1 and testmg (Section 3.2.4). Table 3-1 shows verificatton :
y and validation activities by major life-cycle activity. This
]
2.
Formal peer inspections (e.g., code inspections, table is intended to show the approximate time in the life documentation reviews) cycle that these activities are performed. It is not in-1 tended to be applied rigidly Like all of the guidelines in this document, management and engineering judgment, 4
3.
Informal tests (e.g., unit and integration testmg)
- in conjunction with cost-effectiveness decisions, must be a
used in the application of these guidelines to the project '
at hand.
l l
Testing is the primary method of software validation.
Qualification and acceptance testing, which are formal tests, are validation activities. Validation is accomplished
. by review and demonstration in a live or simulated emi-3.2.1 Verification and Validation' Planning M
ronment. ne objectives of validation activities are to en-Activities
~ sure that:
Planning forverification and validation takes place during x 1
1.
He as-built software correctly and adequately per-the sponsor's initial planning for the project (e.g,the pro.. -
design,and{ implementation majoractivities of the posahtage as well as durmg the regturements definition, -
forms allintended functions l
l'
~. 2. ' 2 De software does not perform any unintended I
cle. Planning activities include:
l l-function that either by itself or in combination with 1.
Development or tailoring of procedures for con 1 other functions can degrade the entire system ducting formal life cycle reviews 3.
All non-functional requirements (e.g., perform-k ance, design constraints, attributes, and external in-2.
pevelopment or tailoring of procedures for review-
- terfaces)are met mg documentation and other dehverables.
- Software validation activities include the development of 3.
' Development or tailoring of procedures for con-1 test plans, test procedures, and test reports-ducting inspections
~
7' l
Table 3-1. Verification and Validation Activities by Major Life Cycle Activity i
Major Life Cycle Activity Verification and Validation Activities Inspect requirements -
. Requirements Definition Develop overall verification and validation plan Conduct the Software Requirements Review Inspect design Design Develop qualification test plan Develop acceptance test plan Conduct the Preliminary Design Review Conduct the Critical Design Review Develop unit test plans Implementation Inspect unit designs, unit code, and unit test plans Perform unit testing Inspect unit test results Develop integration test plans Inspect integration test plans Perform integration testing Inspect integration test results Develop qualification test procedures Perform qualification testing Qualification Testing Write qualification test report Develop acceptance test procedures Perform acceptance testing Installation and Acceptance Write acceptance test report Perform, as appropriate, the verification and Sustaining Enginecting and Operations validation activities defined above for requirements definition, design,- implementation, qualification testing, and installation and acceptance Perform regression testing as well as new tests for all levels of testing, as appropriate 4.
Definition of a detailed test methodology, including 3.2.2 Formal Life Cycle Revims and Audits -
standards for test documentation, specifically for test plans, test procedures, and test reports for both qualification and acceptance testing A formal review, with sponsor and developer manage-ment and technical perscmnel participating, is held at or 5.
Preparation of a validation matrix showing the rela-near the end of a major activity of the life cycle.The ob.
tionship of software requirements to the software jective of the formal reviews is to evaluate the deh.verabic architecture down to the unit level and to the tests used to verify the requirements products, the progress, and to a lesser degree, the.
processes of the most recent life-cycle phase. Table 3-2 6.
Identifying the need for development and test tools, surnmarizes the formal major life cycle reviews and audits equipment, and data by major life-cycle activities.
r 8
l Table 3-2. Formal Life Cycle Reviews and Audits.
Major 1ife Cycle Activily Formal Reviews and Audits Requirements Definition Software Requirements Review Design Preliminary Design Review Critical Design Review Implementation Qualification Test Readiness Review Oualification Testing Software Configuration Audit Installation and Acceptance Software Configuration Audit Post Mortem Review Operations and Sustaining The formal reviews and audits Engmeermg above, as applicable
'Ihe products associated with each formal review are:
3.2.2.1 Software Requirements Review Conduct the Software Requirements Review at the end 1.
The documents to be reviewed (e.g., the software of requirements definition.'lhe primary objective of this requirements documentation for the Software review is to assure that the sponsor and Ihe developer un-Requirements Review) derstand and agree on the intent, completeness, verifi-ability (through testing or other means), consistency, and technical feasibility of the requirements. Secondary ob-2.
The agenda for the review jectives are to review other documentation pmducts available at this time, including, for example, the Soft-ware Project Plan and the overall verification and valida-3.
The hardcopy presentation materials tion plan.
4.
'lhe minutes that document the activities and results 3.2.2.2 Preliminary Design Review Conduct the Preliminary Design Review when the pre-liminary design (software architecture) has been de-5.
'Ihe updated documents that were reviewed signed. The primary objective of this review is to assure that the prehmtnary design is complete (meets all the requirements), venfiable (through testing or other Allow sufficient time for sponsor review participants to rneans), consistent, and technically feasible.
review the documents prior to the review (2 to 3 weeks.
say). Identify in the agenda the review participants and 3.2.2.3 Critical Design Review their specific responsibilities during the review. Assign a perscm to capture key discussion items and actions items, Conduct the Critical Design Review when the design is especially those related to specific assignments for updat.
complete. Design completion criteria should be defined '
ing the documentation that is the object of the review.
clearly in the Software Project Plan. Suggested design Document in the review minutes all proposed revisions to completion criteria are:
the reviewed documents and all actual changes to the re-viewed documents, and place the updated documents un-1.
All software units have been identified and all inter-der ccmfiguration control after approval by the sponsor.
faces between and among the units have defined The paragraphs below discuss each formal life cycle re-2.
All elements of the database have been defined view and audit.
down to the data item level.
9
s De primary objective of this review is to assure that the 3.
Make available,if requested by the sponsor, records design is complete (meets all the requirements and meets that document the results of all peerinspections design completion criteria), verifiable (through testing or other means), consistent, and technically feasible.
For level 2 software, encourage the developer to work toward subjecting each intermediate product and final -
l 3.2.2.4 - Qualification Test Readiness Review product of development and maintenance to an internal '
{
peer inspection.
Conduct the Qualification Test Readiness Resiew when integration testing and the qualification test procedures See Section 3.3 for more discussion of formal peer inspec-j are complete. The primary objective of this review is to tions.
t assure that the as-built software; the software documen-tation; and qualification test data, test tools, test configu-3.2.4 Testing ration, and test team are ready for formal qualification testing.
Testing is the process of exercising or evaluating a soft-ware product or part of a software product by manual or 3.2.2.5 Software Configuration Audit automated means to verify that it satisfies specified requirements or toidentify differences between expected Conduct the Software Configuration Audit twice, first at and actual results. Testing approaches depend on the the completion of qualification testing and second at the number oflevels of testing. For most cases, four levels of completion of acceptance testing.ne primary objective testing are sufficient:
of this audit is to ensure that the as-built software:
1.
Unit testing 1.
Meets its requirements as documented in the soft-ware requirements documentation 2.
Integration testing 2.
Conforms to its tecimical documentation 3.
Qualification testing Acceptance tedng i
3.
Does not contain any unauthorized changes A unit of software is an element of the software design 3.2.2.6 Post Mortem Review that can be compiled or assembled and is relatively small (e.g.,100 lines of high-order language code). Require that Conduct the Post Mortem Review after the software has cach software unit be separately tested.
been accepted. The objective of this audit is to capture the lessons learned from the project for use by future Integration testing focuses on a collection of'related units projects.
that performs an identifiable functional requirement. Re-quire that integration testing be carried out. Both unit 1
3.2.3 Formal Peer Inspections testing and integration testing are classified as informal -
testing because a formal test plan is not required.
A formal peer inspection is a detailed examination of a product on a step-by-step or line-by-line basis. The pur.
-Qualification testing is the process that allows the spon-pose of conducting formal peer inspections is to find er.
sor to determine whether a software product complies rors. The group that performs a peer inspection is com.
with its requirements. Acceptance testing is the process posed of peers of the person who developed the product that allows the sponsor to determine whether a software to be inspected. Peer inspections are objective product complies with its requirements after it has been -
approaches that have been proved very effective in verify.
installed in its opemtional emironment.
ing that products meet requirements.
In many cases. acceptance tests will, to a large degree,'co-i t
For Ixvel 1 software, require the developer to incide with qualification tests. In some cases, qualification -
tests and acceptance tests are the same in all respects, in -
1.
Subject each intermediate product and final product which case the test hierarchy telescopes down to three -
[
levels of testing. Both - qualification testing and of development and maintenance (i.e., all documen, tation, all code) to an internal peer inspection acceptance 1esting are classified as formal testingbecause a formal test plan is required.
2.
Make available to the sponsor the written procedure Testing may be either requirements-driven or design-and the product standards that govern peer inspec-driven. Informal testing may be either requirements-tions driven or design-driven.
i 10
t 3.2.4.1 Design. Driven and Requirements > Driven 2.
Measure progress Testmg Design-driven or white-box testing is the process where
" Informal"in this case does not mean the tests are con -
the tester examines the internal _ workings of the code, ducted in a casual manner, just that no deliverable test Design-driven testing is accomplished by selecting input plan is required, the sponsor is not formally involved, the data and other parameters based on the internal logic witnessing of the testing is not required, and that the paths to be checked.he goals of design-driven testing in-prime purpose of the test is to find errors.
clude ascertaining correctness of:
3.2.4.3 formal Testing 1.
All paths through the code. For most software prod-ucts, this can be feasibly done only at the unit test For Ixvel 1 and Level 2 software defined in this docu-level ment, formal testing is always requirements-driven and its purpose is to demonstrate that the software meets its requirements.ne reader is cautioned not to confuse for.
2.
Interfaces between units mal testing with formal proof-of-correctness methods, '
[
which are formal techniques used to prove mathemati-3.
Size and timing of critical elements of code cally that a computer program satisfies its specifications.
Requirements-driven or black-box testing is done by se-lecting input data and other parameters based on the soft-1.
A sponsor-approved test plan and procedures ware requirements and observing the outputs and reac-t tion of the software. In addition to testing for satisfaction 1 Test witnesses l
of requirements, some of the objectives of requirements-driven testing are to ascertain:
I.
Computational correctness 4.
A test report 2.
Proper handling of boundary conditions, including extreme inputs and conditions that cause extreme If the software is to be developed and delivered in incre-outputs ments or builds, there may be incremental qualification and acceptance tests. As a practical matter, any contrac-3.
State transitioning as expected tually required test is usually considered a formal test; others are informal.,
4.
Proper behavior under stress or high load After acceptance of a software product, all changes to the product should be accepted only as a result of a formal 5.
Adequate error detection, handling, and recovery test. Include regression testing in all post-acceptance -
testing. Regression testmg mvolves rerunnmg previously used acceptance test cases to ensure that the change did Sometimes the term " operational testing"is used. Opem-not introduce error into previously accepted software.
tional testing is either the random or statistically-controlled application of the software in its actual emi-3.3 Techniques and Tools ronment or in a simulated version of the operational I
emironment. An example of such testing is the so-called Perhaps more tools have been developed to aid verifica-beta test use of an applications software package by indi-tion and validation of software (especially testing) than viduals typical of the intended user population. In the any other software activity. The tools available include terminology used above, such operational testing and code tracers, special-purpose memory dumpers and for-beta testing would be qualification testing and are matters, data generators, simulations, and emu;ations.
requirements-driven.
Some tools are essential for testing any significant set of -
software, and, if they have to be developed, may turn out 3.2.4.2 Informal Testing to be a significant cost and schedule driver. Ensure that t he need for test tools is examined during software design.
Require the developer to perform informal tests to:
1.
Ensure softwart units and combinations of software An especially useful technique for finding errors is the units are correct formal peer inspection. The formal peer inspection is 11 e
- c
+
y.ym t
1 1
7 performed by a' team, each member of which has a
- Ihis formal highly structured inspection process has
~
~
' well-defined role.The team is led by a moderator, who is been extremely effective in finding and climinating er,
!- formally trained in the inspection process 'Itc team in-rors. It can be applied to any product of the software ded
}
cludes a reader, wholeads the team through the item to.
velopment process,' including documents, design,.and.
1
- be inspected; one or more reviewers, who look for errors :
. code. One ofits important side benefits isthe direct feed-l in the product; a recorder, who notes the faults; and the
. back to the developer / author, often resulting in signifiJ author, who helps explain the product.
cant improvement in product quality.
i
+t
.l ce Iy
^l: 3 9
i i
i
'I l
4 i
i
[
t
-l
.. ) ;
5 g
+
e
- I c'i
.)
12 4
q j
i 1
4 DOCUMENTATION AND DELIVERABLES 1
l t
4.1 Concepts and Definitions 4.4 Software Design Documentation.
' 'lhis section identifies the documentation and software In software design documentation, specify the ovemil.
deliverables essential to a successful software develop-structure of tbc software so that it can be translated into
- j ment project. ' Itis section should be used as a starting code. Include in this documentation:
point to help determine a realistic set of documentation i
requirements and deliverables for the project at hand. A 1.
A description of the major elements of the software i
realistic set of documentation requirements will result as they relate to the requirements from tailoring the information in this section in light of past experience with similar projects, the size of the soft-2.
A description of the theoretical basis, physical '
ware, and spcmsor requirements. Small and short-model, mathematical model, control flow, data flow, duration projects will normally produce fewer documents control logic, and data structure.
'i or combine related documents.
3.
An identification and detailed definition of the soft-4.2 Software Project Plan ware units and data elements of the software archi-tecture.
- A result of the developer's planning process, a Software Project Plan is written by the developer and details how the developer will manage the software project.The Soft.
4.5 Software Implementat. ion ware Project Plan is discussed in detail in Section 5.2.4.
Documentation Software implementation documentation includes unit 4.3 Software Reymrements designs (usually presented as a commentary prologue to
- Documentallon the unit's source code)and the umt code itself.
Software requirements documentation specifies the 4.6 Software Verification and Valida-regmrements that the software to be devekiped/ main-tion Documentation -
tamed must meet. include m this documentation the fol-lowing, as applicable:
Software verification and validation documentation in-1.
Functionality-the functions that the software is to.
perform 1.
An overall verification and validation plan that in-cludes a description of:
2.
Performance-the time-related requirements of software operation such as speed, response time, a.. ' 'Ihe objectives and processes for each review '
etc.
andinspection
~b
'Ihe test methodology including the objectives 3.
Design constraints imposed on a.mplementation ac-of each level of testing (c.g., unit, integration, tivities-any elements that will restrict design op-qualification, acceptance) -
tions (e.g., specifying the hardware platform or the programminglanguage) c.
Contents of each level of formal test dacumen-tation (test plans, procedures, reports) -
4.
Attributes-characteristics of the software, its ac-M nand validation documentation ceptance, or use (e.g., portability, acceptance crite-o d W nWi@f min's -
na, access control, availability, maintamabih,ty, etc.)
inspections, and tests to requirements and dei.
5.
Ext ernal interfaces-intcractions with people, hard-ware, and other software 2.
Agenda, presentation materials, and minutes for -
formal life-cycle reviews and audits An item can be called a software requirement only if its
. achievement can be verified and validated. It is important 3.
Results of formal peer inspections that each software requirement be traceable throughout the stages of the software life cycle.
4.
Informal test plans for unit and integration testing.
13
5.
Informal test procedures for unit and integration 4.9 Deliverables.
testing Documentation deliverables are discussed in Section 4.9.1, and software deliverables are discussed in Sec.
6.
Informaltest reportsforunit and m.iegration testmg tion 4.9.2.
7.
Formal test plans for qualification and acceptance 4.9.1 Documentation Deliverables testing The sponsor must decide what the contract deliverables -
should be. For large projects, the following documenta-8.
Formal test procedures for qualification and acceP-tion deliverables are suggested:
tance testing 1.
Software Project Plan 9.
Formal te<t reports for qualification and acceptance testing 2.
Requirements documentation 3-0"*'811 **'ifiC"i " ""d ** lid ^ti " P 8" l
4.7 User Documentation In user documentation, include:
4.
Design documentation (delivered three times: in-itially at the Preliminary Design Review; updated at -
1.
A description of the user'sinteraction with the soft-the Critical Design Review; and updated after ac-ware, and a description of any required training nec.
ceptance testing)
~
essary to use the software 5.
Qualification test plan 2.
Input and output specifications and formats, inct ud-ing sample cases 6.
Qualification test procedures 7.
Qualification test report 3.
A description of the limitations of the software 8.
Acceptance test plan 4.
A description of anticipated errors and how the user can respcmd 9.
Acceptance test procedures 5.
For each error message, provide the message, an ex-
- 10. Acceptance report planation of the message, bow the message may have come about, and actions that may or should be taken
- 11. Userdocumentation For smaller projects. documents can be combined. For 6.
Information about obtaining user and sustaining en-example:
gineering support 1.
The requirements documentation can be combined 4.8 Other Documentation with the design documentation Other documentation may include the following:
2.
The overall verification and validation plan can be.
combined with the Software Project Plan 1.
Software Operations Concept 3.
Test plans and test procedure documents can be 2.'
' Standards and Procedures Manual combined 3.
Software Maintenance Manual 4.9.2 Software Deliverables
'Ihe decision about what software deliverables to require 4.
Software Engineering Notebooks depends on numerous ccmsiderations, including 14
\\
)
1.
Whether the software will be implemented and de-6.
Software and job control language necessary to es-livered in segments or builds (for large software
. tablish and maintain the software development li-pmducts, builds have been proved to be a very effec-brary live risk-reduction technique)
- 2.
What organization will perform maintenance and the emironment needed to perform maintenance.
g, ggf If the maintainer is different from the developer, a maxi-mal subset of the following list of possible software 9.
Non-developmental software -
deliverables should be chosen:
1.
source code 4.10 Techniques and Tools Numerous tools exist for generating documentation:
2.
Object code word processing programs, desktop publishing programs, graphics programs, spelling checkers, grammar checkers, 3.
Executable code etc.
There are numerous standards for documentation that 4.
Test cases for formal testing, including machine-readable test procedures should be consulted before deciding on the documenta-tion requirements to be levied on the developer.
5.
Requiredjob control language, e.g., to compile, link, Consult an experienced project manager for his/her crpe-load, and execute the software riences when deciding what deliverables to choose.
15
7 _
I-1.
Whether the software will be implemented and de-6.
. Software and job control language necessary to es.
livered in segments or builds (for large software tablish and maintain the software development li-products, builds have been proved to be a very effec-brary tive risk-reduction technique) 2.
What organization will perform maintenance and the environment needed to perform maintenance.
8.
SM irom if the maintainer is different from the developer, a maxi-r.
mal subset of the following list of possible software 9.
Non-developmental software deliverables should be chosen:
1.
source code 4.MdnWs and Ms Numerous tools exist for generating documentation:
2.
Object code word processing programs, desktop publishing programs, graphics programs, spelling checkers, grammar checkers, 3.
Executable code etc.
There are numerous standards for documentation that 4.
Test cases for formal testing, including machine-should be consulted before deciding on the documenta-'
readable test procedures tion requirements to be levied on the developer.
5.
Required job control language, e.g., to compile, link, Consult an experienced project manager for his/her expe-load, and execute the software riences when deciding what deliverables to choose.
15
9 i
5 PROJECT MANAGEMENT 5.1 Concepts and Definitions 5.2.1 Required Inputs to the Contract Assign the responsibility for each software development Hold the sponsor project manager responsible for devel-or maintenance effort within an NRC sponsor organiza-oping the following inputs to the contract: the statement tion to a project manager.nis sponsor project manager of work, top-level schedule, list of deliverables,.
should be an experienced NRC employee trained in man.
identification of applicable standards, and software speci-aging the technical and personnel aspects of the project.
ficauon.
He or she is assigned by sponsor management the respon-sibility for the successful completion of the project,i.e.,
The statement of work defines the' activities required of for meeting tecimical objectives within cost and schedule the developer. De statement of work should:
constraints. Delegate to the sponsor project manager the 1.
Define what the developer must do, not what the authority to negotiate. via the Government's Cont racting Officer, commitments with the developer.
software must do (the software specification defines what the software must do)
Similarly, the developer is expected to assign a project manager who will be responsible for meeting the devel-2.
Contain explicit tasks modeled after the life cycle ac-oper's contractual commitments.
tivities defined in Section 2 and the verification and '
validation activities defined in Section 3 i
The Iwo basic project rnanagement activities, discussed in Sections 5.2 and 5.3, respectively, are: 1) project planning 3.
Identify the deliverable documentation and soft-and organizing and 2) project tracking and oversight.
ware required of the developer (See Section 4) 5.2 Project Planning and Organizing 4.
Require the developer to perform project planning.
Project planning and organizing involves:
and organization activities resulting in the Software Project Plan (See Section 5.2.4) 1.
Development, by the sponsor project manager, of required inputs to the contract, e.g., the statement 5.
Require the developer to perform project tracking of work, schedule, list of deliverables, identification and oversight activities and deliver periodic progress of applicable standards, and software specification reports as indicated in Section 5.3 2.
The definition, by the sponsor project manager, of 6.
Require the developer to perform configuration work elements necessary to develop or maintain the management ' activities as indicated in Section 6 -
- required software.The work elements are defined in a statement of work that will be a part of the ccmtract 7.
Require the developer to establish and maintain a with the developer nonc(mformance reporting and corrective action system as indicated in Section 7 3.
De establishment, by the developer and approval by the sponsor, of budgets and schedules for each 8.
Require the developer to establish and maintain a defined work element quality assessment and improvement program as in-dicated in Section 8 4.
The establishment, by the developer, of a project or-ganization for implementing the project; and assign-Develop the top-level schedule around the:
ment of work elements, budgets, and schedules to each organizational entity 1.
Formal life-cycle reviews and audits discussed in Section 3.2.2 (e.g., Software Requirements Review, 5.
Documentation of the overall plan for approval by Preliminary Design Review, etc.)
l the sponsor project manager 2.
Deliverables Consider requiring that the developer's organization as-signed to plan for and perform formal testing be different The list of deliverables should contain:
from and independent of the organization (s) that de-signed and implemented the software.
1.
The software end products
-17
-[
2.
Required documentation -
5.2.4 The Software Project Plan Require the developer to submit, for sponsor project 3.
Agenda, presentation materials, and minutes for manager approval, a Software Pmject Plan that appropri-f nnal renews ately and realistically documents the required software activities and contractual commitments. When approved 4.
Progress reports by the sponsor project manager, the Software Project Plan becomes the baseline management plan. Figure 5-1 shows a suggested table'of contents for the Software The identification of applici;1e standards may include:
Project Plan.
1.
Programming language standards (e.g., FORTRAN Section 1 of the Software Project Plan should be kept 77) brief. Because the plan should be kept up to date, con-sider requiring the developer to submit any changes to the
" #P 2.
Coding standards Section 2 documents the developer's management ap-3.
Documentation standards proach. The following paragraphs provide guidance to the developer for the contents of each subsection:
4.
De facto standards embedded in software and docu-1.
2.1-Planning Approach. Briefly describe the ap-mentation to be maintained proach used to plan the project.
The software specification documents the requirements the software is to satisfy. The software specification is 2.
2.2-Tracking and Oversight Approach, Briefly-often preliminary and subject to analysis and expansion by describe the approach used to track: technical the developer during the requirements definition proc.
progress, conformance to the planned schedule, and ess.The software specification should contain:
costs as related to ' actual work performed. Include approach to: supplier control; metrics; security; training;and risk management 1.
Technical goals and objectives 2.'
identification of users and their interaction with, 3.
2.3-Organization, Tasks, ~ and Responsibilities.
and use of the software Describe the project organization. Show how the tasks of the statement of work are assigned to re-3.
The characteristics presented in Section 4.3 4.
_2.4-Scheduling. Provide the initial, top-level 5.2.2 Estimating project schedule and the rationale for arriving at this schedule.
Both the sponsor and developer should derive estimates for the size of the software products and documentation, software development resources and costs, and critical 5.
2.5-Resources. Identify project resources, target computer resources.These estimates should he de-including staffing, software engineering facilities rived from in-house experience-based data using docu-and environment, and support tools. Identify Gov-mented procedures. Discuss overall projected software ernment Furnished Equipment (GFE) and Govern--
size (estimated combined with actuals) at each formal re.
ment Furnished Information (GFI) required by the '
view.
developer.
5.2.3 Methodology, Standards, and 6.
2.6-configuration Management. Identify and Procedures define project baselines. Include or reference -
procedures for: change ccmtrol; determining status Developers should work toward basing software planning of baselines, proposed changes, and implemented t
(and monitoring) activities on documented methodolo-changes; release control; the software development gies, standards, and procedures.
library; and code, access, and media control.
L IS
3.
33-Nonconformance Reporting and Corrective SECTION 1 -INTRODUCflON Action Approach. Dercribe the nonconformance re-porting and corrective action process, including 1.1 Project Background and Objectives n nconformance detection and reporting, impact assessment and corrective action, and tracking and -
1.2 Plan Scope and Organization management reports. Identify the interrelation-ships, if applicable, with the status accounting func-13 Plan Maintenance tion of cor'igui: tion management. Identify any techniques and tools used (e.g., use of a data base SECTION 2 - MANAGEMENT APPROACil 4.
3.4-Ouality Assessment and Improvement Approach. Describe the quality assessment and 2.1 Planning Approach improvement approach. Include the elements ad--
dressed in Section 8.
2.2 Tracking and Oversight Approach 5.
33-Deliverables. Identify all deliverables and the 23 Organization, Tasks, and Responsibilities dates they are due.
2.4 Scheduling 2.5 Resources ti.
3.4-Standards, Procedures, Conventions, and' Metrics.1dentify all standards, procedures, conven-2.6 Configuration Management -
tions, and met-ics to be used. Identify both product standards (e.g., documentation standards, coding standards) and process procedures (e.g., inspection SECTION 3 - TECIINICAL APPROACII and review procedum).
3.1 Implementing the Life Cycle Tasks of the 5.3 Project Tracking and Oversight -
Statement of Work Project tracking and oversight involves:
3.2 Verification And Validation Approach 1.
Monitoring, assessing, and reporting. technical 3.3 Nonconformance Reportm.g and Corrective Action progress 3.4 Quality Assessment and improvement 2.
Determining and reporting schedule and cost status Approach 3.5 Deliverables 3.
Developing and implementing corrective action plans as required q
7; p3 Monitoring, assessing, and reponing technical progress Table of C<mtents for a Software Project Plan requim imckmg actual results and performance ot' the software project against the Software Project Plan. Im-plementation of planned verification and validation, con-figuration management, and quality assessment and im-Section 3 documents the developer's technical approach.
provement activities are part of the ordinary imcking and The following paragraphs provide guidance to the devel-oversight functions.'Ihe key to monitoring progress on an oper for the contents of each subsection:
ongoing basis is to maintain communications at all levels of the developer and sponsor organizations, Use formal 1.
3.1-Implementing the Life Cycle Tasks of the mechanisms such as reviews and reports and informal
' Statement of Work. Describe briefly how each ma.
mechanisms such as rnectings and brainstorming sessions !
jor life-cycle task of the statement of work will be im-to keep project members and the project manager in-plemented.
formed. Track technical progress, costs, critical target.
computer resources, the schedule, estimates for lines of 2.
3.2-Verification and Validation Approach. De-scribe the verification and validation approach. In-Hold the developer project manager responsible for de-clude the elements addressed in Section 3.
termining and reponing schedule and cost status in terms i
19 1
3
L 1
p of variances from the. baseline plan. Require the devel-(e.g., accuracy, data access times, response time, E
oper to report the reasons for schedule and cost vari-number of errors found in requirements).
ances, e.g., unexpected problem complexity and changes in requirements.
2.
For each parameter, specify the requirement and develop a time-phased profile with tolerance bands Take co:Tective actions when the actual results and per-that depict the acceptabic range of performance as
~formance of the software pmject deviate significantly the project progresses.
from the Software Project Plan and current schedule. Ba-sic corrective actions may include adding staff, extending 3.
Plan for periodic analysis and predictions of these the work week, and/or upgrading (or downgrading) the parameters, especially in conjunction with formal skill mix.
Life-cycle reviews 5.4 Supplier Control 4.
Keep the sponsor project managerinformed of all If the developer plans to use subcontractors or vendors, unfavorable trends and the corrective action plans the sponsor and developer project managers should en-bemg mitiated to resolve them sure that:
5.6 Security 1.
The developer selects qualified subcontractors and vendors
'Ihe Computer Security Act of 1987 requires Federal; agencies to identify each computer system that contains sensitive information and to prepare and implement a 2.
The software standards, procedures, and product plan for the security and privacy of these systems. OMB -
requirements for the subcontract comply with the Bulletin No. 90-08 provides guidance for p.cparing such prime contractor's contractual commitments plans, but does not address the unique security require- '
ments of kwal area networks. NUREG/BR--0166,'In--
3.
A Software Project Plan as outlined in Section 5.2.4 nructions for Preparing Security Plans for Local Area Net-is required of the major subcontractors works in Compliance IVith _OMB Bulletin No 90-08.
provides guidelmes for preparing security plans for local area networks and contains OMB Bulletin No. 90-0S as 3.
Comrnitments between the prime contractor and an a ppendix. If the proposed project will be a sensitive ap-subcontractor are understood and agreed to by both plication IRM/DISS should be notified on the sensitive -
parties system suncy form.
4.
The prime ccmtractor tracks the subcontractor's ac.
5.7 Training tual results and performance against the commit-The right people properly toined are necessary for a suc-cessful project. Required training includes both manage-ment and technical training in the knowledge and skills of 5.
Potential technical and business risks are identified a variety of disciplines. Sponsor and developer project and managed managers need to evaluate training needs for:
1.
Sponsor management and technical personnel Plan to measure both the products being developed (the
. Developer management and technical personnel
. software and its documentation) and the processes being used. Process-related metrics (e.g., number of errors 3.
Maintainer management and technical personnel found in the requirements or design) are often useful in evaluating the programmatic risks involved in software 4.
Operations management and technical personne1' development.
When the evaluation is complete, invest in the required.
Establish a technical performance measurement pro-
- training, gram, using the following steps-1.
Develop a comprehensive list of key technical per-formance p.trameters, associated with both products initiate the risk management program while the techni-and processes, that can be predicted and estimated cal, schedule, and budget planning efforts are under way.
20
" Die following activities are typical of a risk management 5.9 Techniques and Tools program:
Numerous commercially available tools exist for:
' 1.
Identify, assess, document, and rank technical, cost, resource, and schedule risks 1.
Project management support 2.
Estimating software costs and schedules '
2.
Develop a risk mitigation plan 3
Formalize the risk management program Many organizations have made the commitment to pro-4.
Resiew the risk management program regularly vide in-depth training to software project managers.
i T
L 0
k e
I t
I
-I 21
.j i
(
L L
e 6 CONFIGURATION MANAGEMENT c.
Configuration management is implemented
.6.1 Concepts and Definitions for externally-deliverable products and for ap-For a project to be successful, the developer and sponsor propriate products used inside the organization must establish and maintain integrity of the software and its documentation as they evolve throughout the life cy-cle. Because requirements, the design, the code, and the d.
All projects have a repository for storing key test emironment can change significantly and often, it is software engineering elements and associated essential that change be managed successfully. Briefly configuration management records stated, configuration management is change manage-ment.
c.
He software baselines and configuration man '
agement activities are audited on a regular basis Fundamental to configuration management are the con-cepts of a baseline and change control. A baseline is a 3.
A group that is responsible for coordinating and im-document or software that has been formally reviewed and agreed upon by the developer and sponsor, that plementing configuration management for the r
thereafter serves as the basis for further development project exists oris utablished and that can be changed only through formal change con-
. Adequate resources and budget for performing con-trol procedures. Change control is the process by which a 4.
change to a baseline is proposed, evaluated, approved or figuration management activities are provided rejected, scheduled, and tracked.
5.
Members of the configuration management group -
Here are four major functions of configuration manage-are trained in the objectives, procedures, and meth-ment:
ods for performing their assigned activities :
i 1.
The identification and establishment of baselines 6.
ne configuration management activities are re-siewed with the project manager on a regular basis 2.
Controlling both changes to baselines and the re-lease of baselines 6.2 Baselines -
3.
Recording and reporting the status of baselines, Establish controlled and stable baselines for planning, manaE "8, and building the 8YStem. Explicitly identify as I
change requests, and implemented changes project baselines software products (e.g., source code, ob-
. ject code, test cases) and software process specifications:
4.
Verifying, through auditing, the correctness and (e.g., standards and procedures) that are needed to estab- ~
completeness of base, lines prior to release lish and maintain stability of the software activities.
Establish a naming or labeling system that:-
For a software configuration management program to be successful, experience has shown that most of the follow-1.
Uniquely identifies all project entities (e.g., docu-ing conditions exist:
ments, software elements, test cases) 1.
The content and status of the softvare and docu-2.
Identifies changes by revision or version mentation baselines are known at all times 3.
Uniquely identifies each configuration / version of!
i 2.
The developer follows a written configumi. ton man-revised software for use.
agement polig that has the follawing characteris-tics:
Establish the following baselines that will be controlled -
by the sponsor's configumtion control board (CCH)(See f a.
Responsibility for configuration management Section 6.3):
for each project is explicitly assigned L ne project management baseline consist ng of the i
b.
Configuration management isimpicmented on Software Project Plan, documented standards and =
products throughom the product's life cycle pnredures, and up-to-date budgets and schedules 23
.~
n!
I
'l f
.2.~
He requirements baseline consisting of the soft-L. Establish and follow a documented procedure to re -
ware requirements documentation plus approved cord the status of baselines and change requests j
changes j
2.
Create and distribute to affected groups and indi-
-l
- 3.
The product baseline consisting of software and viduals standard reports documenting the configura-is documentation resulting from the qualification test.
tion management actmties ing activity 6.5 Software Development Library
. 4.
He operational baseline consisting of software and Require the developer to establish and maintain a soft-documentation resulting from the installation and ware development library (SDL). An SDL is a controlled 1 acceptance activity that is placed into operation collection of software, documentation, and associated
'l tools and procedures used to facilitate the orderly devel-ij De developmental configuration is the developer's soft-opment and subsequent maintenance of software.He :
i ware and associated technical documentation that defines SDL contains the developmental configuration as part of the evolving software products during development. It its contents. An SDL provides storage of and controlled l
contains the software design and implementation prod-access to software and documentation in human-readable ucts (software design documentation, code, test cases, form, machine readable form, or both. The SDL may also -
and related information). Require the developer to apply
. contain management data pertinent to the software de-internal configuration control procedures to the develop-velopment project. The SDL becomes the repository for mental configuration as it evolves. See Sections 63 and the software baselines when the product baseline and the L 6.6.
operational baseline are established.
t 6.3. Change Control 6.6 Software, Access, and Media; j
Control' a
Once a baseline has been established, changes to the baseline can be made only in accordance with formal Require the developer to establish and maintain the fa-change. control procedures. To manage changes to cilities and procedmes used to baselines:
)
1.
Maintain, store, secure, and document controlled ;
a 1.
Establish a board (i.e., a configuration control board versions of the software throughout the life cycle.
j 1
(CCB)) controlled by the sponsor project manager nis may be implemented with the SDL (See Scc-that has the authority for managing the software tion 6.5) baselines and approving or rejecting proposed j
changes to them 2.
Permit authorized'and prevent unauthorized access to the software and documentation '
j
- 2. - Establish and follow a documented procedure forin-i itiating, recording, reviewing, approving or reject-3.
Identify the media for each software product and the ing, and tracking change requests for baselines documentation required to store the media, incloo-ing the copy and restore process l
I 3.
Establish and follow a documented procedure for ensu'ing that all changes, especially those to the 4..
Protect software physical media.from unauthorized -
requirements and design, are appropriately re.
awess, on inadvertent damage or degradation q
viewed for " ripple" effects and incorporated into all throughout the life cycle.
related activities
- }
6.7 Configuration Audits -
j 4.
Estab%t and follow a documented procedure to create and control the release of software baseline Require the developer to plan and execute documenta-j ti n audits, software configuration audits, and m-process -
]
products
! audits. Require the developer to establish and follow a.
documented procedure to prepare for, conduct, report
- 6.4 Status of Baselines and Changes-results from, and track action from configuraen audits. :
Track accurately the current status of baselines and A documentation audit is a line-by-line comparison of re- '
.}
changes throughout development and maintenance.To vised documentation against the previous version of the
- j track status accurately:'
documentation to ensure that only approved changes j
24 j
- j g.
.y,,.
p
,n.
have been incorporated. A documentation audit is typi-eedures are being followed and how effective they are in cally performed after a formal life-cycle review (e.g, after managing the software baselines.
1 the Critical Design Review to ensure only CCB-approved changes to the software requirements documentation and 6.8 Techniques and Tools software design documentation have been incorporated).
Use a data base management system as a tool in tracking As indicated in Section 3.2.2.6, the Software Configura-yep rting on proposed and actual changes to an
+
)
tion Audit is executed twice, fir;1 at the comple' ion of selmes. 0ften the data base of proposed and actual qualification testing and second at the completion of ac-anges imtegmteM tWaMasy used to track and ceptance testing.
report on nonconformances and associated corrective ac-tion (see Section 8).
Periodic in-process audits are performed to assess how In addition, choose a software tool, often a part of the op-well the configuration management standards and pro-erating system utilities, to help manage the 3DL
.. j L
I 1
+
l 2.5 t
r 1
- a q
n 7 NONCONFORMANCE REPORTING AND CORRECTIVE ACTION
!o l
7.1 - Concepts and Definitions 2.
Impact assessment and corrective action (SectionL 7.2.2) i A nonconfounance, often called a problem, discrepancy,
'l fault, or error, is any failure of any document, code, data
. structure, or process to meet its requirements or stan-TrackinF and management repons (Section 7.2.3) i!
3.
dards. Corrective action is a general name for the process -
by which nonconformances are corrected, verified, and 7.2.1 Nonconformance Detection and -
controlled.
Reporting.
Require the developer to establish and maintain a Allow nonconformance reports to be filed against any.
nonconformance reporting and corrective action system product in any part of the software life cycle by anyone G
and associated procedures. The purpose of a noncon-associated with the project. Normally a nonconformance I
formance reporting and corrective action system is to re-reporting and corrective action system is used after a j
port, analyze, correct, and verify nonconformances and product is first approved or baselined by its developerand.
o mllect informatior from which reports on the overall released for wider use. For example, while a developeris -
J status of nonconformances can be made.
unit testing and integration testing the code, errors found E l
may be tracked only locally'and 'not in the noncon--
The need for a nonconformance reporting and corrective formance reporting and corrective action'systemiAfterL action system arises early in the software life cycle, as the mde is declared correct and released for qualification l
soon as the first documents and otherproducts are devel.
testmg by the implementation group," the-noncon-oped. A nonconformance reporting and corrective action formance reporting and corrective action system is used j
system shauld:
to inform the users of the code and 1he designer / program-~ -
l mer about nonconformances and to assure that the non-.
1.
Define a nonconformance report form conformances are corrected,~ verified.and not over-j looked.
2.
Identify the organization (s) and procedures for:
Examples of the information that a nonconformance re-
'I pon form might contain are:
j a.
Analyzing the nonconformance 1.
Date and time' of the detection of the noncon6 b.
Assigning priorities Communicating with the person who reported 2..
Nonconformance identification (report number) c.
the nonconformance 3.
Reporting individual and organization d.
Correcting the nonconformance
?
4.
e.
Verifying the correction and/or the corrective Reponingindividual's determination of the critical '
I ity of the nonconformance i
action l
t
. 3.
Track the status of the nonconformance and correc-5.
Statement of the nonconformance tive action 6.
Organization responsible for analysis of the 1
3
- 4.
j Produce management reports
" "*"I ""*"C f
7.
Result of the analysis of the nonconformance
.17.2 Activities it
. There are three basic activities associated with a noncon-8.
The project's determination of the criticality of the ~
ll non nfonnance formance and corrective action system:
g 1.
Nonconformance detection and reporting (Section 9.
Organization (s) responsible for designing, imple.'
i 7.2.1) menting, and verifying the corrective action or Tix" 1
27 l
a.
- ~ -
- ~.
.=
- 10. Identification of the unit (s) of code, data, or docu-tracking of nonconformances and providing management mentation in which corrective action must be taken reports. Entering the nonconformance report electroni-cally can increase productivity funher.
- 11. Summary of the test case results (or other verifica-Pr vide m. the nonconformance tracking and reporting tion activity) indicating that the corrective action was system management reports contammg such information successfully implemented as nonconformance and correction status, the number of nonconformance found per product, and the criticality of
- 12. Identification of1he date or version of the product (s) open problems. 'lhe data enable the impact of the or baseline m which the correction will be included nonconformance to be evaluated so that the use of re-sources may be prioritized.
- 13. Date on which the nonconformance is closed 7.3 Interrelationships 7.2.2 Impact Assessment and Corrective The nonconformance reporting and corrective action sys-Action tem is a basic and fundamental tool for project manage-ment and for assuring quality products. As such it impacts Evaluate all nonconformances for criticality and level of and interacts with many software management, verifica-importance. Consider the following factors:
tion and validation, and quality assessment and improve-1.
'Ihe impact of not correcting the nonconformance ment activities. For example, it interacts with:
1.
Configuration management activities that deal with 2.
'Ihe resources required for correcting the noncon.
product changes and versions that result from cor-formance recting nonconformances 3.
The impact on other baselined items if the noncon-2.
Requirements management activities because some formance is corrected nonconformance reports will ccmtain requirements changes disguised as nonconformances. 'lhese re-Ensure that a written procedure exists to control the cor-ports should result in the opening of a change re-rective action process. Include in this procedure a follow-quest up acthity to ensure the nonconformance has been docu-mented and corrected in the appropriate version of mprovement activities that study trends in software and to assure that adequate testmg, meludm, g re-nonconformances in specific products or processes gression testing, has been done.
7.2.3 Tracking and Management Reports 7.4 Techniques and Tools After the nonconformance report is prepared by the indi-Consider using an automatG tracking system for vidual who found the nonconformance, enter the report nonconformance reports and an automatic capability to data into a controlling system. Data base management identify and record changes that occur as a result of the systems are often used to increase productivity in the resolution of the nonconformances.
28
8 QUALITY ASSESSMENT AND IMPROVEMENT 8.1 Concepts and Definitions plans". His is a collective term used to describe the de-veloper's plans, methodologies, standards, and proce-Require the developer to institute a quality assessment dures for software management, software engineering, and improvement program.The objective of this program software verification and validation, software documenta-lI is assess and improve the quality of:
tion, software product evaluation, and software configu-ration management.
1.
Deliverable software and documentation l
Table 8-1 identifies the products and processes to be as-l 2.
De processes used to produce deliverable software sessed and the objectives of the assessments.
t t:
l 3.
Non-deliverable software (software not required to 8.5 Quality Records Collection, Ma. tenance, and Retention be delivered by the contract) la Require the developer to prepare and maintain records 8.2 Responsibility For Quality Assess.
of the quality assessment and improvement activities.
ment and Improvement Require the developer to prepare a software quality as-Allow the developer the freedom in assigning responsibil.
sessment record for each assessment required by the con-ity for meeting the objectives of the quality' assessment tract. Require these records to contain as a minimum:
and improvement program. However, for level I soft-j ware development and maintenance efforts, require that 1.
Assessment date j
the persons responsible for the assessments of a product or activity be independent of the persons who developed 2.
Assessment participants the product, performed the activny, or are responsible for the product oractivity. This restriction does not preclude members of the development team from panicipating in 3.
Assessment criteria these assessments.
4.
Assessment results including detected prob! cms, 8.3 Documentat. ion For Quality with reference to the appropriate nonconformance Assessment and Improvement reports, as applicable ne developer's approach to quality assessment and im.
provement will be documented in the Software Project 5.
Recommended corrective action Plan.nis approach will be implemented throughout the development or maintenance effort. Developers should Include in these required records the nonconformance work toward definm' g in detail their methodology for reports that are the basis of the nonconformance report-quality assessment and improvement in written procc-ing and corrective action system outlined in Section 7. Re-
- dures, quire the developer to make quality records available for sponsor review and to maintain them for the life of the 8.4 Quality Assessments contract.
Require the developer to assess:
8.6 Quality Improvement 1.
Software Encourage the developer to integrate quality assessment '
aethities with quality improvement activities (which may
' 2.
Software documentation be part of the developer's approach to total quality man-agement).
3.
Processes used in software development -
SJ Techniques and Tools A prerequisit e to any assessment is ayardstick or standard Checklists for quality audits and inspections and auto-against which a product or process can be measured oras-mated code standards analyzers are examples of tools sessed. A key yardstick is the developer's " software used in quality assessment activities.
29
Table 8.1 Assessments of Products and Processes Used in Software Development Product or Process To Be Assessed Assurance Objectives Software Compliance with the contract and adherence to the software plans All software plans required by the contract have been
.j Software plans documented q
" Die software plans comply with the contract
=
Each software plan is consistent with other software plans e
Each document adheres to the required format.
Deliverable softw2re documentation Each document complies to the contract Software management Compliance with the contract ar<d adherence to the software plans Software engineering Compliance with the contract and adherence to the software plans The qualification plans and procedures include prosisions Software qualification e
for all requirements Software qualification is conducted as required by the e
contract and as specified in the software plans The version number of each item being qualified and each e
item used in qualification is documented The results of required qualifications are accurately e
recorded and analyzed to determine whether the software meets its specified requirements All required facilities for qualification are assilable e
Software configuration Compliance with the contract and adherence to the software plans management Software corrective actions Compliance with the contract and adherence to the software plans and:
All nonconformances detected in processes and in products that are under developer or sponsor control are promptly reported and entered into the software corrective action process Each nonconformance is classified, as required by the contract, and analysis is performed to identify trends in the.
nonconformances reported Action is initiated on the nonconformances and adverse -
trends, resolution is achieved, status is tracked and reported, and records are maintained for the life of the contract Corrective actions are evaluated to: 1) verify that problems.
e have been resolved, 2) verify that adverse trends have been reversed,3) verify that changes have been correctly implemented in the appropriate processes and products, and
- 4) determine whether additional problems have been introduced 30 i
Ii.
a
t Table 8.1 (continued)
Product or Process To Ile Assessed Assurance Objectives Documentation and media Compliance with the contract and adherence to the e
i.
distribution software plans Evaluation of the controls exercised on the internal e
distribution of deliverable media and documentation Storage, handling, and Compliance with the contract and adherence to the e
delivery software plans Evaluation of the storage, handling, packaging, shipping, and e
external distribution of deliverable software and associated documentation Software development The library and its operation comply with the e
library contract and adhere to the software plans The most recent authorized version of the materials e
under configuration control are clearly identified and are the ones routinely available from the library Previous versions of materials under configuration control are clearly identified and controlled to provide an audit trail that permits reconstniction of l
all changes made to each baseline l
Non-developmental Evaluate each item of non-developmental software to software be incorporated into deliverable software to assure that:
Objective evidence exists, prior to its incorporation, that it performs required functions It was placed under configuration control prior to its incorporation The data rights provisions are consistent with the contract e
Non-deliverable software Evaluate cach item of non-deliverable software used in the qualification or acceptance of deliverable software to assure that:
Objective evidence exists, prior to its intended use, that it performs required functions It was placed under configuration control prior to its use e
Deliverable elements of the Evaluate cach deliverable element of the software software engineering and test engineering and test environments to assure that:
environments It complies with the contract and adheres to the software plans Objective evidence exists, prior to its use, that it e
performs required functions 11 was placed under configuration control prior to its use
'Ihe data rights provisions are consistent with the contract 31
Table 8.1 (continued)
Product or Process To Be Assessed Assurance Objectives l
Subcontractor management Evaluate all subcontractor activity to assure that:
All subcontractor-developed software and related.
e documentation deliverable to the sponsor satisfies the requirements of the prime contract A set of baselined requirements is established and -
e maintained for the software to be developed by the subcontractor Applicable software quality assessment and improvement e
- requirements are included or referenced in the subcontract or purchase documents Access is available for developer reviews at subcontractor and vendor facilities The sponsor has the right to review all software products and e
activities required by the subcontract, at subcontractor facilities, to dctermine compliance with the contract. Sponsor review does not constitute acceptance, nor does it in any way replace assessment by the developer or otherwise relieve the developer of his responsibility to furnish acceptable software and associated documentation Compliance with the contract and adherence to the software Acceptance inspection and e
preparation for delivery plans Evaluation of the controls exercised on the Mternal e
distribution of deliverable media and documentation i
Prior to each formal review and audit, assure that 1) all Participation in formal reviews e
and audits required products will be available and ready in sufficient -
time for sponsor review before the review meeting and 2) all-required preparations have been made At each formal review and audit, require the developer to e
present an assessment of the status and quality of each of the j
development products reviewed.
Following each formal review and audit, require the developer to assure that all software-related action items assigned to the developer have been performed
{
9
-i
'[
4 32 4
t e
9 SOFTWARE DEVELOPED BEFORE ISSUANCE OF THIS DOCUMENT.
All the detailed guidelines in this document obviously 1.
Establish and-maintain a software development cannot be applied in a cost-effective manner to software libary developed before this document was issued.
2.
Institute a nonconformance and corrective action I
However, Ixvel 1 software developed before this docu-ment was issued should receive as-is verification and vali-dation or certification based on its length of service anci 3.
Develop a set of acceptance test cases error profile. In addition, the software should be placed under configuration control in accordance with the guid-4.
Institute a clearly defined test program ance in ASME Standard NQA 2, Part 2.7, Section 10.2.
5.
Institute a well-defined configuration management Level 1 and level 2 software developed before this docu-
'"**#'E #""Y" " **
ment was issued can benefit from selected application of the software quality assurance principles presented in 6.
Begin code inspections this document.The followinglist provides,in relative pri-
- ority order, suggested actions that can be taken on a step-Ultimately, however, the extent to which new techniques by-step basis - to enhance existing software cost-can or should be introduced into ongoing maintenance ef-effectively:
forts is a matter of managerial and engineering judgment.
33
-1 l
1 1
i 9 SOImVARE DEVELOPED BEFORE ISSUANCE OF THIS DOCUMENT All the detailed guidelines in this document obviously 1.
Establish and maintain a software development cannot be applied in a cost-effective manner to software library
- developed before this document was issued.
I 2.
Institute a nonconformance and corrective action However, Level I software developed before this docu-f ment was issued should receive as-is verification and vali-dation er certification based on its length of service and 3.
Develop a set of acceptance test cases error profile. In addition, the software should be placed under configuration controlin accordance with the guid-4.
Institute a clearly defined test program ance in ASME Standard NOA 2, Part 2.7, Section 10.2.
5.
Institute a well-defined configuration management
"#"'#'E# ""#
- 12 vel 1 and level 2 software developed before this docu-ment was issued can benefit from selected application of '
6.
Begin code inspections the software quality assurance principles presented in this document. The following list provides, in relative pri-ority order, suggested actions that can be taken on a step-Ultimately, however, the extent to which new techniques by-step basis to enhance existing software cost-can or should be introduced into ongoing maintenance ef-effectively:
forts is a matter of managerial and engineering judgment.
33'
o F
APPENDIX A Sample Software Project Plan Appendix A consists of a sample Software Project Plan.
5.
Task 5-Installation and Acceptance Although the project and the sample plan are fictitious, 2
the sample plan is provided to indicate an acceptable 6.
Task 6-Verification and Validation level of detail.
The statement of work for the project is divided into ten 7.
Task 7-Project Management tasks as follows:
8.
Task 8-Configuration Management 1.
Task 1-Requirements Definition 9.
Task 9-Nonconformance Reporting and Correc-2.
Task 2-Design tive Action 3.
Task 3-Implementation
- 10. Task 10-Quality Assessment and Improvement 4.
Task 4-Qualification Testing The sample software project plan follows.
k P
t 5
35
i SAMPLE SOFTWARE PROJECT PLAN SECTION 1 -INTRODUCTION
' 1.1 Project Objectives Section 2, Management Approach, summarizes ABC's -
planning approach (Section 2.1); tracking and oversight h
The ABC Corporation has been developing, enhancing, approach (Section 2.2); project organization, including I
and maintaining the.XYZ analysis tool for NRC in-house tasks and responsibilities (Section 23); the top-level L
use for the past 8 years.The current contract requires the schedule (Section 2.4); the resources required (Section ABC Corporation to update Version 3.4 of XYZ by:
2.5); and configuration management approach (Sec--
J
- tion 2.6).
. 1.
Adding two new major capabilities, C1 and C2 Section 3, Technical Approachisummarizes ABC's ap-proach to implementing the life-cycle tasks of the state. -
2.
Analyzing and implementing corrective action for30 -
ment of work (Section 3.1); verification and validation ap-known nonconformances in Version 3.4 of XYZ proach (Section 3.2); nonconformance reporting and corrective action approach (Section 33); quality assess-
. 3.
Analyzing and implementing a corrective action for ment and improvement approach (Section 3.4). The con-as many as 20 yet-to-be-determined nonconfor.
tractually required dehverables are listed in Section 3.5.
mances in Version 3.4 of XYZ.
Section 4 lists all the standards, procedures, conventions, He contract period of performance is 2 years from the and metrics that will be applied on the contract.
contract start date.
1.3 Plan Maintenance 1.2 Plan Scope and Organization -
This document is intended to be an up-to-date statement '
his software project plan defines ABC's management of ABC's plan for managing the contract. Therefore, and technical approach to meet the requirements of the changes to the document will beissued as change pages as.
contract. It also identifies the standards, procedures, con- --
required. In accordance with the contract, change pages ventions, and metrics that will be applied throughout the will be submitted to the NRC sponsor monthly as an at-project.
tachment to the monthly progress report.
j 37
u s
r SECTION 2-MANAGEMENT APPROACH
' ABC's management approach responds to Task 7, The above-discussed plannmg actmties will be repeated :
Project Management, and Task 8, Configuration Man-if re-planning becomes necessary during the life of the :
agement, of the statement of work.
project.-
~
{
2.1 Planning Approach 2.2' Tracking and' Oversight Approach s ABC's planning approach, which was'used to generate Project tracking and oversight involves:
l p
this plan, consists of the following steps performed in an 1.
Monitoring, assessin8,- and reporting. technical.
tterative process.
1 progress 1.
Defining the work L_
2.
Determining and reporting schedule and cost
- progress 2.
Estimating the schedule and staffing l
3.
Developing and implementing corrective action 3.
Planning for technical performance measure nent plans, as required
. 4.
Plann.mg nsk management ABC will track actual technical results and perfonnance.
d against this baseline project plan.' Implementation of '
Hl planneti verification and validation, configuration man-
~
. 5.
Performing detailed planning agement, and quality assessment and improvement actisi- -
l ties will be part of the day-to-day tracking and oversight.
'. lhe work elements were derived from an analysis of the statement of work. Then, based on the analysis results Each ABC manager will be a hands-on manager, i.e., he/
and ABC's -past experience, the work elements were she will monitor technical,' schedule, and cost ' status structured into a work breakdown structure. In develop-ing the master schedule, ABC took into consideration the progress on an ongoing basis through: daily person-to-
- technical complexities, the NRC's required milestones.
person and telephone contact with their assigned people.
task interdependencies, the estimated number of lines of weekly staff meetings, the monthly progress meeting sith their sponsor counterparts, and internal ABC informa-cource code, and t he expected staff skill mix.
tion systems.
,M Analysis of the C1 and C2 capabilities resulted in the The ABC Project Manager will keep in close conta'ct with l-identification of the following three key technical per-.
the NRC Project' Manager by telephone. The ABC..
formance measures- (1) the estimated source lines of Project Manager will plan for and lead the monthly pro- -
code: (2) the time needed to perform the Monte Carlo gress meeting where
- analysis for the C1 capability; and (3) the rate of conver-
. gence of the new eigemalue algorithm in the C2 capabil.
1.
Technical, schedule, and cost status :
ity.
. Two major risks have been identified: (1) meeting the re-quirement to incorporate an undetermined number of 3.
Work planned for the next month' new nonconformances into the new release, Version 4.0, within schedule and cost constraints and (2) the ability to meet the required rate of convergence of the new eigen.
4.
' Risks, problems, and concerns and recommended value algorithm in the C2 capability.
solutions will be discussed.'
l.
Cost accounts were planned in detail to create a sound ba ~
. Within ABC,weeklyprogressreportsfrom ABCmanagi
o 1
}
. g s y
a 3 5 1
g 3
3
}
I I
3
}
1 i
h3 42.
nummmmmmm s,
m,e.w m_
.s.4.s m
st, m _,s. n
_,e. u,
,s. 4 m is.
- m _ is. 2m
.j 5
1 is. so; 8_
o t6 uor
_ <e.,
j
- m, i
[r-ts. m i
e m.s tshw j
cs. n ts.uv te. x,
_m.
mca.,
m y,.J W, " ""
i i
i i
i y
a e
c w
n o
puuos;ad =J PJ 43 i
L________
._________u_____________________________.________________
-d
_ _ - _ = _ - - - - - - - - - - _ _ - - - _ - - - _ - - - - _ _ - - - _ - - _ _ _ _ _ _
l 1.
Uniquely identifies ali project entities (e.g., docu-y 2.6 Configuration Management ments, software elements, test cases) 2.6.1 Overview 2.
Identifies changes by revision or version ABC will support the NRC in the four major functions of configuration management:
3.
Uniquely identifies each configuration / version of
- revised software for use 1.
De identification and establishment of baselines
(
We will support NRC in the establishment of the follow-2.
Controlling both changes to baselines and the re-ing baselines that will be controlled by the NRC XYZ lease of baselines configuration control board (CCB):
l 1.
The project management baseline consisting of the 3.
Recording and reporting the status of baselines, S ftware Project Plan, documented standards and change requests, and implemented changes procedures, and up-to-date budgets and schedules 4.
Verifying, through auditing, the correctness and 2.
The requirements baseline consisting of the soft-completeness of baselines prior their release ware requirement:; documentation plus approved l
ABC will ensure that:
3.
The product baseline consisting of software and 1.
He content and status of the software and docu-documentation resulting from the qualification test-i mentation baselines are known at all times ing major activity l
2.
Configuration management is implemented for 4.
The operational baseline consisting of software and -
j externally-deliverable products and for appropriate documentation, resulting from the installation and products used inside the ABC Project aaeptance activity, that is placed into operation 3.
Here will be a repository for storing key software The XYZ CCB is controlled by the NRC.
LI engineering elements and associated configuration management records ABC will establish and control the developmental con-figuration, which will contain the software design and im-p emema np (s twam design hmema@n, 2
4.
The software baselines and configuration manage-code, test cases, and related information) of the evolving ment activities are audited on a regular basis Version 4.0 of XYZ. We will apply proven ABC internal
+
configuration mntrol procedures and automated tools, 5.
The Oualification Testing and Configuration Man-used in previous XYZ work, to the developmental con-agement Group will be responsible for coordinating figuration as it evolves.
configuration management for the project 2.6.3 Change Control 6.
Adequate resources and budget for performing con-ABC will support the NRC in the implementation of con-figuration management activities have been allo-figuration control procedures established on previous XYZ upgrade projects. Specifically we will support the
= NRC in the management of changes to baselines as fol-7.
Staff members responsible for coordination of mn-lows:
figuration management activities have been trained.
in the objectives, procedures, and methods of per-1.
Implement the directives of the NRC XYZ CCB forming their duties (XYZ Procedure CM-03) 2.
Follow XYZ Procedure CM-02 for initiating, re-2.6.2 Baselines cording, reviewing, approving or rejecting, and ABC wil.1 establish controlled and stable baselines for tracking requests for changes to baselines planning, managing, and building Version 4.0 of XYZ.
3.
Follow XYZ Procedure CM-04 for ensuring that all.
ABC will establish a naming or labeling system that:
changes, especially those to the requirements and 44-1
'i
i the top-level design, are appropriately reviewed for 1.
Identify the media for each software product and the
" ripple" effects and incorporated into all related ac-documentation required to store the media tivities 2.
Protect software physical media from unauthorized 4.
Update, gain approval, and follow XYZ Procedure access and inadvertent damage or degradation CM-05, to create and control the rclease of software throughout the life cycle.
j baseline products 2.6.7 Configuration Audits 2.6.4 Status of Baselines and Changes ABC will follow existing XYZ configuration manage-ABC will track the current status of baselines and ment procedures to prepare for, conduct, repon results changes throughout development and maintenance by:
I*m, and track action items based on results of documen-tation audits, software configuration audits, and m-1.
Following XYZ Procedure CM-06 to record and re.
Pmcess audits.
port the status of baselines and change requests ABC will conduct documentation audits on updates to all.
documents in accordance with XYZ Procedure CM-07.
2.
Creating and distributing to affected groups and in-dividuals standard reports, in accordance with XYZ.
Software configuration audits will be performed at the Procedure CM-06, documenting the configuration conclusion of both qualification testing and acceptance management activities testing in accordance with XYZ Procedure CM-08.
ABC will conduct in-process audits at least once a year to 2.6.5 Sofhvare Development Library assess how well the configuration management standards -
and procedures are being followed and how effective they
'i ABC will continue to maintain the XYZ software devel-are in managing the software baselines. In-process audits.
h opment library (SDL)in accordance with XYZ Procedure will be conducted in accordance with XYZ Procedure '
6 CM-01.This procedure will be updated if required.
CM-09.
2.6.6 Software, Media, and Access Control 2.6.8 Techniques and Tools-ABC will maintain the facilities and the associated XYZ ABC will continue to use the dBASE III XYZ Nonccm-Procedure CM-01 used to maintain, store, secure, and formance and Corrective Action System on an industry-document controlled versions of the software throughout standard Personal Computer workstation to track non-the life cycle.
conformances, action items, and their resolutions. In-1 addition, the commercially available DDD software tool ABC will establish and maintain the facilities and proce-will be used in the management of the sof tware develop-dures used to ment library.
s 3
'3 I
e 45 t
F SECTION 3 -- TECHNICAL APPROACH Section 3 responds to the following eight technical tasks When NRC approves the requirements after the Soft-of the statement of work-ware Requirements Review, the approved requirements will form the basis for the software plans, products, and i
Task 1-Requirements Definition (Section 3.1.1) activities.
Task 2-Design (Section 3.1.2)
ABC will ensure that the documented requirements de--
fine the response of the software to anticipated classes of Task 3-Implementation (Section 3.1.3) input data (including crroneous data) and provide the in-formation and detail necessary to design the software Task 4-Qualification Testing (Section 3.1.4)
(e.g., mathematical models, equations, data require.
j i
ments).
j Task 5-Installation and Acceptance (Section 3.1.5)
Dlj ABC will perform verification and validation planning ac.
Task 6-Verification and Validation (Section 3.2) tivities in parallel with requirements analysis and defini-l tion activities.
1 Task 9-Nonconformance Reporting And Corrective Ac-tion (Section 3.3)
Requirements changes will be controlled throughout the l
development and maintenance efforts in accordance with Task 10-Quality Assessment and Improvement Pro-the proven configuration management procedures cited ~
gmm (Section 3.4) in Section 2.6 of this plan.
Section 3.5 identifies the deliverables, their duc dates, 3.1.2 Task 2--Design and the standards that they will follow, and Section 3.6 l.
identifies the standards and procedures that will be used..
ABC will use structured design techniques as docu-I mented in ABC Standard SD- 01 to analyze, both indi-L 3.1 Implementing the Life-Cycle Tasks vidually and collectively:
l of the Statement of Work 1.
The baselined requirements for the C1 and C2 capa-De statement of work calls for the Contractor to perform bilities the following life-cycle major activities:
2.
The requirements of Version 3.4 of XYZ 1.
Requirements Defm' ition 3.
De requirements associated with nonconformances 2.
Design to Version 3.4 of XYZ 3.
Implementation This analysis will be conducted to determine the optimal software design for Version 4.0 of XYZ such that:
4.-
Qualification Testing 1.
Changes to the existing XYZ software architecture are minimized 5.
Installation and Acceptance 2,.
Existing XYZ capability remains operable 3.1.1 Task 1-Requirements Definition 3.
Changes to the design meet the requirements asso-ABC will analyze the CI and C2 Requirements Docu.
ciated with.
ment provided by NRC. We will use ABC's structured analysts approach (use of data flow diagrams, data, dic-a.
He C1 and C2 capabilities tionaries, and mini. specifications) to perform the requirements analyses as documented in ABC Standards b.
Corrective actions to all nonconformances SA-411, SA-02, and SA-03. ABC will provide suggested changes to the CI and C2 Requirements Document and As the design evolves, events (e.g., identification of will ensure that the requirements are correct, complete, additional nonccmformances) will necessitate the modifi-verifiable, ccmsistent with XYZ Version 3.4 require-cation of the requirements and design documentation, ments, and technically feasible.
ABC will manage changes to formally baselined 47
b d
(requirements) documentation and internally baselined 3.
New or modified test cases that will qualify the cor-l (design) documentation in accordance with the pmced-rective actions to the major nonconformances that t
urcs cited in Section 2.6 of this plan.
will be incorporated into Version 4.0
. 3.1.3 Task 3-Implementation Qualification tests will be witnessed by a member of the Quality Evaluation and Improvement Group. ABC will ABC will design and code all new software units and review and analyze the qualification test results to assure make changes to existing software units in accordance NRC that the implemented softwate rnects requirements with XYZ Standard IM-01, which defines XYZ Project and that the software produces correct results for all ap-coding standards. All new and modified units will un-proved test cases.
dergo three inspections:
1.
An inspection of the unit design and unit test plan in 3.1.5 Task 5-Installation And Acceptance accordance with ABC Procedure INS-01 After qualification testing has been successfully con-cluded, ABC will install Version 4.0 of XYZ at NRC's 2.
An inspection of the unit code m. accordance with Rockville. Maryland, headquarters. nen, with an NRC I
ABC Procedure INS-02 sponsor representative present, ABC will conduct the ac-ceptance tests, which will be a subset of the qualification -
3.
An inspection of the unit test results in accordance tests.
with ABC Procedure INS-03 3.2 Verification and Validation ABC will conduct integration tests to ensure that all in-Approach terfaces among the new and changed units perform cor-rectly, ABC will inspect all integration test plans and His section:
procedures in accordance with ABC Procedure INS-04 and integration test reports in accordance with ABC Pro-1.
Summarizes the verification and validation activities cedure INS-05.
that' ABC plans to perform (Section 3.2.1).
As the software is implemented, events (e.g., additional 2,
Discusses the formal life cycle reviews and audits msight into data flow patterns) may necessitate the modi-that ABC will conduct (Section 3.2.2) fication of the design, requirements, and/or verification and validation documentation. ABC will manage changes to documentation in accordance with well-defined change 3.
Discusses the formal peer inspections that ABC will control procedures.
use (Section 3.2.3) 3.1.4 Task 4-Qualification Testing 4.
Identifies thelevels of testing that ABC willconduct (Section 3.2.4)..
ABC will perform formal qualification testing at ABC's Windy Canyon site using a set of test cases based on:
3.2.1 Summary of Verification and 1.
The existing baseline qualification test cases for Validation Activities Version 3.4 of XYZ Table 3-1 summarizes the verification and validation ac-
.2.-
New test cases that will qualify the new capabilities, thities that ABC plans to perform on the XYZ System C1 and C2 Upgrade Project.
s I
e r
48
Table 3.1 Verification and Validation Activities by Major Life Cycle Activity Major Life Cycle Activity Verification and Validation Actitifies Requirements Definition Inspect requirements e
(
Conduct the Software Requirements Review e
I Design Inspect design Develop qualification test plan Develop acceptance test plan Conduct the Design Review Implementation Develop unit test plans Inspect unit designs, unit code, and unit test plans e
Perform unit testing e
Inspect unit test results Develop integration test plans Inspect integration test plans and procedures Perform integration testing e
Inspect integration test results Develop qualification test procedures e
Qualification Testing Perform qualification testing Witness qualtfication testing (by independent group)
)
Write qualification test report e
Develop acceptance test procedures Installation and Acceptance Perform acceptance testing e
Witness acceptance testing (ly NRC sponsor)
Write acceptance test report e
Participate in the Post Mortem Review e
3.2.2 Formal Life Cycle Reviews and Audits I.
The documents to be reviewcd A formal review. with NRC and ABC management and 2.
The agenda for the review techmcal pcrsonnel participating, will be held at or near the end of each major activity of the life cycle. The objcc-tive of the formal reviews is to evaluate the deliverab!c 3.
The hardcopy presentation materials used at the re-products, the progress, and to a lesser degree, the proc-view esses of the most recent life-cycle phase. Table 3-2 sum-marizes the formal major life cycle reviews and audits that will performed on the contract.
4.
The mmutes that document the activities and results of the review AllC will deliver five classes of dchverables associated with each formal review.
5.
The updated documents that were reviewed 49
Table 3.2 Formal Life Cycle Reviews and Audits Major Life Cycle Activity Formal Reviews and Audits Software Requirements Review Requirements Definition Design Review I
Design Qualification Test Readiness Review implementation e
Software Configuration Audit Qualification Testing Software Configuration Audit Installation and Acceptance 1'
Post Mortem Review For each formal review, ABC will:
3.
Assure NRC that ABC understands and agrees on the intent, completeness, verifiability (through test-1.
Deliver documents for NRC review 2 weeks prior to ing or other means), consistency, and technical feasi-l the start date of the formal review bility of the requirements 4.
Review the Software Project Plan 2.
Identify in each review agenda the ABC review par-ticipants and their specific responsibilities during the review 3.2.2.2 Design Review Because the architecture for XYZ is well understood by 3.
Assign a person to capture key discussion items and both NRC and ABC, there will be only one design review, actions items, especially those related to specific as-The objectives of the Design Review are to ensure that:
signments for updating the documentation that is the object of the review.
1.
The proposed design is complete (meets all the requirements and design completion critcria),verifi-able (through testing or other means), consistent, 4.
Document in the review minutes all proposed revi-and 1cchnically feasible sions to the reviewed documents and all actual changes to the reviewed documents, and place the 2.
All new and modified software units have been iden-updated documents under configuration control af-tified and all interfaces between and among the ter approval by the NRC units have defined The paragraphs below discuss each formal life cycle rc~
3.
All elements of the database have been defined view and audit.
down to the data item level 3.2.2.1. Software Requirements Review 4.
Qualification and acceptance test plans are re-viewed and the test environment is ready to meet ABC will conduct the Software Requirements Review at pr ject needs the end of requirements definition activity. The objec-tives of this review are to:
3.2.2.3 Qualification Test Readiness Review 1.
Review the requirements associated with the new ABC will conduct the Qualification Test Readiness capabilities C1 and C2 and the known nonconfor.
Review when integration testing has been successfully -
mances completed and thequalification test procedures are ready for NRC review. The objective of this review is to assure 2.
. Review ABC's suggested changes to the draft that the as-built software; the software documentation; requirements specification supplied by NRC and qualification test environment is ready for formal 50
.I l
F F.
qualification testing. In particular, the thoroughness of 3.
Qualification testing informal (unit and integration) testing will be resiewed.
3.2.2.4 Software Configuration Audits ABC will conduct two Software Configuration Audits:
Eacn new and modified software unit will be separately the first at the completion of qualification testing and sec.
unit-tested. In unit testing, all paths through the code will
. ond at the completion of acceptance testing. The objec-be tested. Software components that contain one or more 7
tive of this audit is to ensure that the as-built software:
new or modified software units will be integration-tested. Timing of critical elements of code will be tested
- 1.
Meets its requirements as baselined in the software at the unit or integration level, as appropriate.
requirements documentation Qualification testing and acceptance testing were dis L 2.
Confonns to its technical documentation cussed in Sections 3.1.4 and 3.1.5, respectively.
3.2.5 Software Test Environment 3.
Does not contain any unauthorized changes The XYZ software test environment that was baselined 3.2.2.5 Post Mortem Review in July 1992 will be used on the project.-
3j ABC will suppon the NRC in the Post Mortem Review 3.3 Nonconformance Reporting and -
after the software is accepted. This review will capture Corrective Action Approach the lessons learned from the ABC XYZ System Upgrade Project for use by future XYZ and other similar projects.
ABC will use the in-place XYZ Project nonconformance; reporting and corrective action system, baselined by the 3 2.3 Formal Peer Inspections XYZ CCB on April 12,1992, and documented in XYZ Procedure NRCA-01.
Because XYZ is Level 1 software, ABC will:
3.4 Quality Assessment and Improve-1.
Subject each intermediate product and final product of development and maintenance (i.e., all docu-ment Approach mentation, all code) to an internal peer inspection ABC 'will use the quality assessment and improvement-approach documented in the' ABC repon Continuous 2.
Make available to the NRC the written procedure Improvement at the ABC Corporation, March 1991.
and the product standards that govern peer inspec-ties 3.5 Deliverables 13.
Make available, if requested by the NRC. records Contract deliverables are listed in Table 3-2.'All docu- -
' that document the results of all peer inspections mentation will be inspected in accordance with ABC Pro-cedure INS-12 3.2.4 Testing 3.6 Standards and' Procedures -
ABC will use four levels of testing-Table 3-3 summarizes the standards and procedures that will be used on the contract. Documentation standards 1.
Unit testing for th e principal software documents are not listed; the de facto documentation format standards of the existing.
2.
Integration testing software documentation will be used.
51 V
v L
Table 3.3 Contract Deliverables Due Date (Weeks aner Deliverable Contract Award)
ID ',.
1.1 Draft Suggested Changes to the XYZ Requirements 12 weeks -
1.2 Agenda for the Software Requirements Review 12 weeks 13 Hardcopy Presentation Materials for the Software Requirements Review 12 weeks 1.4 Minutes for the Software Requirements Review 15 weeks 1.5 Final Suggested Changes to the XYZ Requirements Specification 16 weeks -
2.1 Draft Suggested Changes to the XYZ Design Document 46 weeks 2.2 Draft Qualification Test Plan 46 weeks -
2.3 Draft Acceptance Test Plan
' 46 weeks.
2.4 Agenda for the Design Review 46 xecks-2.5 Hardcopy Presentation Materials for the Design Review
' 46 weeks 2.6 Minutes for the Design Review 49 weeks -
l 2.7 Final Suggested Changes to the XYZ Design Document 50 weeks 2.8 Final Qualification Test Plan 50 weeks 2.9 Final Acceptance Test Plan 50 weeks e
3.1 Draft Qualification Test Procedures 87 weeks 3.2 Draft Acceptance Test Procedures 87 weeks 3.2 Agenda for the Qualification Test Readiness Review 87 weeks 33 Hardcopy Presentation Materials for the Qualification Test 87 weeks Readiness Review 3.4 Minutes for the Qualification Test Readiness Review 89 weeks 3.5 Final Qualification Test Procedures 90 weeks '
3.6 Final Acceptance Test Procedures 90 weeks 4.1 Qualification Test Report 98 weeks 4.2 Software Configuration Audit Report 1 99 weeks 5.1 Acceptance Test Report 101 weeks 5.2 Software Configuration Audit Report 2 102 weeks 7.1 Monthly Progress Reports Monthly 9.1 Nonconformance Reporting And Corrective Action System Reports As generated 52 i
m Y,
Table 3.4 Standards and Procedures identification Title lm XYZ Procedure CM-01 XYZ Software Development Library Procedures XYZ Procedure CM-02 XYZ Change Request Procedure XYZ Procedure CM-03 XYZ Configuration Control Board Procedures XYZ Procedure CM-04 XYZ Change Request Ripple Effects Procedures XYZ Procedure CM-05 XYZ Baseline Release Procedure XYZ Procedure CM-06 XYZ Status Reporting Procedures XYZ Procedure CM-07 XYZ Documentation Auditing Procedure XYZ Procedure CM-08 XYZ Software Configuration Audit Procedure XY7 Procedure CM-09 XYZ In-Process Configuration Management Auditing Procedure XYZ Standard IM-01 XYZ Coding Standard XYZ Procedure NRCA-01 XYZ Nonconformance Reponing and Corrective Action Procedures ABC Standard SA-01 ABC Data Flow Diagram Standard ABC Standard SA-02 ABC Data Dictionary Standard ABC Standard SA-03 ABC Minispecification Standard ABC Standard SD-01 ABC Structured Design Standard ABC Procedure INS-01 ABC Unit Design and Unit Test Plan Inspection Procedure ABC Procedure INS-02 ABC Unit Code Inspection Procedure ABC Procedure INS-03 ABC Unit Test Results Inspection Procedure ABC Procedure INS-04 ABC Integration Test Plan and Test Procedure Inspection Procedure ABC Procedure INS-05 ABC Integration Test Results Inspection Procedure -
ABC Procedure INS-12 ABC Documentation Inspection Procedure 53
y APPENDIX B Glossary adaprire maintenance. Maintenance performed to make a derclopmentalcorfiguration. The developer's software and
[
software product usable in a changed environment.
associated technical documentation that defines the
/
cvohing software products during development. The -
baseline. A product (software or documentation or both) developmental configuration is under the develop-
.i that has been formally reviewed and agreed upon ly er's internal configuration control and contains the the developer and sponsor, that thereafter serves as software design and implementation products (soft-the basis for further development, and that can be ware design documentation, code, test cases, and re-changed only through formal change control proce-lated information).
dures.
requirements baseline. De baselined documentation crror. A discrepancy between a computed, observed, or that specifies the requirements that a software measured value or condition and the true, specified, or theoretically correct value or condition.
[
product must meet.
product bascline. The software and documentation formaltesting. The process of conducting testing aethities that are baselined at the successful completion and reporting results m accordance with an approved of qualification testing.
test plan.
operationalbascline. nc software and documentation independent verification and ralidation UV&l). Verifica-l that are l)baselined at the successful completion tion and validation by an organization that is both-of installation and acceptance testing and 2) are technically and managerially separate from the or-L placed into an operational status as a production ganization responsible for developing the software.'
i product.
See verification. See validation.
change control ne process of evaluating, appro ing or disapproving, and coordinating changes to baselines.
informaltesting. De process of conducting testing aethi-Also called configuration control.
ties without an approved test plan.
I code. One or more computer programs or part of a com-L'rel I software. Technical application software used in a puter program.
safety decision by the NRC.
t configuration contivl. See change control.
Level 2 software. Technical'or non-technical application software not used in a safety decision by the NRC.
configuration management. He process of 1) identifying and defining the baselines associated with a software product; 2) controlling the changes to basehnes and Lrel 3 software. Technical or_ non-technical application release of baxlines throughout the life cycle; 3) re' software not used in a safety decision by the NRC and cording and reporting the status of basehnes and the having local or limited use by the NRC.
proposed and actual changes to tne baselines; and 4) verifying the correctness and completeness of nonconformance. Any failure of any software docunient, baselines.
code, data structure, or process, to meet its require-ments or standards. Often called a problem, discrep-correctire action. General name for the process by which ancy, fault, or error.
nonconformances are corrected, verified and con-trolled.
non-derciopmental softwarc. Deliverable software that is not developed under the contract but is provided by '
corrective maintenance. Maintenance performed specifi-the developer, the Government, or a third party.
cally to overcome existing faults.
Non-developmental software may be referred to as reusable software, Government-furnished software, developer. The organization, usually a ci atractor. that de-or commercially available software depending on its 4
velops or maintains the software source.
1 55
qualification testing. A process that allows the sponsor to_
tools, firmware devices, and hardware necessary to determine whether a software product complies with test software. The automated tools may include but -
its requirements. quality assurance. A planned and
. are not limited to test tools such as simulation soft-1 systematic pattern of all actions necessary to provide -
ware, code analyzers, etc. and may also include those adequate confidence that a software product con-tools used in the software engineering environment.'-
forms to established technical requirements.
software unit. An element of the software design that can releasc. A configuration management action whereby a be compiled or assembled and is telatively small (e.g.,
d particular version of software is made available for a 100 lines of high. order language code).
4
- L specific purpose (e.g., released for test, released to opemtions) sponscr. %e NRC organization that sponsors and man-ages the software development / maintenance effort.
ne sponsor acts as the acquirer or buyer for the user.
reusabic software. Software developed in response to the requirements for one application that can be used, in sustaining engineering. The process that includes software
-- t o
whole or in part, to satisfy the requirements of an.
mamtenance and the software engmeermg activities other application.
that ensure that the integrity of the software's origi.
3 nal requirements set and design are retained.
software engineering enrironment. The set of automated tools, firmware devices, and hardware necessary 1 testing. The process of exercising or evaluating a software -
perform the software engineering effort, meluding o
of a dware product by manual or establishmg and maintaining the software deselop-automated means to verify that it satisfies specified i ment library. The automated tools may melude but requirements or to identify differences between ex-
[
are not limited to computer-aided software engmeer-pected and actual results.
ing (CASE) tools, compilers, assemblers, linkers, loaders, operating system, debuggers, simulators, scst case. A specific set of test data and asso'ciated procc-emulators, test tools, documentation tools, and data dures developed for a Particular objective, such as to base management systems.
exercise a particular program path or to verify com-
.-1 pliance with a specific requirement, t
software development library. A controlled collection of software, documentation, and associated tools and test plan. A document prescribing the approach to bel procedures used to facilitate the orderly develop-taken for intended testing activities. The plan typi-f I
ment and subsequent maintenance of software.De cally identifies the items to be tested, the require-software development library contams the develop-ments being tested, the testing to be performed, test d
mental configuration as part of its contents. A soft-schedules, personnel requirements, reporting re-ware developmem library provides storage of and quirements, evaluation criteria, any risks requiring.
5 controlled access to software and documentation in contingency planning.
human-readable form, machine readable form, or both(ne software development library may als testprocedure. Detailed instructions for the setup, opem-contam management data pertinent to the software tion, and evaluation of results for a given test. A set of development project.
associated procedures is often combined to form a.
test procedures document.
i software life cycle. The period of time that starts when a software product is conceived and ends when the test report. A document describing the conduct and results i
product is retired from use.
of the testing of a software product ora component of -
a software product.
j softwarc rnaintenance. Modification of a software product after delivery to correct faults, to improve perform-user. The organization or persons who will use the soft _
j ance or other attributes, or to adapt the product to a ware product being developed.
t changed environment.
scrification. The process of determining whether or not.
softwatcplans. A collective term used to describe the de.
the products of a given activity or phase of the soft.
eloper's plans, methodologies, standards, and pro-ware development life cycle meets its requirements.
v
.cedures for software management, software engi-neering, verification and validation, documentation, rafidation. He process of evaluating a software product at j
product evaluation, and ccmfiguration management.
the end of the software development process to en-R software test environment. The set of automated sure compliance with software requirements.
56
V APPENDIX C Reference Documents 1.
ANSI /ASME NOA-1-1983, Quality Assurance 7.
ANSI /IEEE Std 829-1983, lEEE Standard for Soft-Program Requirements for Nuclear Facilities ware Test Documentation 2.
ANSI /ASME NQA-2a-1990 Addenda (Part 2.7) to 8.
ANSI /IEEE Std 1012-1986, IEEE Standard for ASME NOA-2a-1989 Edition Quality Assurance Software Verification and Validation Plans Requirements for Nuclear Facility Applications 9.
DOD-STD-2167A,- Military Standard, Defense 3.
ANSI /IEEE Std 730-1989, IEEE Standard for Soft-System Software Development,29 February 1988 ware Quality Assurance Plans
- 10. DOD-S'ID-2168, Military Standard, Defense -
l I * ' ** # "" # ##E***
^E 4.
ANSI /IEEE Std 983-1986, IEEE Guide for Soft-ware Quality Assurance Planning 11.
DOD-STD-1521B, Military Standard, Technical Reviews and Audits for Systems, Equipments, and 5.
ANSI /IEEE Std 729-1983,IEEE Standard Glossary Computer Software,4 June 1985.
of Software Engineering Terminology 12.
Capability Maturity Model for Software. CMU/
6.
ANSI /IEEE Std 828-1983, IEEE Standard for Soft-SEl-91-TR-24, Software Engineering Institute,.
ware Configuration Management Plans Camegie Mellon University, August 1991 i
t 4
i 1
57
i v
d q
e i
i
~
2 3
ll l
)
7 Printed on recycled paper-1 1
i Federal Recycling Program o
l l
~"
- bl!A/U\\ALL13A&MhdMJUU Num.wgnuw em.
>, n;i I
~
f b.
' UNITED STATES-FIR $T CLAS'i Matt--
h-
- NUCLEAR RE;ULATORY COMMISSION
- POSTAGE AND FEES PAID
[-
WASHINGTON, D.C. 20555-0001 -
USNRC~
PERMIT NO. G 67 ~
p.
- OFFICIAL BUSINESS PENALTY FOR PRIVATE USE, $300 -
-h li r
l h
s 9
t
,E A
r p
r-u h '. -
_ naL]Mu;g':-s..
u 4 '_:u.
?'
- Q l -'
3 x
.._..-i:
'-L:_:
_ _