ML20215E399
| ML20215E399 | |
| Person / Time | |
|---|---|
| Site: | South Texas  |
| Issue date: | 10/07/1986 |
| From: | Kadambi N Office of Nuclear Reactor Regulation |
| To: | Goldberg J HOUSTON LIGHTING & POWER CO. |
| References | |
| NUDOCS 8610150276 | |
| Download: ML20215E399 (25) | |
Text
-
i i
07 E Docket Nos.:'-
50-498 and 50-499 Mr. J. H..Goldberg Group Vice President-Nuclear Pouston I.ighting and Power Company P. O. Box 1700 Houston, TX 77001
Dear Mr. Goldberg:
4
SUBJECT:
REPORT ON THE THIRD AUDIT OF THE QDPS AT SOUTH TEXAS PROJECT, UNITS 1 AND 2 The NRC staff audited the design process and the verification and validation plan for a part of the Qualified Display Processing System (QDPS) being built for you by Westinghouse Electric Corporation. The audits were performed during July 15-16, 1986, at the Westinghouse manufacturing facility. The enclosure to this letter provides reports on the audits.
Based on our audit of the design prucess and of the verification and validation activities for the QDPS, the staff concludes that it is generally acceptable for you to continue the design and manufacture of this system and to execute the verification and validation plan. Powever, this acceptance is conditional on the (1) confirming that the validation plan has been executed as described during the third audit, and (2) verifying that acceptable methods are being used that will prevent inadvertent PROM usage. The staff's review of these items will be conducted at the fourth audit which has been tentatively scheduled for November 17,18, and 19,1986.
Sincerely, N. Prasad Kadambi, Pro.iect Manager PWR Pro.fect Directorate No. 5 Division of PWR I.icensing-A
Enclosure:
1.
Audit Reports, Houston I.ighting 8 Power Company, Qualified Distribution:
Display Processing System Docket Files JPartlow NRC PDR BGrimes cc:
J. Joyce, NRC (RSIB/DSR0) local PDR EJordan L. Beltracchi, NRC (EICS8/PWR-8)
PD#5 R/F ACRS (10)
J. Mauck, NRC (EICSB/PWR-A)
TNovak MRushbrook F. Rosa, (EICS8/PWR-A)
NPKadambi PD#5 NPKadambi:es 10/]/86 e610150276 861007 PDR ADOCK 05000498 A
[4
%oo 9
UNITED STATES 7
NUCLEAR REGULATORY COMMISSION
)
3 y
WASHINGTON, D. C. 20655 OCT e 7 m Docket Nos.:
50-498 and 50-499 Mr. J. P. Goldberg Group Vice President-Nuclear Pouston lighting and Power Company P. O. Box 1700 Pouston, TX 77001
Dear Mr. Goldberg:
SUBJECT:
REPORT ON TPE THIRD AUDIT OF TPE ODPS AT SOUTH TEXAS PROJECT, UNITS 1 AND 2 The NPC staff audited the design process and the verification and validation plan for a part of the Qualified Display Processing System (00PS) being built for you by Westinghouse Electric Corporation. The audits were performed during July 15-16, 1986, at the Westinghouse manufacturing facility. The enclosure to this letter provides reports on the audits.
Rased on our audit of the design process and of the verification and validation activities for the 0DPS, the staff concludes that it is generally acceptable for you to continue the design and manufacture of this system and to execute the verification and validation plan. Powever, this acceptance is conditional on the (1) confirming that the validation plan has been executed as described during the third audit, and (2) verifying that acceptable methods are being used that will prevent inadvertent PROM usage. The staff's review of these items will be conducted at the fourth audit which has been tentatively scheduled for November 17, 18, and 19, 1986.
Sincerely, nK-N. Prasad Kadambi, Pro. ject Manager PWR Project Directorate No. 5 Division of PWR I.icensing-A
Enclosure:
1.
Audit Reports, Pouston I.ighting
& Power Company, Qualified Display Processing System cc:
J. Joyce, NRC (RSIB/DSRO)
L. Beltracchi, NRC (EICS8/PWR-8)
J. Mauck, NRC (EICS8/PWR-A)
F. Rosa, (EICSB/PWR-A)
Mr. J. H. Goldberg e
Houston (fahting and Power Company South Texas Project cc:
Brian Berwick, Esq.
Resident Inspector / South Texas Assistant Attorney General Droject Environmental Protection Division c/o U.S. Nuclear Regulatory Commission P. O. Box 12548 P. O. Box 910 Capitol Station Bay City, Texas 77414 Austin, Texas 78711 Mr. Jonathan Davis Mr. J. T. Westermeir Assistant City Attorney Manager, South Texas Project City of Austin Houston I.ighting and Power Company P. O. Box 1088 P. O. Box 1700 Austin, Texas 78767 Houston, Texas 77001 Ms. Pat Coy Mr. H. l. Peterson Citizens Concerned About Nuclear Mr. G. Pokorny Power City of Austin 5106 Casa Oro P. O. Box 1088 San Antonio, Texas 78233 Austin, Texas 78767 Mr. Mark R. Wisenberg Mr. J. B. Poston Manager, Nuclear licensing Mr. A. Von Rosenberg Houston lighting and Power Company City Public Service Boad P. O. Box 1700 P. O. Box 1771 Houston, Texas 77001 San Antonio, Texas 78296 Mr. Charles Halligan Jack R. Newman, Esq.
Mr. Burton L. Lex Newman & Holtzinger, P.C.
Bechtel Corporation 1615 1. Street, NW P. O. Box 2166 Washington, D.C.
20036 Houston, Texas 77001 Melbert Schwartz, Jr., Esq.
Mr. E. R. Brooks Baker & Botts Mr. R. L. Range One Shell Plaza Central Power and light Company Houston, Texas 77002 P. O. Box 2122 Corpus Christi, Texas 78403 Mrs. Peggy Buchorn Executive Director Citizens for Equitable Utilities, Inc.
Route 1 Box 1684 Brazoria, Texas 77422 J
l i
, ~,,.
_ ~ -. _.. _ _ _
__y y,_,
y
Houston lighting & Power Company South Texas Project
-o.
cc:
Regional Administrator, Region IV U.S. Nuclear Regulatory Commission Office of Executive Director for Operations 611 Ryan Plaza Drive, Suite 1000 Arlington, Texas 76011 Mr.1.anny Sinkin, Counsel for Intervenor c
Citizens Concerned about Nuclear Power, Inc.
Christic Institute 1324 North Capitol Street Washington, D.C.
20002 licensing Representative Houston lighting and Power Company Suite 1309 7910 Woodmont Avenue Bethesda, Maryland 20814 1
i i
l l
l
ENCLOSURE 1 THIRD AUDIT REPORT HOUSTON LIGHTING AND POWER COMPANY QUALIFIED DISPLAY PROCESSING SYSTEM 1.
BACKGROUND The Houston Lighting and Power Company (HL&P) is developing a microcomputer based system, which will perform functions that will directly impact upon the safe operation of its South Texas Project (STP). The STP is a dual 1250 MW Westinghouse Pressurized Water Reactor (PWR) Nuclear Generating Station, which is currently scheduled for completion and licensing by June 1987 The_ microcomputer based system is being designed by Westinghouse and is called the Qualified Display Processing System (QDPS). This system is described in the applicant' FSAR and it is being designed to perform the following functions:
Data acquisition, processing, and qualified (Class IE) display for Post Accident Monitoring, Data acquisition, display, and analog control for Safe Shutdown and to address separation / isolation concerns for a postulated Control Room / Relay Room fire, Data acquisition and digital processing of steam generator water level signals and primary coolant system hot leg temperature signals and transmission of these processed signals for use by the Reactor Trip System.
The staff's review of the QDPS began with two separate audits of the Verification and Validation Plan. These audits were conducted during August 26-29, 1985 and March 24-27, 1986. The staff's audit results and reconsnendations of these two audits are presented in Reference 1 and Reference 2 respectively.
In preparation for the third staff audit on the ODPS, the applicant requested a meeting with the staff, which was held on July 9,1986, at NRC Headquarters in Bethesda, Maryland. During the meeting, the applicant identified and discussed the validation plan that was being used and the methods to be used to resolve l
open issues from its previous audits. Also, the applicant identified the design documents that would be available for staff audit. As discussed in the previous audits, because of problems encountered during the verification phase, the valida-tion phase should be sufficiently broad in scope such that the computer software is tested in its entirety, and tested in an environment corresponding to that which is its intended use environment. A detailed plan for validation testing i
should be drawn up and approved by V8V management in advance of the beginning of the validation test program. The validation plan should be well prepared and comprehensive to assure that the system performs as required. Detenninations should be made that the docementation of the computer program accurately describes what is in the progran and how it is used and it should be ascertained that every specification has been properly met by the computer program. The validation plan should include reviews of the program and its documentation for reliability, error recovery, and performance under a broad range of inputs (including bad inputs).
i Orderly procedures for changes, testing of changes, and quality control of changes should be spelled out before the program is delivered.
Independence is a necessary criterion for V&V. The importance of organizational independence is proportional to the degree of the systems importance to safety. To gain maximum confidence, the V&V team should be sufficiently independent from the design group to be free from conceptual bias and motivational conflicts.
The staff's third audit of the QDPS was conducted during July 15-16,1986, at Westinghouse Electric Corporation's Nuclear Facilities, located in Monroeville, Pen 6sylvania. The audit was conducted by Mr. J. L. Mauck and Mr. S. Weiss of
~
the NRC Staff and Ms. J. Frawley of SoHaR (NRC consultant). Enclosure 2 con-tains a list of personnel at the audit. A copy of the NRC consultant's report 4
is provided as Enclosure 3.
II. SCOPE OF AUDIT The purpose of this audit was to review the QDPS validation plan. As discussed in the second audit, the validation plan must be sufficiently broad in scope to address any discrepancies in the design process and account for the lack of independent, fonnal design verification. This means that the validation plan should include a technique which demonstrates completeness between Functional Requirements and Software Design Specifications that were turned over to the validation team.
In addition, we identified other issues during the previous audits that were also addressed during the third audit.
i
4 III.
FIRST AND SECOND AUDIT OPEN ISSUES Issue 1 - No Evidence of the Use of a Requirements Matrix to Structure The Decomposition of the Functional Requirements Decomposition From Functional Requirements to Software Design Specification May Be Incomplete Examination of the documents from the second audit indicate that the functional requirements from HL&P were not well documented and were the result of dynamic evolution.
Prior to the second audit there was an accumulation of the documenta-t' ion 'for the functional requirements in the appropriate form with the appropriate levels of signatures, the documentation of the software design documents and a functional requirements matrix prepared by the design group showing an audit trail from the functional requirements to the corresponding software unit. However, the completeness of this matrix could not be demonstrated to the satisfaction of the second audit team. The abbreviated format of the matrix was judged to in-dicate incompleteness by restricting the entries to software functional require-ments and by not including those which were addressed by hardware or by other subsystems.
Prior to the third audit a second functional requirements matrix was prepared by two engineers who were independent of the design team.
For each subsystem every requirement was listed by document number and paragraph number with a full
.xa
-c.
description of the requirement, a statement of where the requirement was met, and the functional test required for validation. The third audit team reviewed this matrix and its associated documentation and found it acceptable. How-ever, the final acceptance of the completed validation phase will be performed at the fourth audit.
Issue 2 - Question of Independent Design Verification As identified in our first audit report, we noted the lack of independent design veR fication within the design process.
It is our engineering judgment that the validation plan should be sufficiently broad in scope to address discrepancies in the design process and to account for the lack of independent, formal design verification. Therefore, based on this judgment, the question of independent design verification within the design process was shifted to the validation plan.
The details regarding the present status of the validation phase was presented at the third audit.
The response by the applicant is that the validation plan will be sufficiently broad in scope to address deficiencies in the design process and to account for the lack of independent formal design verification.
Our evaluation of the verification / validation plan as implemented supports this response:
Verification / validation is performed on each software unit following uniform and exhaustive procedures. A test requirement was written fror.i each sentence of the functional requirement. 400-500 validation tests are being done with each one being signed off. Fail documents will be issued which will explain the failures and determine the corrective action to be taken (to be reviewed at the fourth audit).
A design review process which evaluates the system's design methodology and implementation (referred to by Westinghouse as Prudency Design Review)
~ has been added to the functional requirements testing. This review examines the software units for good software design and for any oversights in the implementation whTch would lead to the introduction of unexpected or in-correct results. While the functional requirements testing will validate the software against the requirements, the prudency review performs many of the functions of a design review.
The audit trail for verification will be strengthened by the broader valida-tion plan.
It is apparent that there has been a large commitment of time and money to per-form V and V.
The functional decomposition has been done in great detail.
The coding decomposition from the test procedure documents correct the problems noted in the first audit (e.g., lack of independent verification).
The prudency review
.~
is a good addition that will study the software's internal workings. This addresses the issue of independent verification.
The staff will confirm that the validation plan has been properly executed during the fourth audit.
Issue 3 - Software Criticality The response by the applicant was that the same level of V&V effort is performed
~
on all software. units because of the interactions between QDPS subsystems software.
1 This issue was raised in the first audit because of the concern that the manpower and computer intensive V&V plan proposed could not be accomplished within the time and cost constraints. An examination of the verification audit trail shows that the level of effort is the same for all units and that the verification and reverification is proceeding in a timely fashion. The trouble reports which docu-ment both lack of correspondence between code and documentation and software code problems are being cleared within a reasonable time frame.
The staff and the consultant (SoHar) have concluded that this issue is resolved.
i
Issue 4 - Clarification of Physical Media and Verification of Program Listing One of the open itens in our consultants (SoHar) audit report was the verifica-tion of the physical media that represents the program. The applicant and Westinghouse requested clarification / interpretation of this item. The verifi-cation of physical media means those activities performed to ensure that the burned in programmable read only memory (PROM) contains the authorized program (i.e., security and safeguard measures).
~
The V&V Team has. control of the V&V Configuration Management System (CFMS) which contains the authorized programs.
The programs are not directly accessible by the Design Teams.- The V&V Team controls the physical media (i.e., PROMS) which contain the programs utilized during the Validation process and perform the following to insure its integrity:
Down-loading of the executable load module (i.e., HEX file) from the o
V&V CFMS on the VAX 8600 Computer System to the Intel PROM burner.
NOTE: HEX file contains checksum which insures that the program transfer to the PROM burner is accurate.
o Burning of PROMS.
o Verification that PROMS were burned correctly.
o Marking of the PROMS.
i o
Reverification of PROMS against the HEX file after Validation testing is complete to insure that the PROMS still contain the proper HEX fi.le programs.
We found the strict configuration management procedures acceptable. However, the steps of manually labelling each PROM with the subsystem, the cabinet, the slot and the unique version identifier did not entirely convince us that the correct version and the correct PROM would always be installed and not be subject to malicious mischief. The design does not take advantage of some of the capabil-ities of digital systems. Programmable systems are not only' capable of executing diagnostics but also of reporting version identifiers, installation dates and other information if so designed.
This item remairi an open issue.
(See consultant report.) Further clarification of procedures and safeguards will be discussed at the fourth audit.
10-Issue 5 - Reliability Analysis A study was performed on a representative subsystem on the Qualified Display Pro-cessing System (QDPS), in order to compare the reliability of the existing digital QDPS and a hypothetical implementation of the same function using analog hardware.
The reliability information derived is in the form of availability data, where availability is defined to be the probability that the system will be operational at a randomly selected future instant in time.
The~ study determined availability data utilizing the GO methodology developed by Kaman Sciences Corporation and module-level reliability data. The module-level reliability data.was obtained from the following sources:
o Accelerated life testing of Intel board level products that is conducted at the manufacturer's facilities.
o IEEE Standard 500, Reliability Data for Nuclear Power Generating Stations.
I o
Westinghouse 7300 system failure information.
o Data from manufacturers, i
-n
-wn-,
r _.-,.
+
The data that is available on the individual components, when combined with the GO methodology, allows the overall system availability to be determined to a degree of confidence that is commensurate with the comparative nature of this study.
Due to the comparative nature of the study, the availability was determined for one channel. This increased the clarity of the comparison.
For the analog system two. assumptions were utilized. Assumption 1 was that the plant would be shutdown immediately to repair a failed Resistance Temperature Detector (RTD),
modeled as an AND gate in the GO model. Assumption 2 was that the plant would continue to operate and repair of a failed RTD would occur at the next scheduled shutdown, modeled as an OR gate in the G0 model.
Several functions performed by the digital SGWLCS are so complex, that a direct analog counterpart would not, in practice, be constructed. These functions include failed RTD signal rejection and self diagnostics.
Benefits of these functions were not quantified or included in this study.
To summarize, the study demonstrates that digital and analog systems carefully designed and maintained have high availability. The digital system is shown to have an availability / reliability as high as or slightly higher than an analog system. The staff has concluded that the Qualified Display Processing System
provides a highly reliable system for its application at the South texas Project and that its assessed reliability is acceptable.
Issue 6 - Software Maintenance Practices o
Software maintenance practices prior to and during operational use The applicant has committed to utilize the existing V&V program for all software maintenance / modifications until such time that an "in-house" utility program has been developed and received an appropriate staff ap-proval. There appears to be strict control within the present V&V config-uration management system and adequate procedures for issuing new systen revisions. -Therefore, this issue is resolved until such time that HL&P provides changes in the software maintenance practice area. The consultant agrees with this position (see Enclosure 3).
o User and Maintenance Documents Detailed user and maintenance documents were examined by the staff and the consultant and found to be adequate. Therefore, this issue is resolved.
Issue 7 - Instrumentation and Control The following review topics were highlighted by the EICSB during the second audit for subsequent review during one of the remaining QDPS audits or during additional design review meetings, which would be scheduled at a later date.
o Interface with alternate remote shutdown capability.
o Isolation devices (interface with non Class IE systems).
o Interface with Class IE systems.
o Testability (R.G. 1.22, 1.118 and IEEE-338) (including proposed Technical Specifications).
o Bypassed and Inoperable Status Indication (R.G.1.47).
o EMI Susceptibility.
o R.G.1.75 Separation of redundant safety trains within ODPS.
o Manual initiation methods, o General compliance with IEEE-279 (e.g., single failure criterion, resetcapability).
The applicant presented design details regarding the above EICSB items. Based on these details the staff concludes, that with the exceptions discussed below, these items can be resolved during the staff's Chapter 7 review and during
t
- e the EICSB site visit. Details regarding these items will be provided in a supple-ment to the Safety Evaluation Report. The exceptions are:
(1)
Isolation Devices (2) EMI Susceptibility (3) R.G.1.75 Separation of Redundant Safety Trains within QDPS The applicant has committed to conduct fault and RFI tests during the fourth quarter of TY 86. The results will be documented at a future date and reviewed as part of the Chapter 7 EICSB review.
V.
SUMMARY
AND CONCLUSION Based on our audit of the design process and the verification and validation plan for the QDPS, the staff concludes that it is generally acceptable for the appli-cant to continue the design and manufacture of this systen and to continue to execute the verification and validation program. The staff's review of the valid-ation information provided during the third audit has done much to restore con-fidence in the verification and validation of the QDPS and to correct the defi-ciencies noted in the first and second audits. The commitment to provide ade-quate documentation and e complete functional requirements matrix has been noted with approval. The independence and dedication of the V&V team have been demon-strated in the review, as well as their organized, automated, and in-depth approach.
1 However as stated in Issue 1 of this evaluation, the acceptance is conditional on the staff confirming that the validation plan has been properly executed.
In addition, we have identified another open issue (Issue 4), Clarification of Physical Media. These issues will be addressed during the fourth audit of the ODPS. The instrumentation and control issues (Issue 7) will be reviewed as part of the Chapter 7 EICSB review.
REFERENCES
- 1. ~ Letter from N. P. Kadambi, NRC to J. H. Goldberg, Houston Lighting and Power Company,
Subject:
Audit Report on the QDPS at South Texas Project, Units 1 and 2, dated January 30, 1986.
2.
Letter from N. P. Kadambi, NRC to J. H. Goldberg, Houston Lighting and Power Company,
Subject:
Audit Report on the QDPS at South Texas Project, Units 1 and 2, dated May 19, 1986.
~
r
ENCLOSURE 2 QDPS Audit Attendance List July 15-16, 1986 J. Amin, BEC Chuck Cor1, W Tom Crawford? HL&P ENG Yvonne Williams, BEC Mary Moreton, BEC Joanna Frawley, SoHaR Jerry L. Mauck, NRC S. H. Weiss, NRC
'N. Prasad Kadambi, NRC Dennis Adomaitis, W I&C Carl Vernon, W STP Lic Team Jack Bailey, NL&P, ENGR Len Casella, HL&P, ENGR Ralph Delucia, W Sim ENGR G. Lang, W Lic -
John Waclo, W Sim Eng.
Paul
'H. Mathewes, Jr. MASI, Inc.
Gary B. Glisan, MASI, Inc.
Eric P. Casteel, CIS
ENCLOSURE 3 SoHaR incorporated H. Hecht, President 19 July 1986 L86-111 Mr. G. Mauck Nuclear Regulatory Commission 7920 Norfolk Ave.
Bethesda MD 20014
Subject:
Audit of QDPS V&V Planning, 15-16 July
Dear Mr. Mauck:
' Enclosed'please find the report on the subject audit prepared by Ms. Joanna Frawley of our company. As noted, considerable progress has been made in developing a V&V methodology that is auditable and on par with current standards in the Industry. We are pleased that we could be of help in achieving this result.
Please do not hesitate to contact us if there are any questions in relation to the report or to other inatters pertaining to the audit. We look forward to participating in the final audit on the project and will appreciate hearing from you as soon as a firm date has been set.
Sincerely yours,
/
r
/Y'Yd6db{
HH:1J Encl. as noted.
1040 South La Jolla Avenue
- Los Angeles, California 90035
- Telephone (213) 935-7039 other Locations 8500 Wilshire Bl., #1027, Beverly Hills, CA 90211
- 5225 Pooks Kill Rd., #1513S, Bethesda, MD 20814
ENCLOSURE 3 REPORT ON T}ilRD AUDIT OF QDPS AT WESTINGHOUSE PITTSBURGH 7/15 - 7/16/86 GEhERAL COMMENTS The objectives of the third audit of the QDPS(Qualified Display Processing System) were resolution of open issues from previous audits evaluation of the validation plan evaluation of validation process (to date).
Because the second audit was added as a result of deficiencies noted in the first audit and did not include SoHaR representation. this audit report will include not only the observations made during the third audit but also those roulting from a lengthy examination and discussion of the material f rom the second audit.
This approach is necessary because resolution of the open issues is based on material provided ~ln both the second and the third audit.
The body of this report will address each open issue which relates to software verification / validation, evaluation of the validation plan, evaluation
'of the validation process to date, and conclusions.
OPEN ISSUES FROM PREVIOUS AUDITS ISSUE 1 (IST AUDIT) - NO EVIDENCE OF THE USE OF A REQUIREMENTS MATRIX TO STRUCTURE THE _ DECOMPOSITION OF THE FUNCTIONAL REQUIREMENTS ISSUE I (2ND AUDIT) - DECOMPOSITION FROM FUNCTIONAL REQUIREMENTS TO SOFTWARE DESIGN SPECIFICATIONS MAY BE INCOMPLETE Examination of the documents from the second audit Indicate that the functional requirements from HL&P(Houston Power & Light) were not well documented and were the result of dynamic _ evolution.
Freezing of the design was done late in the life-cycle and may account for many of the deficiencies noted in the first audit.
Prior to the second audit there was an accumulation of the documentation for the functional requirements in the appropriate form with the appropriate levels of signatures, the documentation of the software design documents, and a functional requirements matrix prepared by the design group showing an audit trail from th7 functional requirements to the corresponding software unit.
- However, the completeness of this matrix could not be demonstrated to the satisf action of the second audit team.
The abbreviated format of the matrix was judged to indicate incompleteness by restricting the entries to software functional requirements and by not including those which were addressed by hardware or by other subsystems.
Prior to the third audit a second functional requirements matrix was prepared by two engineers who were independent of the design team.
For each subsystem every requirement was listed by document number and paragraph number with a f ull description of the requirement, a statement of where the requirement was met, and the functional test required for validation.
This mr.trix in combinstion eith the raudit trail from ths srcond audit is judged to be an appropriate response.
It is recommended that this issue be closed.
a ISSUE 2 (IST AUDIT) - LACK OF INDEPENDENT DESIGN VERIFICATION WITHIN THE DESIGN PROCESS The response by the applicant is that the validation plan will be suf ficiently broad in scope to address deficiencies in the design process and to account for the lack of Independent formal design verification.
Evaluation of the verification / validation plan as implemented supports this response:
Verification / validation is performed on each software unit following uniform and exhaustive procedures.
A design review process which evaluates the system's design methodology and implementation (referred to by Westinghouse as Prudency Design Review) has been added to the functional requirements testing.
This review examines the software units for good software design and for any oversights in the implementation which would lead to the introduction of unexpected or incorrect results.
While the functional requirements testing will validate the software against the requirements, the prudency review performs many of the functions of a design review.
- The audit tralI for verification wIll be strengthened by the broader validation plan.
It is recommended that this issue be closed at the fourth audit af ter assuring that the validation plan has been executed as described.
ISSUE 3 (IST AUDIT) - SOFTWARE CRITICALITY The response by the applicant is that the same level of V&V effort is performed on alI software units because of the Interactions between QDPS subsystems sof tware.
This issue was raised in the first audit because of the concern that the manpower and computer intensive V&V plan proposed could not be accomplished within the time and cost constraints.
An examination of the verification audit trail shows that the level of effort is the same for all units and that the verification and reverification is proceeding in a timely f ashion.
The trouble reports which document both lack of correspondence between code and documentation and software code problems are being cleared within a reasonable +1me frame.
It is recommended that this issue be closed.
ISSUE 4 (IST AUDIT) - VERIFICATION OF PHYSICAL MEDIA OTHER ISSUES (2ND AUDIT) - VERIFICATION OF PROGRAM LISTING The third audit included a discussion of the activities performed 4
to ensure that the programmable read only memory (PROMS) contains the authorized program. The main points of the discussion wsre:
l
- The V&Y team controls all modifications to computer system modules and the issuing of authorized versions The automated configuration management system assures that each version including alI program units. source code. object code, documentation, and hex representation which will be burned in the PROMS is permanently stored in a separate directory on the VAX.
- After initial Installation and validation the PROMS are removed and compared with the authorized version of the program.
- A unique checksum - for each PROM is stored in non-volatile memory.
The internal diagnostics calculate a checksum for each PROM every 10 seconds and perform comparison checks.
The third audit team had no difficulty with the strict configuration management procedures to this point.
The next steps of manually labelling each PROM with the subsystem, the cabinet, the slot, and the unique version Identifier did not entirely convince them that the correct version and the correct PROM would. always be Installed and not be subject to malicious i
mischief.
The design does not take advantage of some of the capabilities of digital systems.
Programmable systems are not only capable of executing diagnostics but also of reporting version identifiers. InstalIation dates and other Infccmation if so designed.
I It was recommended that this issue be left open for further clarification of procedures and safeguards at the fourth audit.
ISSUE 5 (IST AUDIT) - PERFORM RELIABILITY ANALYSIS i
This issue is limited to hardware reliability and does not include software reliability.
No recommendation will be made on this issue.
OTHER ISSUES (1ST AUDIT) - SOFTWARE MAINTENANCE PRACTICES The software maintenance practices examined during the third audit are considered to be adequate. This conclusion is based on the strict control within the configuration management system and the procedures for issuing new system versions.
i No information was provided to form a basis for judging the adequacy of any future practices, such as a change in the organization performing the maintenance.
~. -.
~
lt is recommend:d that this issus b3 closed end reopan:d only if chrng:s in softwtr@ maintan:nco prsctic:s occur.
OTHER ISSUES (IST AUDIT) - USER AND MAINTENANCE DOCUMENTS Detailed user and maintenance documents were examined and were judged to be adequate.
It is recommended that this issue be closed.
EVALUATION OF THE VALIDATION PLAN Positive aspects of the validation plan observed at the third audit:
Independent V&V team comprised of Westinghouse personnel from organizations other than the design group and consultants Use of highly automated tools Tools have had V&V applied Preparation of the functional requirements matrix and validation tests by engineers independent of the design group Thorough and comparable V&V testing of each program unit Addition of a prudency review to assure not only that the unit meets the. functional requirements (black box approach) but that it does so according to good program design practices Centralized control of configuration management by the VaV team.
The validation is judged to meet acceptable validation plan standards.
EVALUATION OF THE VAL IDATION PROCESS (TO DATE)
The implementation of the validation process to date is judged to be
- adequate, it provides good audit trails and easily understandable reports.
CONCLUS lONS The efforts to date to correct the deficiencies noted in the first and second audits have done much to restore confidence in the verification and validation of the QDPS.
The commitment to provide adequate documentation and a complete functional requirements matrix has been noted with approval. The independence and dedication of the V&V team have been demonstrated in the review, as welI as their organized, automated, and in-depth approach.
Much has been done to resolve the issues noted in the first audit and if the fourth audit confirms that the validation has been completed as described, the level of confidence that the QDPS sof tware meets the fundamental requirements will be restored.
,..