ML20205Q662
| ML20205Q662 | |
| Person / Time | |
|---|---|
| Site: | Rancho Seco |
| Issue date: | 03/30/1987 |
| From: | Julie Ward SACRAMENTO MUNICIPAL UTILITY DISTRICT |
| To: | Miraglia F Office of Nuclear Reactor Regulation |
| References | |
| JEW-87-379, NUDOCS 8704060013 | |
| Download: ML20205Q662 (34) | |
Text
r t
h i gSMUD
~'
SACRAMENTO MUNICIPAL UTILITY DISTRICT O P. O. Box 15830, Sacramento CA 95852-1830,(916) 452-3211 AN ELECTRIC SYSTEM SERVING THE HEART OF CALIFORNIA JEW 87-379 March 30,1987 Director of Nuclear Reactor Regulation Attention: Frank J. Miraglia, Jr.
Division of PWR Licensing-B U S Nuclear Regulatory Commission Washington D C 20555 Docket 50-312 Rancho Seco Nuclear Generating Station Unit #1 CRDR/SPDS RESPONSE TO NRC REQUESTS
Dear Mr. Miraglia:
Your letter dated February 2, 1987 " Report on September 29, 1986 to October 2, 1986 Audit at Rancho Seco Site", identified additional information required to complete your review of the Action Plan, Control Room Design Review (CRDR),
and upgrade of the Safety Parameter Display System (SPDS). Your letter also requested a schedule for providing this information.
Attachments 1 & 2 provide a partial response and a schedule for the District's response to the remaining items in your information request. In addition, several items have arisen during the series of phone calls and meetings with the staff in February, which are also addressed in Attachment 2.
It is the District's position that this letter addresses all known CRDR/SPDS issues required prior to restart of the plant. If you have any questions please contact John Atwell of my staff at extension 3906.
Sincerely, oh . ar L eputy General Manager, Nuclear Attachment cc: George Kalman, NRC - Bethesda A. D'Angelo, NRC - Rancho Seco P
P i DISTRICT HEADQUARTERS El 6201 S Street, Sacramento CA 95817-1899 k
p--r-o ATTACHMENT 1 ACTION PIAN AND CONTROL ROOM DESIGN REVIEW (CRDR)
Action Plan Documentation Request:
A description of the startup test program (with acceptance criteria) for-the main feedwater control system and a summary of the test results. .The staff desires to review the scope and depth of the test program prior to the test execution. .
Response
A copy of the Main Feedwater (MFW) System Status Report (SSR) and test plan outlines for Main Feed Pump Speed Test, Main Feed Pump Performance Test and Main Feed Pump Protection Test were presented to Leo Beltracchi, NRR/NRC, in Lynchburg, VA, on February 11, 1987. A copy of the Integrated Control System (ICS) SSR was also presented. Upon review, Mr. Beltracchi stated that these tests and status reports were not what was wanted but that the test procedures, acceptance criteria, and results of ICS/MFW control tuning at approximately 15% reactor power are needed. The major concern is plant stability while transferring from startup control valves to MFW control valves. This is a critical point-(approximately 15 to 40%
reactor power) during both startup and shutdown.
The District will docket the ICS tuning procedure (with acceptance criteria) with the NRC by May, 1987. The tuning results will be docketed after completion of ICS tuning (currently planned for August, 1987).
CRDR Documentation Request 1:
Commitment to perform a task analysis of Emergency Operating Procedures, Cautions, Notes, and Status tasks along with new operator tasks-identified as a result of the Emergency Operation Procedures rewrite project.
Response 1:
A task analysis of the revised Emergency Operating Procedures (EOPs) is underway. This will incorporate all procedure steps including: Caution, Notes, Status and Information pages. This is being performed to account for Control Room modifications and E0P revision and is expected to be completed in April.
Request 2:
Commitment to perform a task analysis of Hydrogen Purge Procedure A.52.
~o
, ATTACHMENT 1 (Cont.)
1 ACTION PIAN AND' CONTROL ROOM DESIGN REVIEW (CRDR) 3 Response 2:-
.A task analysis of A.52, Hydrogen Purge Procedure, will be performed with E0P task analysis. The majority of actions required of. the Control Room operators by this procedure are communications outside the Control Room because most of the actions are performed by auxiliary operators outside ,
the Control Room.
Request 3:
Commitment to compare the information and control requirements identified
- in items 1 and 2 with the control room inventory to verify control and
~
display availability and suitability.
j Response 3:
I The information and control requirements identified by task analysis (EOPs and A.52) will be compared with the updated Control Room inventory to verify control and display availability and suitability.
1 Request 4:
j Commitment to perform an assessment of new Human Engineering Observations identified during additional task analyses.
Response 4:
! The Human Engineering Observations (HEOs) identified during the above task 1 analysis, verification of task performance capabilities, validation of
! Control Room function or Control Room survey of modified components will l be assessed for potential consequences, proposed solutions and restart priority.
, Request 5:
i Commitment to perform a verification that the proposed solutions resolve human engineering discrepancies and do not introduce new human engineering discrepancies.
i
- Response 5
- The proposed solutions of HEOs (identified above) will be evaluated to assure that new human engineering discrepancies are not introduced.
NRC Comment (Enclosure 1, Sec 6.2):
Since Nuclear. Engineering Procedure 4123 was in preparation, the total extent of human factors engineering participation in procedural review 1
could not be determined.
)
P ATTACHMENT 1 (Cont.)
ACTION PLAN AND CONTROL ROOM DESIGN REVIEW (CRDR)
E Response:. {
NEP 4123, Human Factors Engineering Program, has been prepared and has been through the first round of reviews. The comments generated during-the first review cycle do not negate any programmatic aspects and still provide for human factors expert analysis of modifications to the control Room and affected emergency operating procedures. NEP 4123 will be available for review upon its final approval.
i l .
i t
l l
l i
i 1
t i
i a
i 3
i
- - , - - . .~ .
4
- l. ATTACHMENT 2 I
SPDS ITEMS (2/2/87).
i Question 8.1:
The staff requested that a copy of the final design of the' alphanumeric display formats be included in the revised SPDS SAR.
Response 8.1:
Since providing Mr. Leo Beltracchi with photographs and functional
- description at the February 11, 1987 audit at Lynchburg, VA, some changes have been made. The District will provide photographs and descriptions of I the final version of all the SPDS displays by April 1987.
i
! -Question 8.2:
! If the radiation displays of the rad waste area stack monitor, the reactor j
building effluent monitors, the auxiliary building effluent monitor, and the A and B main steam line monitors are not contained in the revised i alphanumeric displays, the licensee should justify why they were lef t -out 4 of the displays.
Response 8.2: ;
The radiation displays described in the question will be available on the l SPDS.
i Question 8.3:
'I As the SPDS will be upgraded to safety grade status, provide data on the
- isolation devices used to suitably isolate the SPDS from electrical
} interference with equipment and sensors that are used in non-safety i systems.
(
Response 8.3: ,
As requested in a phone call with the staff on February 18, 1987 the 1
District has reviewed the isolators in the SPDS. Based on this review it '
has been determined that isolator numbers 8, 9,10, 'and 11 of the SPDS Loop Diagram drawing are being used to provide isolation between. channels and Class 1 and 2 equipment.
! Per discussions with the staff a meeting will be held at Rancho Seco, tentatively scheduled for the week of April 6,1987, to provide a walkdown
- of the isolation requirements. At this meeting preliminary test results t of the isolators will also be available for review by the staff. The data-
! availabla will provide all the information requested by the NRC in Attachment 1 to Enclosure 2 of the February 2,1987 letter (item la-g).
l L
v i [- -
J
' ATTACHMENT 2 (CNitI) ,
5, h SPDS ITEMS (2/2/87)'- ~n Question 9.1:
The audit team reviewed the schedules and observed that the CRTs and the sof tware modifications are critical path items and may impact the i schedule. The current schedule appears to be acceptable. The staff
- - requested the licensee to inform the NRC Project Manager of any significant changes to the schedule. 'b Response 9.1
- The District will complete the SPDS upgrade prior to restart. The seismic
- display monitors are on-site and will be installed prior to restart.
l Sof tware. modifications are scheduled to be complete in April 1987 and will ,
be installed and functionally tested prior to restart.
l Question 9.2:
The licensee was requested to docket their commitment to meet Appendix
< "R". The licensee was also required to provide for staff review test reports to demonstrate that the isolators between the CCU and the IDAD' {
will meet the isolation functions and the single failure criteria. 1 gj i
s'i' Response 9.2: ~' '
)t '
1 The District intends to meet the applicable requirements of Appendix R.\
j Further discussion with respect to Appendix R considerations was provideds j in the November 20, 1986 District letter (item 4.6.a.6.0). For information on isolators see the response to question 8.3.
Question 9.3: ;
J The licensee will compare the Rancho Seco SPD',' verification and validation (V&V) process against the requirements of RG 1.152 and provide results of l the comparison to the staff. '
l Response 9.3:
The District has compared the SPDS verification and validation (V&V)-
process with the requirements of Regulatory Guide 1.152. This comparison was presented as part of the SPDS' audit conducted et B&W in Lynchburg, Va.
I February 10-12, 1987. At this audit the report was found acceptable. The report is provided with this letter as Enclosure 1 to this attachment.
{
}
i J
l i
I i
i
__..._.,.,_.____,,.__,--._,_m. . - , - _ . . .
l l
,, 2 \
C , ATTACHMENT 2 (Cont.)
i SPDS ITEMS (2/2/87)
! . Question 9.42 The audit team and the licensee agreed that the paper walk-through was a ,
preview of a more detailed V&V audit to be conducted pending completion of the software effort by the licensee. The walk-through was tentatively scheduled for January 1987, at the B&W offices in Lynchburg, Virginia. ,
Response 9.4:
a This walk-through occurred at Lynchburg, Va. on February'10-12, 1987. The a hardware and software of the containment pressure and RCS cold leg Y r temperature were reviewed and found to be satisfactory. .MinorLupdating~of the drawings was required as a result of the audit. These changes are .
I currently being made. No further action is required on this ites.
]
, Question 9.5:
i i The licensee will. address the aspects of common mode failure in the 4
software and submit their findings to the staff.
Response 9.5:
The aspects of common mode failure in the software was addressed by the
, District in a letter'to the NRC dated November 20, 1986 (item 4.6.a.2.0).
Question 9.6:
The-team's concerns centered around the Technical Specifications (TS) for 4
i the SPDS and how the TS would be handled in the event of modification and changes to the SPDS. The licensee will redo the response to this question
- addressing the TS aspects of the SPDS in greater detail and will docket ,
the response. .In the revised response the licensee should also discuss.
j how configuration control for both hardware and sof tware will be achieved
]J and how modifications will be qualified and tested.
Response 9.6:
The District will submit the amendment to the Rancho Seco Technical j Specification by restart of the unit. This submittal will address i modifications and changes to SPDS as noted in the staffs concern. The
{ District intends to administrative 1y adhere to the provisions of the i proposed Tech Spec while remaining in compliance with existing applicable Technical Specifications until approval of the amendment.
The requested information/ procedures for maintenance and configuration
- control programs will be provideu in April 1987. I i i 1
j 1
I i
i j
_._,-__-._._..._,___..._...-_,-_..___._;.,_.____,___a_.,-___ _ _. _ ._
x o
ATTACHMENT 2 (Cont.)
SP'DS ITCitS (2/2/87)
Question 9.7: ,
The licensee'will update the SPDS r611 ability analysis and submit the results for staff review. The licensee will perform a reliability comparison between a digital data / display channel versus an analog data / display channel. " Th'.s comparison will, he performed on a non-interference basis with[the plint's restart schedule. The results of the comparison will be submitted for staff raview.
Response 9.7: ,
The District included the revised reliability analysis of the SPDS in the January 12, 1987 submittal. A compa.rison of the digital SPDS with an analog system will be performed on a non-interference basis, as suggested by the staff, after restart of the unit. This comparison will be provided to the staff in October 1987.
Question 9.8:
Provide the documents and data thr.t illustrate that the upgraded SPDS complies with industry and regulatory criteria and standards for safety-related systems (i.e., IEEE Std. 323, 1974, Standard for Qualification of Class 1E Equipment for Nuclear Power Generating Stations, IEEE Std. 344-1975, Recommended Practices for Seismic Qualification of Class 1E Equipment for Nuclear Power, etc.), per RG 1.97.
Response 9.8:
The District will provide the documents and data currently available to show compliance wiu. 'STJ: 323, 1974 and 344-1975 in the April 1987 submittal. For the cura-st ongoing modifications this documentation will be provided when it becomes available in May 1987.
Question 9.9:
Describe the steps needed to restart an upgraded SPDS upon total loss-of fower t' the system.
Response 9.9:
The restart of SPDS upon loss of power was addressed in the November 20, 1986 submittal (item 4.6.a.1.0).
Question 9.10:
Provide and discuss the Technical Specification that is to be used for the operational system. Discuss the scope and depth of the tests used to.
evaluate operability of the system. Also, discuss the technical basis used to select the test period.
. ,. ,. -,m. ._.... - ..- - n y ,%.. - . _ ., , , , , , _.,m , ., __ -
, , , ..,,.,y .
ATIACHMENT 2 (Cont.)
SPDS ITEMS (2/2/87).
Response 9.10:
i The District will submit the amendment to the Rancho Seco Technical Specification by restart-of the unit as described in the response to Question 9.6.
General Question:
J Provide a power supply schematic for SPDS.
Response
The schematic will consist of a block diagram showing power supplies and will be provided in the April 1987 submittal.
Attachment 1 to Enclosure 2 Question 1:
Apply questions "a" through "g," as appropriate, to all devices that are to be used as either isolators or interfaces between Class 1E systems and ;
non-Class 1E systems. Examples of such devices are isolation devices, multiplexers, fiber-optic cable, circuit breakers, inverters, converters, and uninterruptable power supply systems.
Please provide the following:
- a. A description of the specific testing performed to demonstrate that ,
4 the device used to accomplish electrical isolation is acceptable for 1 3' its application (s). This description should include elementary diagrams, when necessary, to indicate the test configuration and should discuss how the maximum credible faults were applied to the device.
l
- b. The data to verify that the maximum credible-faults applied during the tests were the maximum voltage (AC and DC) at. the maximum i potential current to which the device could be exposed, and define how the maximum voltages / currents were determined.
'l l
l i
l r
__m._ _._ .. , _ - . . . - , . . - - . . . . . , - . . - _ - . _ - . . - - .
ATTACHMENT 2 (Cont.)
SPDS ITEMS (2/2/87)
Question 1 (Cont.):
- c. The data to verify that the maximum credible faults were applied to i the output of the device in the transverse mode (between signal and return) and that other faults were considered (i.e., open and short .I circuits). l
- d. The definition of the pass / fail acceptance criteria for each type of :
isolation device.
- e. A commitment that the isolation device complies with the ,
environmental qualifications (10CFR50.49) and with the seismic qualifications that were the basis for plant licensing. ,
- f. A description of the measures taken to protect the safety systems from electrical interference (i.e., Electrostatic Coupling, EMI, -
Common Mode and Crosstalk) that may be generated by the SPDS/RG 1.97.
- g. The information to verify that the class 1E isolation devices are powered from a Class 1E power source (s).
Response 1:
Per discussions with the staff a meeting will be held at Rancho Seco, tentatively scheduled for the week of April 6,1987, to provide a walkdown of the isolation requirements. At this meeting preliminary test results of the isolators will also be available for review by the staff. The data available will provide all the information requested by the NRC in Attachment 2 to Enclosure 2 of the February 2, 1987 letter (item la-g).
Question 2:
Provide data that the SPDS/RG 1.97 Category 1 variables will be available-following a design basis event occurrence.
Response 2:
The District will provide the documents and data currently available to show compliance with IEEE 323, 1974 and 344-1975 in April 1987. For the current ongoing modifications this documentation will be provided when it becomes available in May 1987.
Question 3:
Provide data showing that the equipment is dedicated to the SPDS/RG 1.97 and is not required for the safe shutdown of the plant. -
m
.t ,- . - - - , , - - - -y - .,-s ,.-n, , , . - e , . + - - . . _ - e4 - -- e
ATTACHMENT 2 (Cont.)
SPDS ITEMS (2/2/87)
Response 3:
There is no interface between SPDS and the control of the Reactor.
Protection System. Thus upon receipt of a trip signal, the reactor will' s be put in a cafe shutdown condition (i.e., hot shutdown) without reliance L upon SPDS . Beyond tripping of the reactor, the operator will rely on -the
. qualified (IE) SPDS to perform safety actions.
i Question 4:
Provide a block diagram of the SPDS/RG 1.97. The block diagram should be of sufficient detail to show both the signal and power Class 1E to-1 non-Class 1E interfaces and the type of device used at the interface.
Response 4:
This information will be provided as part of the April 1987 submittal.
i Lynchburg meeting Feb. 10-12, 1987 Request 1:
The staff expects the District to docket (send to the NRC) the functional specification for the upgraded SPDS for a detailed review.
4 Response 1:
The functional specification will be provided to the staff in April 1987.
Request 2:
All validation documentation needs to be docketed with the NRC.
Validation of the SPDS includes both 'the V&V performed by B&W and the I testing performed by the District.
Response 2:
The results of the V&V being performed by B&W will be complete and available to the staff in May 1987. Final testing of the system will occur prior to restart of the unit. <
ATTACHMENT 2 (Cont.)
SPDS ITEMS (2/2/87)
Request 3:
Operator training on the use of SPDS needs to be extensive.
Response 3:
The operators are receiving training on the use of the SPDS. In addition, the recent revisions to the Emergency Operating Procedures direct the operators to use SPDS as the preferred method of indicstion-Note - The remainder of the Lynchburg meeting action .<tems have been addressed in response to the questions found in the February 2,1987 letter.
Telephone Comments:
! Comment 1:
Does the SPDS have a reflash capability?
! Response 1:
No reflash capability currently exist on SPDS.
~
Comment 2:
Does the SPDS upgrade affect any of the existing Technical Specifications for interfacing instrumentation?
Response 2:
i The Technical Specification amendment which will be submitted prior to restart will not affect the District's compliance with the current version of the specifications (see Question 9.6).
t
ATTACHMENT 2 ENCLOSURE 1 COMPARISON OF SMUD SPDS V&V TO REGULATORY GUIDE 1.152 l
I Ta Daughtrey Power Computing Company February 1987 l
l
COMPARISON OF SMUD SPDS V&V TO REGULATORY GUIDE 1.152 P.agg i 1. Introduction 3
- 2. Re fe ren c e s 7 3 Computer System Requirements 9
- 4. Software Development. 11
- 5. Hardware-Software Integration 13
- 6. Computer System validation 14 f
- 7. Verification 15
- 8. Summary and Conclusions 21 Figures
- 1. Illustration of Processes Required by ANSI /IEEE-ANS-7-4.3.2-1982 4
- 2. Relation of Verification and Validation to SMUD SPDS Software Development 5 3 Discrepancy Processing 17 i
- 4. V1V Notification and Resolution Form 18 j i
Table ,
- 1. Sof tware V&V Performance Compared Against " Application Criteria" R2 2
- , - - - --- . . , , - , _n - . + , - - - . - ..--n u -
- 1. INTRODUCTION NRC Regulatory Guide 1.152, " Criteria for Programmable Digital Computer System Software in Safety-Related Systems of Nuclear j Power Plants," endorses the standard ANSI /IEEE-ANS-7-4.3.2-193- ,
as establishing acceptable methods for o designing software, !
o verifying software, ,
o implementing software, and !
o validating computer systems, f
Th is report evaluates the performance of a pa r tic ula r Babcock &
Wilcox verification and validation project against the relevant portions of that standard, the "American National Standard Application Criteria for Programmable Digital Computer Systems in Safety. Systems of Nuclear Power Generating Stations." Figure 1 from the standard is reproduced on the next page.
Babcock and Wilcox's Special Products and Integrated Field.
Services (SPIS) has developed Safety Parameter Display Systems for three dif fe ren t utilities. SPDS software has been subjected to independent verification and validation since 1982 by.
l Engineering Software Services (formerly known as Engineering Applications Programming). The range of V&V applications is summarized in the author's " Experiences in Conducting Independent Verification and Validation of Safety Parameter Display System Software" (Ref. 21, pp. 267-273).
t I
The SMUD system was first examined in November 1984, within the guidelines already established for SPDS V&V. !!o p a r t ic u la r modifications to V&V procedures were undertaken because of SMUD's 4
position on using the system to satisfy certain regulatory obligations not claimed by the other utilities receiving SPDSs.
, Although the original intent of the V&V effort was to evaluate the SMUD S P. DS after-the-fact (it had already been in s t alled in the plant), a subsequent request for software modification afforded an opportunity to perform V&V in parallel to the work. of SPIS developers. Figure 2 depicts development documents and V&V 3 activities for the SMUD project.
3 I _
9 Amancan National Standard ANSI /IEEE ANS 7-4.3.21982 Safety System Requirements for Programmable Ditigel Computer System f
l Computer Systam Requirements i Section 3 ( l l
l l
1 1
I 1
I j l
i Hardware . Integration Software ,'
I Requirements Requirements Requirements '
A I i
i L____v-_______________ h_______________p____J CV Hardw arr Developrr.en- f
- Plan Per IEEE 60 j Section 4 d Design
$ i
( Implement Section 5 / v Hardware-Software Integratioa y
Section 6 Computer System Validation NOTE: @ Represents " Verification" per Section 7 i
)
Fig.1 Illustration of Processes Required by ANSI'IEEE ANS 7 4.3.21982 4
1
l !-NUREG0696 -NdREG 0737 6
l j
SYSTEM REQUIREMENTS fMs0S!an~%Ih$
i -SPIS Working Instruction #4 t___________________J
.A SOFTWARE -d cument 51-1121938 REQUIREMENT VERIFICATION REQUIREMENTS -drawing 1134042A SPECIFICATION AA ESIGN VER ICATION -SOFTWARE p DESIGN -arawing ti37422A DESCRIPTION CODE VERIFICATI0 CODE
> LISTINGS
- awing 1137402A SOFTWARE VALIDATION EMBEDDED SOFTWARE Figure 2. Relation of Verification and Va1idation to SMUD SPDS Software n. -
- I r:. rir
Figure 1 (from 7-4.3.2) depicts seven points at which verification activities are to occur:
(1) hardware requ'irements (2) software requirements (3) hardware-software integration requirements (4) software development plan (5) software design (6) software . implementation (7) hardware-software integration The discussion in this report, following a section of-references,_
is arranged to parallel the organization of 7-4.3.2:
Computer System Requirements [Section 3]
Hardware Requirements [3 1]
Software Requirements [3 2]
Hardware-Software Integration Requirements [3 3]
Sof tware Developmen t (Section 4 ]
Software Development P'an (4.1]
Software Design [4.2]
Software Implementation [4.~3]
Hardware-Software Integration [Section 5]
-Computer System Validation [Section 6]
Ve rif ica t ion [Section 7]
Organization [7.1]
Review and Audit Procedures [7.2]
Software Test and Analysis [7 3]
A final section is then included for conclu sion- to be drawn.
The concluding table summarizes the present understanding of how thoroughly the V& V performed for the SMUD' SPDS satisfies the intent of the position expressed in Regulatory Cuide 1.152.
t 4
i 6
- - .m, -- + , , , - - - - .e--, ,, - , , - - . , ,
- 2. REFERENCES Documents Produced by V&V:
- 1) 51-1155495 "SMUD SPDS Software Verification and i I
Validation Plan" (11-28-84)
- 2) 51-1155496 "SMUD SPDS Software Verification Plan" (11-28-84) ,
i
- 3) 51-1155625 "SMUD SPDS Functional Description Review" (01-21-85, revised 05-17-85) ;
s
- 4) 51-1156616 "SMUD SPDS Design and Software Listing i Verification " (03-26-85, revised 05-17-85) '
- 5) 51-1155637 "SMUD SPDS Software Validation Plan" (01-22-85, revised 04-12-85, 04-16-85, 05-30-85, and 05-30-85)
- 6) 51-1156740 "SMUD SPDS Validation Report" (04-12-85, revised 04-16-85, 05-30-85, and 05-31-85)
- 7) 51-1158312 " Final Software Verification and -Valida tion Report for SMUD SPDS" (05-29-85) ,
Sources of System Requirements:
- 8) NUREG-0696 " Functional Criteria for Emergency Response Facilities" (February 1981)
- 9) NUREG-0737, " Clarification of TMI Action Plan Suppl. 1-No. 1 Requirements" (January 1983)
- 10) NUREG-0835 " Human Factors Acceptance Criteria for Safety Parameter Display System" (October 1981)
- 11) Regulatory " Instrumentation for Light-Water-Cooled Guide 1.97 Huclear Power Plants to Assess Plant and Environ Conditions During and Following. an Accident" (May 1983)
- 12) NSAC/39 " Verification and Validation for Safety Paraceter Display Systems" (December 1981)
1 l
i f
.14) SPIS Working " Software Development" (April 1982)
Instruction #4 SPIS Documents Subjected to V&V:
- 15) 51-1121938 "SMUD Safety Parameter Display System" (revision 1)
- 16) 1134042A " Safety Parameter Display System Function Description" (revisions 5 and 6)
- 17) 1137442A " Flowchart of SMUD SPDS Software" (revisions 5 and 6)
- 18) 1137402A "SPDS Listings and ROM Dump" (revision 5) 0*,he r Re ferences :
- 19) IEEE Std-1012-1986 "IEEE Standard for Software Verification and Validation Plans" (November 1986)
- 20) (draft) ANS-10.4 " Guidelines for the Verification and Validation of Scientific and Engineering Computer Programs for - the Nuclear Indust ry" (May 1985) i 1
- 21) Proceedings of the 1985 International Topical Meeting on l Computer Ap plica t ions for Nuclear Power Plant Operation and Control, American Nuclear Society, pp. 267-273 l
8 l
o
- 3. COMPUTER SYSTEM REQUIREMENTS The " Application Criteria" (7-4.3 2)incall thefordevelopment verification of to occur at three parallel stages requirements: hardware requirements, software requirements, and hardware-software integration requirements .
3 .1 Hardware Requirements The hardware requirements are to be verified for specification of o inp ut/ o u tp u t o control of program and data changes o initialization o diagnostics o human factors o timing and memory margins ;
o in ter ru p t s Explicit verification of hardware requirements was beyond the scope of V&V activities performed by Engineering Software Services. The V&V team did examine requirements . documentation that included hardware features, such as a watchdog timer, provision of software in ROM, and SPDS initialization upon power-up. These features were evaluated for clarity of expres s io n and consistency with system requirements, and subsequent V&V No separate activities traced and tested their implementation. for hardware.
requirements document was actually produced 3.2 Software Recuirements )
The software requirements are to be verified for specification of o inputs !
o support software o algorithms !
o data files )
o outputs o initialization o responses to system failures o operator interface l o diagnostics \
o timing o idle time and excess memory o security The V&V effort began with an investigation Thesethat assembled system the requirements d oc umen t a t ion of system requirements.
9
(
O then served as the basis for evaluation of the software requirements specification (which SPIS terms a " Functional Description" of the software).
The V&V team reviewed the documentation specifying system requirements and identified eighteen NRC-mandated items, d r a wn from References 8-11. Ten additional requirements were imposed by SPIS procedures (Ref. 12, 13), and the final seven came from guidance document on V&V from SPDS (Ref. 14). All_ aspects in the
" Application Criteria," with the exception of excess memory, were thereby addressed.
3.3 Hardware-Software Integration Recuirements The hardware-software integration requirements are to be verifi-:
for specification of o integration plan o test procedures, including acceptance criteria o system test configuration o quality assurance for integration and change control Explicit verification of hardware-software integration requirements was beyond the scope of any V&V activities performed
~,
by Engineering Software Services. Because there were no separate
-hardware requirements, integration was not treated as an identifiable activity to be verified. Software validation did depend on configuration management (change control) for the integrated sof tware and hardware.
10
F ..
. Q
- 4. SOFTWARE DEVELOPMENT 4.1 Develocuent' Plan The software-development plan is to be verified-for:
.o organization and procedures for each phase o development standards and procedures o consistence, accuracy, error tolerance, and modularity requirements and methodology for achieving o assurance of auditability and testability in all phases o quality assurance program SPIS software development was under the direction of their Working Instruction #4, " Software. Development" (revision 2, da ted 04/08/82). No project-specific software d e v elo p men t plan _w as issued for the SPDS. The V&V team included the working Instruction, as well as the SPIS Quality Assurance Plan ( d oc umen t 56-0134-02), within the system requirenents against wh ic h - the software requirements specification was verified. In partic ular ,
these items were extracted from the development- instruction and~
included in all verification phases:
o inclusion of task summaries o description of inputs rnd outputs o description of initialization features o provision for power failure o description of error checking o inclusion of interface requirements o documentation internal to flowcharts and coding o provision of information to supplement code listings o preparation and review of operations nanual.
V&V provided a thorough. evaluation of the adequacy of development procedures at SPIS.
4.2 Design Design is to,be verified for:
o consistency with development methodology o correlation with each-software requirement The ongoing attention to system requirements drawn from SPIS's development instructions addressed the-issue of consistent methodology.
V&V examination of the software design description focussed on 11
tracing each item back to the software requirements specification. Any requirements not found in the design were identified as missing-features, any requirements improperly translated in the design were identified as incorrect features, and any design items not traceable to a requirement were ident.ified as ext raneous fe atures .
4.3 Implementation Implementation is to be verified for:
o programming techniques o documentation standards o coding conventions o test requirements Again, compliance with techniques and standards was verified by continuing V&V attention to system requirements for development. The clarity of the code listings was also directly addressed.
The actual content of the coding was studied for traceability back to the design description and, in turn, to the software requirements specification.
a 4
12
r
- 5. HARDWARE-SOFTWARE INTEGRATION The software-hardware integration is to be verified for:
o identification of software and hardware used in testing o test equipment and calibration o simulation models used o test results o discrepancies and corrective action taken Verification of software-hardware integration was beyond the scope of any V&V activities performed by Engineering Software Services. Because there were no integration requirements, there was no integration process for the V&V team to verify. SPIS-conducted shop tests, whose d oc ume n t a t io n the V&V team did ' have occasion to examine, may suffice for integration testing.
13 t
- 6. COMPUTER SYSTEM VALIDATION The computer system is to be validated against:
o computer system requirements o safety system requirements The identification of system requirements, as described in Section 32, was the starting point for planning validation testing by the V&V team. A capabilities tracking matrix was established, to trace each relevant system requirement through the subsequent software development stages. In addition, a column in this matrix recorded the validation test cases to be associated with each requirement. An explicit record of validation coverage was thereby generated: at least one test case demonstrated each requirement, and tests w re clearly correlated to specific portions of each document produced in the software d e ve lo p men t process.
Validation testing was conducted on the completed SPDS by injecting simulated sensor signals in digital form only. The unavailability of Anatec equipment at the SPIS location meant additional system-oriented tests had to be run at _ Rancho Seco.
-These system validation tests were:
o STP-636, Safety Parameter Display System Acceptance Test, o STP-915, Energization and operational Test of the Anatec Data Acquisition System for IDADS, and o STP-649,. Containment High Range Radiation Monitors SPDS Display Acceptance Test.
Three sets of revised test cases were produced, to reflect changes in test predictions by the V& V team or mod i fic a tio ns to the software by developers at SPIS. All open issues were ultimately resolved by runnind a test in which predicted and observed results agreed.
An examination of the site testing plans and reports indicates the level of control and doc ucen ta t ion was consistent with that of the V&V valida tion tes t ef fort. The totality of tests satisfy the " Application Criteria" requirement for computer system validation.
14
r
- 7. VERIFICATION Verification is unders tood to consist of two' components:
(1) understandability by independent evaluator (ii) correctness of translation from previous phase.
Verification tasks are likewise characterized into two broad categories:
(i) review and audit (of design documentation.
s p e cif ica t ion s , and plans)
(ii) test and analysis The independence and competence of V&V personnel enabled them to provide the requisite evaluation of each development document and of traceability to antecedent documents. Validation' testing was conducted to supplement this evaluation, and to reveal any shortcomings in the software that- would appear only upon execution.
s 7.1 Organization Specified qualifications of V&V personnel:
o organizational independence frot those responsible for system design o technical qualifications comparable to those of the design team o communication between design and verification groups-d oc umen t ed in written reports The V&V personnel maintained organizational and technical independence, to ensure an unhindered expression of results arising from a "second opinion" evaluation of the software under-development.
The present structure of Engineering Software Services (ESS) with in the Power Computing Company means V&V personnel are not in the same operating unit of B&W as are SPIS personnel. ESS has, throughout the time of this project, been organized in a chain-of-command substantially parallel to that of SPIS.
An additional degree of fu n c t ion al independence was in t r od u c ed by a procedure within the V&V Plan that invested in the V&V team the final say in determining when a discrepancy was to be considered 15
l i
resolved. The process for identifying and resolving !
discrepancies is shown in Figure 3 A notification and .
resolution form (Figure 4) was used to convey information j formally between the V&V team and the developers at SPIS.
An addendum was produced for each verifica tion repor t, to record {
the final disposition of each discrepancy identified during evaluation of the d.lelopment documents.
l f
r i ,
O l
j 16
Figure 3. Discrepancy Processing DEVELOPED ITEM ,
V&v REVIEW V
i i
open no open IDISCREPANCY issues issues I#
l NOTIFICATION 1
l____
4 dU5 TION V.
REVISED ITEM \j
< > VERIFIED unresolved resolved ITEM l
1 Figure 4.
L
- V8V NOTIFICATION AND RESOLUTION Number: l PRIORITY:
l A. TO SPIS Initiation Oate: '
Reference Document:
Description B. FROM SPIS Response Date: ,
Proposed Fix:
Document (s) affected by modification: None -
Requirements Specifications Design Code C. BY V&V TEAM Evaluation Date:
Accepted Rejected Con rents : 1 Signature:
8/30/84 Page of 1
l i
18
_, 7 m--w- =v
The initial V&V team consisted of two senior programmers from the Engineering Software Services unit, each with over ten years' experience at B&W, One member was a former nuclear engineer who had designed and conducted the original SPDS V&V in 1982. The i; other had held lead responsibility in es t ab lish ing the.Huclear Power Division certified software configuration management system.
The 1987 software modifications were evaluated by two other senior ESS personnel, who had previously performed V&V for another utility's SPDS. One had over fifteen years' programming e xper ie n ce , and the other was involved in simulator programming including the SMUD SPDS.
Both V&V efforts were supervised by the author, who had been involved in all ESS V&V projects and in development of the IEEE V&V standard.
Communication between the V&V t+am and developers at SPIS was kept quite formal. All questieaable items were forwarded as.
discrepancy notices. Logs were kept of all telephone conversations. Even unsatisfactory test results that were l clearly due to incorrect V&V predictions were recorded and redone only after officially notifying the developers.
7.2 Review and Audit Procedures i
Technical review of development documents was performed by ESS programmers experienced in V&V and working within the established formal procedures.
In addition to specific format and content for each type of development document, V&V r eviewe rs evaluated all documents for:
o correctness, j o completeness, i
o consistency, o und e rs tanda bilit y (lack of ambiguities),
o traceability to previous doc umen ts ,
o feasibility, and o testability.
7.3 Software Test and Analysis As required by the " Application Criteria," the software validation testing was planned, executed, and analyzed by personnel independent of the software development effort, namely the V&V team. The software validation test plan was released as a formal document, and was revised to reflect modifications made before and during conduct of the testing. The test plan was not 19
a e
subjected to formal verification itself, but th e V&V supervisor reviewed and approved each revision of the plan. Software developers were also designated in the V&V plan to perform a review and offer technical comments before execution of any testing.
Each round of testing was performed to planned completien, discrepancy notifications f o r wa rd ed , and a formal test report issued. Any items that needed retesting or whose p r e d ic t ed results had to be modified were then included in the next revision of the test plan, r
Y P
O 3
e l
l 20
'4 9
- 8.
SUMMARY
AND CONCLUSIONS A major claim for the value of Regulatory Guide 1.152 is that
" errors detected during the design phase through the verification process will be far less expensive than if they were not detected until the operation phase [Section 2.1.1]." One data point from the SPDS V& V efforts indicates a verification-driven software change may be at least fif ty times less expensive than a field change after installation (see Ref. 21).
Furthermore, it appears the V&V methods applied were effective in providing "early warnings" to software developers. Nearly 85% of all discrepancy notifications that led to changes in development documents were issued before the embedded software underwent validation testing. (A number of the test-discovered discrepancies, moreover, could have only been uncovered by real-time exercise of the SPDS.) Similarly, fewer than 20% of all changes made in response to V&V involved revision of the coding itself. This represented a true benefit because it is far more difficult to find and fix errors in coding, and it is also less certain that unintended side-effects have not also been introduced.
The performance of V&V contributed to the quality of the final SPDS. On the one hand, specific errors and functional shortcomings were identified and corrected. But, on the other hand, V&V offered a positive benefit even when not finding errors. The comprehensiveness of the independent review provided by V&V enhanced confidence in the correctness of the software.
The conduct of verification and validation also reinforced efforts to communicate (document) information on the development of the software. Clearly, achieving high quality in useful documentation would have been more dif fic u lt without as immediate and responsive an audience as the V&V team.
Table 1 summarizes the extent to which Engineering Software Services V &.V of the SMUD SPDS software satisfies the relevant sections of ANSI /IEEE-ANS-7-4 3.2-1982, and thus Regulatory Guide 1.152, 21
o e
e' Table 1.
Sof tware V&V Performance Compared Against " Applica tion Criteria" V&V Task in 7-4.3.2 Degree. Type of Coverage
[section shown]
hardware requirements verification partial, indirect
[3.1]
software requirements verification full, direct
[3.2] ;
integration requirements verification not applicable i
[3.3]
sof tware development plan verifica tion full, indirect l
[4.1]
software design ve.-i fic a t io n full, direct
[4.2] "
software implementation verification full, direct
[4 3] I hardware-software integration verifica tion not applicable i
[5]
I h
computer system validation partial, direct [
[6] (remainder in j site testing) -
5 B'
9 22