ML16061A467
Text
I n v e. n s *.~ s' i 1"1 V*e.1"15*.!::l 5' TM Triconex Project:
PG&E PROCESS PROTECTION SYSTEM REPLACEMENT Purchase Order No.:
3500880752 Project Sales Order:
993754 Author:
PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT V&V DESIGN PHASE
SUMMARY
REPORT PPSII-IV Document No. 993754-12-861 Revision 0 August 28, 2014 Name M la Montellano Reviewers:
Son Ph an Michael Sh u A
royal:
Kevin Vu
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
2 of 29 Date:
08/28/14 Document Change History Revision Date Change Author 0
08/28/14 Initial Issuance M. Montellano
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
3 of 29 Date:
08/28/14 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 5 2.0 SCOPE.......................................................................................................... 5 3.0 TASKS PERFORMED............................................................................... 5 3.1 PHASE INPUTS.......................................................................................................... 5 3.2 PHASE TASKS........................................................................................................... 7 4.0 TASK RESULTS......................................................................................... 8 4.1 RESULTS
SUMMARY
.............................................................................................. 8 4.2 TASK AND PHASE ITERATION............................................................................ 9 4.3 PHASE OUTPUTS..................................................................................................... 9 5.0 DESIGN PHASE ANOMALIES.............................................................. 10 6.0 SOFTWARE QUALITY........................................................................... 10 6.1 SOFTWARE QUALITY METRICS...................................................................... 11 6.1.1 Defect Count11 6.1.1.1 PTM Defect Count.......................................................................................... 11 6.1.1.2 SDD Defect Count........................................................................................... 11 6.1.2 Defect Density. 12 6.1.2.1 PTM Defect Density........................................................................................ 12 6.1.2.2 SDD Defect Density........................................................................................ 13 6.2 V&V EFFECTIVENESS METRICS...................................................................... 15 6.2.1 V&V Review Completeness15 6.2.2 V&V Reporting Accuracy... 16 6.3 QUALITY EVALUATION
SUMMARY
............................................................... 17 7.0 SOFTWARE SAFETY METRICS.......................................................... 19 8.0 TECHNICAL AND MANAGEMENT RISKS....................................... 20 9.0 RECOMMENDATIONS.......................................................................... 20
10.0 REFERENCES
........................................................................................... 21 11.0 APPENDICES............................................................................................ 22 Appendix 1: PTM Software Quality Metrics Data. 22 Appendix 2: SDD Software Quality Metrics Data. 22
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
4 of 29 Date:
08/28/14 12.0 ATTACHMENTS...................................................................................... 29
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
5 of 29 Date:
08/28/14 1.0 PURPOSE The purpose of this V&V Design Phase Summary Report is to summarize the results of the V&V tasks performed for the design phase of the application life cycle for the Nuclear Safety-Related (Class 1E)
Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting implementation specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
2.0 SCOPE The PPSII-IV V&V design phase activities architectural and detailed design for the implementation of the software and the plan for testing the software to ensure correct implementation of the requirements.
The objectives of the V&V efforts are to ensure that the design is correct, consistent, complete, accurate, readable, testable, traceable and unambiguous and traceable to the requirements.
3.0 TASKS PERFORMED 3.1 PHASE INPUTS The table below contains the design phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] which lists document revisions, deletions or additions are considered as customer inputs during the entire Design phase and all project iteration phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL.
This report represents the project inputs applicable only for PPSII-IV V&V Design phase tasks.
Table 3-1 Design Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 14 06/11/2014 3500897372 Purchase Order #3500897372, dated 12/20/2011 6
12/20/2011 CWA 3500897372 Contract Work Authorization - Change Order Rev9 0
12/18/2013 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification (FRS) 9 11/15/2013 N/A PG&E PPS Replacement Interface Requirements Specification (IRS) 9 11/15/2013 10115-J-NPG PPS Controller Transfer Functions Design Input Specification 4
11/15/2013 993754-35R Project Letter (DCPP operational data and initial tunable parameter settings.
N/A 12/13/2012 993754-64R Project Letter (PG&E Project Letter PPS-IOM-13-021-TCM Configuration Setting Design Input)
N/A 9/25/2013 993754-78R Project Letter (CD-ER 993754-24; Restore Streaming Factor Calculation.)
N/A 12/9/2013
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
6 of 29 Date:
08/28/14 993754-79R Project Letter (PPS Set I, II, III & IV Rev 9 Updated FAT Drawings - Revised Letter)
A 12/19/13 993754-82R Project Letter (CD-ER 993754-25; Interface Requirement Specification; updated Flux values and incorrect paragraph numbering.)
N/A 12/19/13 993754-91R Project Letter (CD-ER 993754-26; Suppress Reactor Coolant Pressure Alarm output when the channel is out of service).
N/A 1/21/2014 993754-92R Project Letter (DIT 68001801-1-0, PPS TSAP Constraints.and Input Scaling Adjustment Factors - Initial Values)
N/A 1/23/2014 993754-101R PG&E Project Letter PPS-IOM-14-020-CDER 27, 28,29 Approvals N/A 3/3/2014 993754-103R PG&E Project Letter PPS-IOM-14-023-PLS Value Update N/A 3/1/2014 993754-130R PG&E Project Letter PPS-IOM-14-051-Credible Input Scenarios N/A 5/8/2014 993754-02R Project Letter, CD-ER 993754-01, PG&E Purchase Order 3500897372 Revision 2 N/A 6/3/2011 993754-13R Project Letter, CD-ER 993754-03, Function Requirements Specification Revision 4 N/A 11/10/2011 993754-43R Project Letter, CD-ER 993754-08, PG&E Functional Requirement Specification, 08-0015-SP-001, Rev. 5 N/A 7/3/2013 993754-45R Project Letter, CD-ER 993754-10; Panel Arrangement, Rack 3 (RNP1C) Non-Safety, Dwg. No. 926240-3, Rev. 0 N/A 7/10/2013 993754-49R Project Letter, CD-ER 993754-11, PG&E Drawing Set 926240-4 Substitutions for the FAT configuration for cabinet and power line filter N/A 8/7/2014 993754-58R Project Letter, CD-ER 993754-15, PG&E Functional Requirements Specification, 08-0015-SP-001, Rev. 5 (Non-Safety Trip Switches)
N/A 9/6/2014 993754-58R Project Letter, CD-ER 993754-16, PG&E Functional Requirements Specification, 08-0015-SP-001, Rev. 5 (Internal Rack Failure Mode)
N/A 9/6/2014 993754-63R Project Letter, CD-ER 993754-17, Panel Arrangement, Rack 8 (RNP2C) NON-SAFETY, Dwg. No. 926240-8 Rev 0.
N/A 9/25/2013 993754-63R Project Letter, CD-ER 993754-18, Panel Arrangement, Rack 9 (RNP2D) Nuclear Safety Related, Dwg. No. 926240-9 Rev 0.
N/A 9/25/2013 993754-125R Project Letter, CD-ER 993754-30, PG&E Functional Requirements Specification, 08-0015-SP-001, Rev, 9 (Electromagentic Compatibility - Emissions)
N/A 4/24/2014 08-0015 Series PG&E FUNCTION BLOCK DIAGRAMS Rev 5 to 8 Refer to TRL for complete listing 926240-xx, and 926240-xx-FAT Series PG&E DRAWINGS, Protection I, II, III, IV Rev 0 to 2 Refer to TRL for complete listing DC-663229-4768 Precautions, Limitations and Setpoints 68 N/A 993754-1-813 Validation Test Plan 3
01/28/2014 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface)
Analysis 5
05/20/2014 993754-1-803 Master Configuration List (MCL) 44 06/12/2014 993754-1-804 Project Traceability Matrix (PTM) 8 06/11/2014
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
7 of 29 Date:
08/28/14 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 2 01/16/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 2 01/16/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 2 01/16/2014 993754-11-809 Software Requirements Specification (SRS) 5 06/10/2014 993754-1-860 V&V Requirements Phase Summary Report 2
01/30/2014 3.2 PHASE TASKS The following are phase tasks associated with the PPSII-IV Design phase. Official document reviews are demonstrated in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
8 of 29 Date:
08/28/14
- 10. Verify Protection Sets II-IV System Validation Test Specification and validate all the details for the system level test approach, features to be tested, test requirements, acceptance criteria to address system validation and factory acceptance testing. Update the VTS if necessary.
- 11. Generate the V&V Design Phase Summary Report (This Report) which covers all the PPSII-IV V&V design activities including any reported deficiencies. This report is updated during re-enter of design phase tasks.
4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The table below contains a summary of PPSII-IV Design phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the Design phase documents during the design phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-1 as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
9 of 29 Date:
08/28/14 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2 outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is no iteration since the start PPSII-IV design phase on 6/13/2014.
Table 4-2: Design Phase Task and Phase Iteration Iteration #
Date Reason Tasks Performed None N/A N/A N/A 4.3 PHASE OUTPUTS The following table lists the final output documents for the Design Phase.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
10 of 29 Date:
08/28/14 5.0 DESIGN PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). For test procedure discrepancies, an Interim Change Notice (ICN) will be generated and the appropriate level of review/approval as defined in PPMs 6.0 and 10.0. Corrective Action Reports (CARs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of Design phase tasks are listed in Table 5-1.
6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the Design phase.
Defect density is the defect count divided by the number of opportunities for error (OFE).
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
11 of 29 Date:
08/28/14 V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the PTM and SDD during IV&V review
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
12 of 29 Date:
08/28/14 6.1.2 Defect Density Defect Density = Defects (PTM, SDD / Opportunities for Error (OFE)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
13 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
14 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
15 of 29 Date:
08/28/14 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
6.2.1 V&V Review Completeness
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
16 of 29 Date:
08/28/14 6.2.2 V&V Reporting Accuracy
%Accuracy = Number of Software Product Defects (Actual) / Number of Reported Product Defects (Potential Defects) x 100
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
17 of 29 Date:
08/28/14 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9 below lists the quality evaluation summary for the Design phase.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
18 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
19 of 29 Date:
08/28/14 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the Design phase are assessed in the metrics below.
The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
20 of 29 Date:
08/28/14 8.0 TECHNICAL AND MANAGEMENT RISKS All of the PPS II-IVdesign phase tasks are complete, deficiencies are resolved, and output documents are issued.
Nuclear IV&V reviewed the Software Design Description (SDD) for PPSII-IV, Revision 0, based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following are satisfied:
- 1) Performance criteria for precision (e.g., precision of decimal places in floating-point calculations).
- 2) System, device, and software control that specify initialization of the Protection Sets critical points (e.g., setting setpoints and tuning constants to initial values).
- 3) Logic, computational, and interface precision that specify truncation and rounding in floating-point computations and data sent to external applications.
All documents released by the Nuclear Delivery in the design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8]
There are no technical issues associated with the design phase outputs or the phase completion.
All documents released by the Nuclear Delivery in the Design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8]
There are no technical issues associated with the Design phase outputs or the phase completion.
9.0 RECOMMENDATIONS Software Quality Metrics - There are no technical defects remain unresolved V&V Effective Metrics - Review percentage is 100%
Software Safety Metrics - There are no technical defects remain unresolved.
Nuclear IV&V recommends exiting the Design phase.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
21 of 29 Date:
08/28/14
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Software Requirements Specification (SRS), 993754-11-809 10.2.10 Master Configuration List (MCL), 993754-1-803 10.2.11 Validation Test Plan (VTP), 993754-1-813 10.2.12 Protection Set II-IV Software Design Description (SDD), 993754-12-810 10.2.13 Hardware Design Description (HDD), 993754-11-917 10.2.14 Project Traceability Matrix (PTM), 993754-1-804 10.2.15 Software Verification Test Specification (SVTS), 993754-1-869 10.2.16 Software Verification Test Plan (SVTP), 993754-1-861 10.2.17 Validation Test Specification (VTS), 993754-1-813
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
22 of 29 Date:
08/28/14 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: PTM Software Quality Metrics Data Appendix 2: SDD Software Quality Metrics Data
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
23 of 29 Date:
08/28/14 Appendix 1: PTM Software Quality Metrics Data 11.1a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Traces Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
24 of 29 Date:
08/28/14 11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: PTM OFE(2) = # of Cells Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2) 11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
25 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
26 of 29 Date:
08/28/14 APPENDIX 2: SDD SOFTWARE QUALITY METRICS DATA 11.3a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.3b SDD Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Requirements Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
27 of 29 Date:
08/28/14 Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1) 11.3c SDD Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: SDD OFE(2) = # of Pages Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
28 of 29 Date:
08/28/14 11.3d SDD V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
29 of 29 Date:
08/28/14 12.0 ATTACHMENTS None.
I n v e. n s *.~ s' i 1"1 V*e.1"15*.!::l 5' TM Triconex Project:
PG&E PROCESS PROTECTION SYSTEM REPLACEMENT Purchase Order No.:
3500880752 Project Sales Order:
993754 Author:
PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT V&V DESIGN PHASE
SUMMARY
REPORT PPSII-IV Document No. 993754-12-861 Revision 0 August 28, 2014 Name M la Montellano Reviewers:
Son Ph an Michael Sh u A
royal:
Kevin Vu
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
2 of 29 Date:
08/28/14 Document Change History Revision Date Change Author 0
08/28/14 Initial Issuance M. Montellano
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
3 of 29 Date:
08/28/14 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 5 2.0 SCOPE.......................................................................................................... 5 3.0 TASKS PERFORMED............................................................................... 5 3.1 PHASE INPUTS.......................................................................................................... 5 3.2 PHASE TASKS........................................................................................................... 7 4.0 TASK RESULTS......................................................................................... 8 4.1 RESULTS
SUMMARY
.............................................................................................. 8 4.2 TASK AND PHASE ITERATION............................................................................ 9 4.3 PHASE OUTPUTS..................................................................................................... 9 5.0 DESIGN PHASE ANOMALIES.............................................................. 10 6.0 SOFTWARE QUALITY........................................................................... 10 6.1 SOFTWARE QUALITY METRICS...................................................................... 11 6.1.1 Defect Count11 6.1.1.1 PTM Defect Count.......................................................................................... 11 6.1.1.2 SDD Defect Count........................................................................................... 11 6.1.2 Defect Density. 12 6.1.2.1 PTM Defect Density........................................................................................ 12 6.1.2.2 SDD Defect Density........................................................................................ 13 6.2 V&V EFFECTIVENESS METRICS...................................................................... 15 6.2.1 V&V Review Completeness15 6.2.2 V&V Reporting Accuracy... 16 6.3 QUALITY EVALUATION
SUMMARY
............................................................... 17 7.0 SOFTWARE SAFETY METRICS.......................................................... 19 8.0 TECHNICAL AND MANAGEMENT RISKS....................................... 20 9.0 RECOMMENDATIONS.......................................................................... 20
10.0 REFERENCES
........................................................................................... 21 11.0 APPENDICES............................................................................................ 22 Appendix 1: PTM Software Quality Metrics Data. 22 Appendix 2: SDD Software Quality Metrics Data. 22
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
4 of 29 Date:
08/28/14 12.0 ATTACHMENTS...................................................................................... 29
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
5 of 29 Date:
08/28/14 1.0 PURPOSE The purpose of this V&V Design Phase Summary Report is to summarize the results of the V&V tasks performed for the design phase of the application life cycle for the Nuclear Safety-Related (Class 1E)
Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting implementation specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
2.0 SCOPE The PPSII-IV V&V design phase activities architectural and detailed design for the implementation of the software and the plan for testing the software to ensure correct implementation of the requirements.
The objectives of the V&V efforts are to ensure that the design is correct, consistent, complete, accurate, readable, testable, traceable and unambiguous and traceable to the requirements.
3.0 TASKS PERFORMED 3.1 PHASE INPUTS The table below contains the design phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] which lists document revisions, deletions or additions are considered as customer inputs during the entire Design phase and all project iteration phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL.
This report represents the project inputs applicable only for PPSII-IV V&V Design phase tasks.
Table 3-1 Design Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 14 06/11/2014 3500897372 Purchase Order #3500897372, dated 12/20/2011 6
12/20/2011 CWA 3500897372 Contract Work Authorization - Change Order Rev9 0
12/18/2013 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification (FRS) 9 11/15/2013 N/A PG&E PPS Replacement Interface Requirements Specification (IRS) 9 11/15/2013 10115-J-NPG PPS Controller Transfer Functions Design Input Specification 4
11/15/2013 993754-35R Project Letter (DCPP operational data and initial tunable parameter settings.
N/A 12/13/2012 993754-64R Project Letter (PG&E Project Letter PPS-IOM-13-021-TCM Configuration Setting Design Input)
N/A 9/25/2013 993754-78R Project Letter (CD-ER 993754-24; Restore Streaming Factor Calculation.)
N/A 12/9/2013
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
6 of 29 Date:
08/28/14 993754-79R Project Letter (PPS Set I, II, III & IV Rev 9 Updated FAT Drawings - Revised Letter)
A 12/19/13 993754-82R Project Letter (CD-ER 993754-25; Interface Requirement Specification; updated Flux values and incorrect paragraph numbering.)
N/A 12/19/13 993754-91R Project Letter (CD-ER 993754-26; Suppress Reactor Coolant Pressure Alarm output when the channel is out of service).
N/A 1/21/2014 993754-92R Project Letter (DIT 68001801-1-0, PPS TSAP Constraints.and Input Scaling Adjustment Factors - Initial Values)
N/A 1/23/2014 993754-101R PG&E Project Letter PPS-IOM-14-020-CDER 27, 28,29 Approvals N/A 3/3/2014 993754-103R PG&E Project Letter PPS-IOM-14-023-PLS Value Update N/A 3/1/2014 993754-130R PG&E Project Letter PPS-IOM-14-051-Credible Input Scenarios N/A 5/8/2014 993754-02R Project Letter, CD-ER 993754-01, PG&E Purchase Order 3500897372 Revision 2 N/A 6/3/2011 993754-13R Project Letter, CD-ER 993754-03, Function Requirements Specification Revision 4 N/A 11/10/2011 993754-43R Project Letter, CD-ER 993754-08, PG&E Functional Requirement Specification, 08-0015-SP-001, Rev. 5 N/A 7/3/2013 993754-45R Project Letter, CD-ER 993754-10; Panel Arrangement, Rack 3 (RNP1C) Non-Safety, Dwg. No. 926240-3, Rev. 0 N/A 7/10/2013 993754-49R Project Letter, CD-ER 993754-11, PG&E Drawing Set 926240-4 Substitutions for the FAT configuration for cabinet and power line filter N/A 8/7/2014 993754-58R Project Letter, CD-ER 993754-15, PG&E Functional Requirements Specification, 08-0015-SP-001, Rev. 5 (Non-Safety Trip Switches)
N/A 9/6/2014 993754-58R Project Letter, CD-ER 993754-16, PG&E Functional Requirements Specification, 08-0015-SP-001, Rev. 5 (Internal Rack Failure Mode)
N/A 9/6/2014 993754-63R Project Letter, CD-ER 993754-17, Panel Arrangement, Rack 8 (RNP2C) NON-SAFETY, Dwg. No. 926240-8 Rev 0.
N/A 9/25/2013 993754-63R Project Letter, CD-ER 993754-18, Panel Arrangement, Rack 9 (RNP2D) Nuclear Safety Related, Dwg. No. 926240-9 Rev 0.
N/A 9/25/2013 993754-125R Project Letter, CD-ER 993754-30, PG&E Functional Requirements Specification, 08-0015-SP-001, Rev, 9 (Electromagentic Compatibility - Emissions)
N/A 4/24/2014 08-0015 Series PG&E FUNCTION BLOCK DIAGRAMS Rev 5 to 8 Refer to TRL for complete listing 926240-xx, and 926240-xx-FAT Series PG&E DRAWINGS, Protection I, II, III, IV Rev 0 to 2 Refer to TRL for complete listing DC-663229-4768 Precautions, Limitations and Setpoints 68 N/A 993754-1-813 Validation Test Plan 3
01/28/2014 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface)
Analysis 5
05/20/2014 993754-1-803 Master Configuration List (MCL) 44 06/12/2014 993754-1-804 Project Traceability Matrix (PTM) 8 06/11/2014
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
7 of 29 Date:
08/28/14 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 2 01/16/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 2 01/16/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 2 01/16/2014 993754-11-809 Software Requirements Specification (SRS) 5 06/10/2014 993754-1-860 V&V Requirements Phase Summary Report 2
01/30/2014 3.2 PHASE TASKS The following are phase tasks associated with the PPSII-IV Design phase. Official document reviews are demonstrated in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
8 of 29 Date:
08/28/14
- 10. Verify Protection Sets II-IV System Validation Test Specification and validate all the details for the system level test approach, features to be tested, test requirements, acceptance criteria to address system validation and factory acceptance testing. Update the VTS if necessary.
- 11. Generate the V&V Design Phase Summary Report (This Report) which covers all the PPSII-IV V&V design activities including any reported deficiencies. This report is updated during re-enter of design phase tasks.
4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The table below contains a summary of PPSII-IV Design phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the Design phase documents during the design phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-1 as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
9 of 29 Date:
08/28/14 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2 outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is no iteration since the start PPSII-IV design phase on 6/13/2014.
Table 4-2: Design Phase Task and Phase Iteration Iteration #
Date Reason Tasks Performed None N/A N/A N/A 4.3 PHASE OUTPUTS The following table lists the final output documents for the Design Phase.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
10 of 29 Date:
08/28/14 5.0 DESIGN PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). For test procedure discrepancies, an Interim Change Notice (ICN) will be generated and the appropriate level of review/approval as defined in PPMs 6.0 and 10.0. Corrective Action Reports (CARs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of Design phase tasks are listed in Table 5-1.
6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the Design phase.
Defect density is the defect count divided by the number of opportunities for error (OFE).
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
11 of 29 Date:
08/28/14 V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the PTM and SDD during IV&V review
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
12 of 29 Date:
08/28/14 6.1.2 Defect Density Defect Density = Defects (PTM, SDD / Opportunities for Error (OFE)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
13 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
14 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
15 of 29 Date:
08/28/14 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
6.2.1 V&V Review Completeness
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
16 of 29 Date:
08/28/14 6.2.2 V&V Reporting Accuracy
%Accuracy = Number of Software Product Defects (Actual) / Number of Reported Product Defects (Potential Defects) x 100
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
17 of 29 Date:
08/28/14 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9 below lists the quality evaluation summary for the Design phase.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
18 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
19 of 29 Date:
08/28/14 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the Design phase are assessed in the metrics below.
The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
20 of 29 Date:
08/28/14 8.0 TECHNICAL AND MANAGEMENT RISKS All of the PPS II-IVdesign phase tasks are complete, deficiencies are resolved, and output documents are issued.
Nuclear IV&V reviewed the Software Design Description (SDD) for PPSII-IV, Revision 0, based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following are satisfied:
- 1) Performance criteria for precision (e.g., precision of decimal places in floating-point calculations).
- 2) System, device, and software control that specify initialization of the Protection Sets critical points (e.g., setting setpoints and tuning constants to initial values).
- 3) Logic, computational, and interface precision that specify truncation and rounding in floating-point computations and data sent to external applications.
All documents released by the Nuclear Delivery in the design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8]
There are no technical issues associated with the design phase outputs or the phase completion.
All documents released by the Nuclear Delivery in the Design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8]
There are no technical issues associated with the Design phase outputs or the phase completion.
9.0 RECOMMENDATIONS Software Quality Metrics - There are no technical defects remain unresolved V&V Effective Metrics - Review percentage is 100%
Software Safety Metrics - There are no technical defects remain unresolved.
Nuclear IV&V recommends exiting the Design phase.
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
21 of 29 Date:
08/28/14
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Software Requirements Specification (SRS), 993754-11-809 10.2.10 Master Configuration List (MCL), 993754-1-803 10.2.11 Validation Test Plan (VTP), 993754-1-813 10.2.12 Protection Set II-IV Software Design Description (SDD), 993754-12-810 10.2.13 Hardware Design Description (HDD), 993754-11-917 10.2.14 Project Traceability Matrix (PTM), 993754-1-804 10.2.15 Software Verification Test Specification (SVTS), 993754-1-869 10.2.16 Software Verification Test Plan (SVTP), 993754-1-861 10.2.17 Validation Test Specification (VTS), 993754-1-813
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
22 of 29 Date:
08/28/14 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: PTM Software Quality Metrics Data Appendix 2: SDD Software Quality Metrics Data
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
23 of 29 Date:
08/28/14 Appendix 1: PTM Software Quality Metrics Data 11.1a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Traces Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
24 of 29 Date:
08/28/14 11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: PTM OFE(2) = # of Cells Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2) 11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
25 of 29 Date:
08/28/14
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
26 of 29 Date:
08/28/14 APPENDIX 2: SDD SOFTWARE QUALITY METRICS DATA 11.3a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.3b SDD Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Requirements Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
27 of 29 Date:
08/28/14 Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1) 11.3c SDD Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: SDD OFE(2) = # of Pages Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
28 of 29 Date:
08/28/14 11.3d SDD V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-861
Title:
V&V Design Phase Summary Report PPSII-IV Revision:
0 Page:
29 of 29 Date:
08/28/14 12.0 ATTACHMENTS None.