ML16061A464
Text
.
n v e. n s*.~ s*
TM i r) v ' e. r) S *.!:l s
- Operations Management Triconex Project:
Purchase Order No.:
Project Sales Order:
PG&E PROCESS PROTECTION SYSTEM REPLACEMENT 3500897372 993754 PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT v &V DESIGN PHASE
SUMMARY
REPORT PPSI Document No. 993754-11-861 Revision 3 January 15,2016 Author:
Reviewers:
A roval:
Kevin Vu
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
2 of 44 Date:
01/15/2016 Document Change History Revision Date Change Author 0
11/14/13 Initial issue S. Phan 1
04/08/14 Updated to incorporate Rev 9 Design phase V&V activities L. Nguyen Revise MCL revision to 38 in Table 4-31a to includes this report.
L. Nguyen 2
08/07/14 Cover Page -Changed document number from 993754 861 to 993754-11-861 and added PPSI in title per SVVP Rev 5. Added PPSI in Sections 2.0, 3.1, 3.2, and 4.1.
Fixed formatting.
M.
Montellano 3
01/15/16 Minor formatting and editorial corrections.
-Table of Contents - Fixed the section numbers
-Added PPSI to header title
-Added Section number 6.1.2.2
-Section 6.2.1 -changed the font to bold.
M.
Montellano
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
3 of 44 Date:
01/15/2016 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 5 2.0 SCOPE.......................................................................................................... 6 3.0 TASKS PERFORMED............................................................................... 7 3.1 PHASE INPUTS........................................................................................................... 7 3.2 PHASE TASKS (RE-ENTER OF DESIGN PHASE).................................................. 8 4.0 TASK RESULTS....................................................................................... 10 4.1 RESULTS
SUMMARY
.............................................................................................. 10 4.2 TASK AND PHASE ITERATION............................................................................. 12 4.3 PHASE OUTPUTS..................................................................................................... 13 5.0 DESIGN PHASE ANOMALIES.............................................................. 14 6.0 SOFTWARE QUALITY........................................................................... 16 6.1 SOFTWARE QUALITY METRICS.......................................................................... 16 6.1.1 Defect Count 16 6.1.1.1 SDD Defect Count........................................................................................... 16 6.1.1.2 PTM Defect Count.......................................................................................... 17 6.1.2 Defect Density 18 6.1.2.1 SDD Defect Density........................................................................................ 18 6.1.2.2 PTM Defect Density........................................................................................ 20 6.2 V&V EFFECTIVENESS METRICS.......................................................................... 22 6.2.1 V&V Review Completeness 22 6.2.2 V&V Reporting Accuracy 23 6.2.2.1 SDD V&V Reporting Accuracy...................................................................... 23 6.2.2.2 PTM V&V Reporting Accuracy...................................................................... 24 6.3 QUALITY EVALUATION
SUMMARY
................................................................... 25 7.0 SOFTWARE SAFETY METRICS.......................................................... 28 8.0 TECHNICAL AND MANAGEMENT RISKS (RE-ENTER OF DESIGN PHASE)...................................................................................... 30 9.0 RECOMMENDATIONS ( RE-ENTER OF DESIGN PHASE)........... 31
10.0 REFERENCES
........................................................................................... 32
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
4 of 44 Date:
01/15/2016 10.1 INDUSTRY DOCUMENTS....................................................................................... 32 10.2 INVENSYS TRICONEX DOCUMENTS.................................................................. 32 11.0 APPENDICES............................................................................................ 33 Appendix 1: SDD Software Quality Metrics Data 33 Appendix 2: PTM Software Quality Metrics Data 33 12.0 ATTACHMENTS...................................................................................... 44
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
5 of 44 Date:
01/15/2016 1.0 PURPOSE The purpose of this V&V Design Phase Summary Report is to summarize the results of the V&V tasks performed for the design phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting requirements specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
6 of 44 Date:
01/15/2016 2.0 SCOPE The PPSI V&V design phase activities include architectural and detailed design for the implementation of the software and the plan for testing the software to ensure correct implementation of the requirements. The objectives of the V&V efforts are to ensure that the design is correct, consistent, complete, accurate, readable, testable, traceable and unambiguous and traceable to the requirements.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
7 of 44 Date:
01/15/2016 3.0 TASKS PERFORMED 3.1 PHASE INPUTS The tables below contain the design phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] lists document revisions, deletions or additions, which are considered as customer inputs during all project development phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report summarizes the project inputs applicable for PPSI V&V design phase (Table 3-1a) and re-enter of PPSI design phase tasks on 02/25/14 (Table 3-11a)
Table 3-1a. Design Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 9 03/07/2013 3500897372 Purchase Order #3500897372, dated 12/20/2011 6
12/20/2011 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification (FRS) 5 04/17/2012 N/A PG&E PPS Replacement Interface Requirements Specification (IRS) 6 06/07/2012 10115-J-NPG PPS Controller Transfer Functions Design Input Specification 2
04/19/2012 08-0015-D Series PG&E PPS Function Block Diagram (FBD) 3 to 6 03/21/2012 993754-1-813 Validation Test Plan 2
12/18/2012 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface) Analysis 1
10/24/2012 993754-1-803 Master Configuration List (MCL) 21 04/03/2013 993754-1-804 Project Traceability Matrix (PTM) 2 06/03/2012 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 2 10/07/2013 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 1 09/12/2012 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 1 09/12/2012 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 1 09/12/2012 993754-11-809 Protection Set I Software Requirements Specification (SRS) 3 10/25/2013 993754-12-809 Protection Set II Software Requirements Specification (SRS) 2 10/17/2012 993754-13-809 Protection Set III Software Requirements Specification (SRS) 2 10/17/2012 993754-14-809 Protection Set IV Software Requirements Specification (SRS) 2 10/17/2012 993754-1-860 V&V Requirements Phase Summary Report 1
10/30/2012 Table 3-11a. Re-enter of Design Phase Inputs Document ID Document Name Revision Date Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 11 01/29/2014 3500897372 Purchase Order #3500897372, dated 12/20/2011 6
12/20/2011
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
8 of 44 Date:
01/15/2016 CWA 3500897372 Contract Work Authorization - Change Order Rev9 0
12/18/2013 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification (FRS) 9 11/15/2013 N/A PG&E PPS Replacement Interface Requirements Specification (IRS) 9 11/15/2013 10115-J-NPG PPS Controller Transfer Functions Design Input Specification 4
11/15/2013 993754-35R Project Letter (DCPP operational data and initial tunable parameter settings.
N/A 12/13/2012 993754-64R Project Letter (PG&E Project Letter PPS-IOM-13-021-TCM Configuration Setting Design Input)
N/A 9/25/2013 993754-78R Project Letter (CD-ER 993754-24; Restore Streaming Factor Calculation.)
N/A 12/9/2013 993754-79R Project Letter (PPS Set I, II, III & IV Rev 9 Updated FAT Drawings - Revised Letter)
A 12/19/13 993754-82R Project Letter (CD-ER 993754-25; Interface Requirement Specification; updated Flux values and incorrect paragraph numbering.)
N/A 12/19/13 993754-91R Project Letter (CD-ER 993754-26; Suppress Reactor Coolant Pressure Alarm output when the channel is out of service).
N/A 1/21/2014 993754-92R Project Letter (DIT 68001801-1-0, PPS TSAP Constraints.and Input Scaling Adjustment Factors - Initial Values)
N/A 1/23/2014 08-0015-D Series PG&E PPS Function Block Diagram (FBD) 3 to 6 06/08/2012 993754-1-813 Validation Test Plan 3
01/28/2014 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface) Analysis 3
01/28/2014 993754-1-803 Master Configuration List (MCL) 36 01/30/2014 993754-1-804 Project Traceability Matrix (PTM) 4 01/28/2014 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 3 01/16/2014 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 2 01/16/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 2 01/16/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 2 01/16/2014 993754-11-809 Software Requirements Specification (SRS) 4 01/21/2014 993754-1-860 V&V Requirements Phase Summary Report 2
01/30/2014 3.2 PHASE TASKS (RE-ENTER OF DESIGN PHASE)
The following are phase tasks associated with the PPSI design phase. Document review results are summarized in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
9 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
10 of 44 Date:
01/15/2016 4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The tables below contain a summary of PPSI design phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the design phase documents during the design phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-11a as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
11 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
12 of 44 Date:
01/15/2016 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2a and 4-21a outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is (2) iteration since last restart of the project design phase last February 2013.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
13 of 44 Date:
01/15/2016 4.3 PHASE OUTPUTS The following table lists the final output documents for the Design Phase.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
14 of 44 Date:
01/15/2016 5.0 DESIGN PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR).
Action Request Reports (ARRs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of Design phase tasks are listed in Table 5-1a.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
15 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
16 of 44 Date:
01/15/2016 6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the Design phase.
Defect density is the defect count divided by the number of opportunities for error (OFE).
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
Acceptance Criteria: IV&V recommends proceeding to the next phase if no technical defects remain at the end of the current phase.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the SDD and PTM IV&V review
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
17 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
18 of 44 Date:
01/15/2016 6.1.2 Defect Density Defect Density = Defects (SDD) / Opportunities for Error (OFE)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
19 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
20 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
21 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
22 of 44 Date:
01/15/2016 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
Acceptance Criteria: IV&V recommends proceeding to the next phase only if the review percentage is 100%.
6.2.1 V&V Review Completeness
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
23 of 44 Date:
01/15/2016 6.2.2 V&V Reporting Accuracy
%Accuracy = # of Software Product Defects (Actual) x100%
- of Reported Product Defects (Potential Defects) 6.2.2.1 SDD V&V Reporting Accuracy
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
24 of 44 Date:
01/15/2016 6.2.2.2 PTM V&V Reporting Accuracy
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
25 of 44 Date:
01/15/2016 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9a below lists the quality evaluation summary for the Design phase.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
26 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
27 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
28 of 44 Date:
01/15/2016 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the design phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
29 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
30 of 44 Date:
01/15/2016 8.0 TECHNICAL AND MANAGEMENT RISKS (RE-ENTER OF DESIGN PHASE)
All of the design phase tasks are complete, deficiencies are resolved, and output documents are issued.
Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set I Rev 1 based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following Requirement-Phase open items are satisfied and closed:
- 1) Performance criteria for precision (e.g., precision of decimal places in floating-point calculations).
- 2) System, device, and software control that specify initialization of the Protection Sets critical points (e.g., setting setpoints and tuning constants to initial values).
- 3) Logic, computational, and interface precision that specify truncation and rounding in floating-point computations and data sent to external applications.
All documents released by the Nuclear Delivery in the design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8]
There are no technical issues associated with the design phase outputs or the phase completion.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
31 of 44 Date:
01/15/2016 9.0 RECOMMENDATIONS ( RE-ENTER OF DESIGN PHASE)
Software Quality Metrics - There are no technical defects remain unresolved V&V Effective Metrics - Review percentage is 100%
Software Safety Metrics - There are no technical defects remain unresolved.
Nuclear IV&V recommends exiting the Design phase.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
32 of 44 Date:
01/15/2016
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Protection Set I Software Requirements Specification (SRS), 993754-11-809 10.2.10 Protection Set II Software Requirements Specification (SRS), 993754-12-809 10.2.11 Protection Set III Software Requirements Specification (SRS), 993754-13-809 10.2.12 Protection Set IV Software Requirements Specification (SRS), 993754-14-809 10.2.13 Master Configuration List (MCL), 993754-1-803 10.2.14 Validation Test Plan (VTP), 993754-1-813 10.2.15 Protection Set I Software Design Description (SDD), 993754-11-810 10.2.16 Hardware Design Description (HDD), all four Protection Sets, 993754-11-917 10.2.17 Project Traceability Matrix (PTM), 993754-13-804 10.2.18 Software Verification Test Specification (SVTS), 993754-13-869 10.2.19 Validation Test Specification (VTS), 993754-13-813
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
33 of 44 Date:
01/15/2016 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: SDD Software Quality Metrics Data Appendix 2: PTM Software Quality Metrics Data
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
34 of 44 Date:
01/15/2016 APPENDIX 1: SDD SOFTWARE QUALITY METRICS DATA 11.1a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
35 of 44 Date:
01/15/2016 11.1b SDD Defect Density Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Design Elements Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
36 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
37 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
38 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
39 of 44 Date:
01/15/2016 APPENDIX 2: PTM SOFTWARE QUALITY METRICS DATA 11.2a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
40 of 44 Date:
01/15/2016 11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Traces Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
41 of 44 Date:
01/15/2016 11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: PTM OFE(2) = # of Cells Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
42 of 44 Date:
01/15/2016 11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
43 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
44 of 44 Date:
01/15/2016 12.0 ATTACHMENTS None
n v e. n s*.~ s*
TM i r) v ' e. r) S *.!:l s
- Operations Management Triconex Project:
Purchase Order No.:
Project Sales Order:
PG&E PROCESS PROTECTION SYSTEM REPLACEMENT 3500897372 993754 PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT v &V DESIGN PHASE
SUMMARY
REPORT PPSI Document No. 993754-11-861 Revision 3 January 15,2016 Author:
Reviewers:
A roval:
Kevin Vu
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
2 of 44 Date:
01/15/2016 Document Change History Revision Date Change Author 0
11/14/13 Initial issue S. Phan 1
04/08/14 Updated to incorporate Rev 9 Design phase V&V activities L. Nguyen Revise MCL revision to 38 in Table 4-31a to includes this report.
L. Nguyen 2
08/07/14 Cover Page -Changed document number from 993754 861 to 993754-11-861 and added PPSI in title per SVVP Rev 5. Added PPSI in Sections 2.0, 3.1, 3.2, and 4.1.
Fixed formatting.
M.
Montellano 3
01/15/16 Minor formatting and editorial corrections.
-Table of Contents - Fixed the section numbers
-Added PPSI to header title
-Added Section number 6.1.2.2
-Section 6.2.1 -changed the font to bold.
M.
Montellano
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
3 of 44 Date:
01/15/2016 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 5 2.0 SCOPE.......................................................................................................... 6 3.0 TASKS PERFORMED............................................................................... 7 3.1 PHASE INPUTS........................................................................................................... 7 3.2 PHASE TASKS (RE-ENTER OF DESIGN PHASE).................................................. 8 4.0 TASK RESULTS....................................................................................... 10 4.1 RESULTS
SUMMARY
.............................................................................................. 10 4.2 TASK AND PHASE ITERATION............................................................................. 12 4.3 PHASE OUTPUTS..................................................................................................... 13 5.0 DESIGN PHASE ANOMALIES.............................................................. 14 6.0 SOFTWARE QUALITY........................................................................... 16 6.1 SOFTWARE QUALITY METRICS.......................................................................... 16 6.1.1 Defect Count 16 6.1.1.1 SDD Defect Count........................................................................................... 16 6.1.1.2 PTM Defect Count.......................................................................................... 17 6.1.2 Defect Density 18 6.1.2.1 SDD Defect Density........................................................................................ 18 6.1.2.2 PTM Defect Density........................................................................................ 20 6.2 V&V EFFECTIVENESS METRICS.......................................................................... 22 6.2.1 V&V Review Completeness 22 6.2.2 V&V Reporting Accuracy 23 6.2.2.1 SDD V&V Reporting Accuracy...................................................................... 23 6.2.2.2 PTM V&V Reporting Accuracy...................................................................... 24 6.3 QUALITY EVALUATION
SUMMARY
................................................................... 25 7.0 SOFTWARE SAFETY METRICS.......................................................... 28 8.0 TECHNICAL AND MANAGEMENT RISKS (RE-ENTER OF DESIGN PHASE)...................................................................................... 30 9.0 RECOMMENDATIONS ( RE-ENTER OF DESIGN PHASE)........... 31
10.0 REFERENCES
........................................................................................... 32
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
4 of 44 Date:
01/15/2016 10.1 INDUSTRY DOCUMENTS....................................................................................... 32 10.2 INVENSYS TRICONEX DOCUMENTS.................................................................. 32 11.0 APPENDICES............................................................................................ 33 Appendix 1: SDD Software Quality Metrics Data 33 Appendix 2: PTM Software Quality Metrics Data 33 12.0 ATTACHMENTS...................................................................................... 44
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
5 of 44 Date:
01/15/2016 1.0 PURPOSE The purpose of this V&V Design Phase Summary Report is to summarize the results of the V&V tasks performed for the design phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting requirements specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
6 of 44 Date:
01/15/2016 2.0 SCOPE The PPSI V&V design phase activities include architectural and detailed design for the implementation of the software and the plan for testing the software to ensure correct implementation of the requirements. The objectives of the V&V efforts are to ensure that the design is correct, consistent, complete, accurate, readable, testable, traceable and unambiguous and traceable to the requirements.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
7 of 44 Date:
01/15/2016 3.0 TASKS PERFORMED 3.1 PHASE INPUTS The tables below contain the design phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] lists document revisions, deletions or additions, which are considered as customer inputs during all project development phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report summarizes the project inputs applicable for PPSI V&V design phase (Table 3-1a) and re-enter of PPSI design phase tasks on 02/25/14 (Table 3-11a)
Table 3-1a. Design Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 9 03/07/2013 3500897372 Purchase Order #3500897372, dated 12/20/2011 6
12/20/2011 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification (FRS) 5 04/17/2012 N/A PG&E PPS Replacement Interface Requirements Specification (IRS) 6 06/07/2012 10115-J-NPG PPS Controller Transfer Functions Design Input Specification 2
04/19/2012 08-0015-D Series PG&E PPS Function Block Diagram (FBD) 3 to 6 03/21/2012 993754-1-813 Validation Test Plan 2
12/18/2012 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface) Analysis 1
10/24/2012 993754-1-803 Master Configuration List (MCL) 21 04/03/2013 993754-1-804 Project Traceability Matrix (PTM) 2 06/03/2012 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 2 10/07/2013 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 1 09/12/2012 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 1 09/12/2012 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 1 09/12/2012 993754-11-809 Protection Set I Software Requirements Specification (SRS) 3 10/25/2013 993754-12-809 Protection Set II Software Requirements Specification (SRS) 2 10/17/2012 993754-13-809 Protection Set III Software Requirements Specification (SRS) 2 10/17/2012 993754-14-809 Protection Set IV Software Requirements Specification (SRS) 2 10/17/2012 993754-1-860 V&V Requirements Phase Summary Report 1
10/30/2012 Table 3-11a. Re-enter of Design Phase Inputs Document ID Document Name Revision Date Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 11 01/29/2014 3500897372 Purchase Order #3500897372, dated 12/20/2011 6
12/20/2011
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
8 of 44 Date:
01/15/2016 CWA 3500897372 Contract Work Authorization - Change Order Rev9 0
12/18/2013 08-0015-SP-001 PG&E PPS Replacement Functional Requirements Specification (FRS) 9 11/15/2013 N/A PG&E PPS Replacement Interface Requirements Specification (IRS) 9 11/15/2013 10115-J-NPG PPS Controller Transfer Functions Design Input Specification 4
11/15/2013 993754-35R Project Letter (DCPP operational data and initial tunable parameter settings.
N/A 12/13/2012 993754-64R Project Letter (PG&E Project Letter PPS-IOM-13-021-TCM Configuration Setting Design Input)
N/A 9/25/2013 993754-78R Project Letter (CD-ER 993754-24; Restore Streaming Factor Calculation.)
N/A 12/9/2013 993754-79R Project Letter (PPS Set I, II, III & IV Rev 9 Updated FAT Drawings - Revised Letter)
A 12/19/13 993754-82R Project Letter (CD-ER 993754-25; Interface Requirement Specification; updated Flux values and incorrect paragraph numbering.)
N/A 12/19/13 993754-91R Project Letter (CD-ER 993754-26; Suppress Reactor Coolant Pressure Alarm output when the channel is out of service).
N/A 1/21/2014 993754-92R Project Letter (DIT 68001801-1-0, PPS TSAP Constraints.and Input Scaling Adjustment Factors - Initial Values)
N/A 1/23/2014 08-0015-D Series PG&E PPS Function Block Diagram (FBD) 3 to 6 06/08/2012 993754-1-813 Validation Test Plan 3
01/28/2014 993754-1-915 Safety (Requirements Criticality/hazard/Risk/Interface) Analysis 3
01/28/2014 993754-1-803 Master Configuration List (MCL) 36 01/30/2014 993754-1-804 Project Traceability Matrix (PTM) 4 01/28/2014 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 3 01/16/2014 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 2 01/16/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 2 01/16/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 2 01/16/2014 993754-11-809 Software Requirements Specification (SRS) 4 01/21/2014 993754-1-860 V&V Requirements Phase Summary Report 2
01/30/2014 3.2 PHASE TASKS (RE-ENTER OF DESIGN PHASE)
The following are phase tasks associated with the PPSI design phase. Document review results are summarized in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
9 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
10 of 44 Date:
01/15/2016 4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The tables below contain a summary of PPSI design phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the design phase documents during the design phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-11a as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
11 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
12 of 44 Date:
01/15/2016 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2a and 4-21a outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is (2) iteration since last restart of the project design phase last February 2013.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
13 of 44 Date:
01/15/2016 4.3 PHASE OUTPUTS The following table lists the final output documents for the Design Phase.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
14 of 44 Date:
01/15/2016 5.0 DESIGN PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR).
Action Request Reports (ARRs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of Design phase tasks are listed in Table 5-1a.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
15 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
16 of 44 Date:
01/15/2016 6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the Design phase.
Defect density is the defect count divided by the number of opportunities for error (OFE).
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
Acceptance Criteria: IV&V recommends proceeding to the next phase if no technical defects remain at the end of the current phase.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the SDD and PTM IV&V review
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
17 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
18 of 44 Date:
01/15/2016 6.1.2 Defect Density Defect Density = Defects (SDD) / Opportunities for Error (OFE)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
19 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
20 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
21 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
22 of 44 Date:
01/15/2016 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
Acceptance Criteria: IV&V recommends proceeding to the next phase only if the review percentage is 100%.
6.2.1 V&V Review Completeness
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
23 of 44 Date:
01/15/2016 6.2.2 V&V Reporting Accuracy
%Accuracy = # of Software Product Defects (Actual) x100%
- of Reported Product Defects (Potential Defects) 6.2.2.1 SDD V&V Reporting Accuracy
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
24 of 44 Date:
01/15/2016 6.2.2.2 PTM V&V Reporting Accuracy
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
25 of 44 Date:
01/15/2016 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9a below lists the quality evaluation summary for the Design phase.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
26 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
27 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
28 of 44 Date:
01/15/2016 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the design phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
29 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
30 of 44 Date:
01/15/2016 8.0 TECHNICAL AND MANAGEMENT RISKS (RE-ENTER OF DESIGN PHASE)
All of the design phase tasks are complete, deficiencies are resolved, and output documents are issued.
Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set I Rev 1 based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following Requirement-Phase open items are satisfied and closed:
- 1) Performance criteria for precision (e.g., precision of decimal places in floating-point calculations).
- 2) System, device, and software control that specify initialization of the Protection Sets critical points (e.g., setting setpoints and tuning constants to initial values).
- 3) Logic, computational, and interface precision that specify truncation and rounding in floating-point computations and data sent to external applications.
All documents released by the Nuclear Delivery in the design phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP) [Ref. 10.2.8]
There are no technical issues associated with the design phase outputs or the phase completion.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
31 of 44 Date:
01/15/2016 9.0 RECOMMENDATIONS ( RE-ENTER OF DESIGN PHASE)
Software Quality Metrics - There are no technical defects remain unresolved V&V Effective Metrics - Review percentage is 100%
Software Safety Metrics - There are no technical defects remain unresolved.
Nuclear IV&V recommends exiting the Design phase.
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
32 of 44 Date:
01/15/2016
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Protection Set I Software Requirements Specification (SRS), 993754-11-809 10.2.10 Protection Set II Software Requirements Specification (SRS), 993754-12-809 10.2.11 Protection Set III Software Requirements Specification (SRS), 993754-13-809 10.2.12 Protection Set IV Software Requirements Specification (SRS), 993754-14-809 10.2.13 Master Configuration List (MCL), 993754-1-803 10.2.14 Validation Test Plan (VTP), 993754-1-813 10.2.15 Protection Set I Software Design Description (SDD), 993754-11-810 10.2.16 Hardware Design Description (HDD), all four Protection Sets, 993754-11-917 10.2.17 Project Traceability Matrix (PTM), 993754-13-804 10.2.18 Software Verification Test Specification (SVTS), 993754-13-869 10.2.19 Validation Test Specification (VTS), 993754-13-813
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
33 of 44 Date:
01/15/2016 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: SDD Software Quality Metrics Data Appendix 2: PTM Software Quality Metrics Data
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
34 of 44 Date:
01/15/2016 APPENDIX 1: SDD SOFTWARE QUALITY METRICS DATA 11.1a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
35 of 44 Date:
01/15/2016 11.1b SDD Defect Density Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Design Elements Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
36 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
37 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
38 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
39 of 44 Date:
01/15/2016 APPENDIX 2: PTM SOFTWARE QUALITY METRICS DATA 11.2a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
40 of 44 Date:
01/15/2016 11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Traces Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
41 of 44 Date:
01/15/2016 11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: PTM OFE(2) = # of Cells Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
42 of 44 Date:
01/15/2016 11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
43 of 44 Date:
01/15/2016
Document:
993754-11-861
Title:
V&V Design Phase Summary Report PPSI Revision:
3 Page:
44 of 44 Date:
01/15/2016 12.0 ATTACHMENTS None