ML16061A468
Text
I n v e. n s*.~ s*
i f'"'! v ' e. f'"'! S *.~ s*
1M Triconex PG&E PROCESS PROTECTION SYSTEM REPLACEMENT Purchase Order No.:
3500880752 Project Sales Order:
993754 Author:
PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT v &V IMPLEMENTATION PHASE
SUMMARY
REPORT PPSII-IV Document No. 993754-12-862 Revision 1 January 15,2016 Name M la Montellano Reviewers:
Son Phan Michael Sh u A
roval:
Kevin Vu
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
2 of 31 Date:
01/15/2016 Document Change History Revision Date Change Author 0
11/19/14 Initial Issuance M. Montellano 1
01/15/16 Minor editorial and formatting corrections.
-Table of Contents - fixed the section heading number
-Section 1.0 - Fixed the page break to move 1.0 to page 3.
-Remove duplicate Section 6.1.1.2 and changed to 6.1.2
-Table 6-4b title - corrected typo, subscript (2) to (1)
-Corrected Section number 6.2.2.3 to 6.2.2.2 M. Montellano
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
3 of 31 Date:
01/15/2016 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 4 2.0 SCOPE.......................................................................................................... 4 3.0 TASKS PERFORMED............................................................................... 4 3.1 PHASE INPUTS.......................................................................................................... 4 3.2 PHASE TASKS........................................................................................................... 5 4.0 TASK RESULTS......................................................................................... 7 4.1 RESULTS
SUMMARY
.............................................................................................. 7 4.2 TASK AND PHASE ITERATION............................................................................ 9 4.3 PHASE OUTPUTS................................................................................................... 10 5.0 IMPLEMENTATION PHASE ANOMALIES....................................... 11 6.0 SOFTWARE QUALITY........................................................................... 18 6.1 SOFTWARE QUALITY METRICS...................................................................... 18 6.2 V&V EFFECTIVENESS METRICS...................................................................... 20 6.3 QUALITY EVALUATION
SUMMARY
............................................................... 22 7.0 SOFTWARE SAFETY METRICS.......................................................... 24 8.0 TECHNICAL AND MANAGEMENT RISKS....................................... 25 9.0 RECOMMENDATIONS.......................................................................... 25
10.0 REFERENCES
........................................................................................... 26 10.1 INDUSTRY DOCUMENTS..................................................................................... 26 10.2 INVENSYS TRICONEX DOCUMENTS............................................................... 26 11.0 APPENDICES............................................................................................ 27 12.0 ATTACHMENTS...................................................................................... 31
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
4 of 31 Date:
01/15/2016 1.0 PURPOSE The purpose of this V&V Implementation Phase Summary Report is to summarize the results of the V&V tasks performed for the implementation phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting implementation specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
2.0 SCOPE The PPSII-IV V&V implementation phase activities include the verification software coding and testing.
The objectives of the V&V efforts are to ensure that the design requirements are clear, consistent, complete, readable, testable, and correctly translate and trace to the design.
Per Project Conditional Release (CR No. 993754-07), the V&V implementation phase activities also include the following tasks for Protection Set II-IV:
- 1. Generate, verify, and issue Hardware Validation Test (HVT) Procedure.
- 2. Perform HVT procedure dry-run and HVT.
- 3. Generate and verify the Factory Acceptance Test (FAT) Procedure.
- 4. Perform the FAT procedure dry-run.
- 5. Continue development of downstream Test Phase documentation.
3.0 TASKS PERFORMED 3.1 PHASE INPUTS The table below contains the implementation phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] which lists document revisions, deletions or additions are considered as customer inputs during the entire implementation phase and all project iteration phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report represents the project inputs applicable only for PPSII-IV V&V implementation phase tasks.
Table 3-1 Implementation Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 17 07/10/2014 993754-1-803 Master Configuration List (MCL) 54 09/15/2014
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
5 of 31 Date:
01/15/2016 993754-1-804 Project Traceability Matrix (PTM) 6 04/11/2014 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 3 06/19/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 3 06/19/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 3 06/19/2014 993754-1-802 Software Verification and Validation Plan (SVVP) 6 08/28/2014 993754-1-915 Design Phase Safety (Criticality/Hazard/Risk/Interface)
Analysis 7
08/27/2014 993754-11-809 Software Requirements Specification (SRS) 6 06/26/2014 993754-12-810 Protection Set II-IV Software Design Description (SDD) 2 04/02/2014 993754-11-917 Hardware Design Description (HDD) 3 06/11/2014 993754-1-811 Failure Modes and Effects Analysis (FMEA) 1 02/24/2014 993754-1-907 Software Development Plan Coding Guidelines 4
05/23/2014 993754-1-817 Maximum TSAP Scan Time 1
04/09/2012 993754-1-868 Software Verification Test Plan (SVTP) 2 05/20/2014 993754-1-869 Software Verification Test Specification (SVTS) 4 08/21/2014 993754-1-813 Validation Test Specification (VTS) 3 06/11/2014 993754-12-806 Protection Set II Input / Output (I/O) List 0
12/19/2013 993754-13-806 Protection Set III Input / Output (I/O) List 0
12/19/2013 993754-14-806 Protection Set IV Input / Output (I/O) List 0
12/19/2013 993754-12-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-12-700 PGE DCPP PPS.PT2 0.0 8/29/2014 993754-13-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-12-700 PGE DCPP PPS.PT2 0.0 8/29/2014 993754-14-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-12-700 PGE DCPP PPS.PT2 0.0 8/29/2014 3.2 PHASE TASKS The following are phase tasks associated with the PPSII-IV implementation phase. Official document reviews are demonstrated in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
6 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
7 of 31 Date:
01/15/2016 4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The table below contains a summary of PPSII-IV implementation phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the implementation phase documents during the implementation phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-1 as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
8 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
9 of 31 Date:
01/15/2016 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2 outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is no iteration since the start implementation phase.
Table 4-2: Implementation Phase Task and Phase Iteration Iteration #
Date Reason Tasks Performed None N/A N/A N/A
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
10 of 31 Date:
01/15/2016 4.3 PHASE OUTPUTS The following table lists the final output documents for the Implementation Phase.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
11 of 31 Date:
01/15/2016 5.0 IMPLEMENTATION PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). For test procedure discrepancies, an Interim Change Notice (ICN) will be generated and the appropriate level of review/approval as defined in PPMs 6.0 and 10.0. Corrective Action Reports (CARs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. The summary of anomalies or deficiencies that were identified during the completion of implementation phase tasks are listed in Table 5-1, 5-2, and 5-3.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
12 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
13 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
14 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
15 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
16 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
17 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
18 of 31 Date:
01/15/2016 6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the implementation phase.
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the PTM and SDD during IV&V review
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
19 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
20 of 31 Date:
01/15/2016 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
6.2.1 V&V Review Completeness
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
21 of 31 Date:
01/15/2016 6.2.2 V&V Reporting Accuracy
%Accuracy = Number of Software Product Defects (Actual) / Number of Reported Product Defects (Potential Defects) x 100 6.2.2.1PTM V&V Reporting Accuracy 6.2.2.2 SDD V&V Reporting Accuracy
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
22 of 31 Date:
01/15/2016 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9 below lists the quality evaluation summary for the implementation phase.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
23 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
24 of 31 Date:
01/15/2016 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the implementation phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
25 of 31 Date:
01/15/2016 8.0 TECHNICAL AND MANAGEMENT RISKS All of the PPSII-IV implementation phase tasks are complete, most of the deficiencies are resolved except for SIDRs 917, 920, 923 (which required FAT execution for closure) 959 (replacement part is already ordered), and output documents are issued. Issued CARs 2507 and 2923 does not pose any technical or management risk to PPSI-IV project. CAR 2507 corrective actions implementation has been completed and CAR 2923 action plan will be implemented during the FAT.
Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set II-IV, Revisions 1Aand 2A based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following are satisfied:
- 1. Functionality (e.g., algorithms, state/mode definitions, input/output validation, exception handling, reporting and logging);
- 2. Process definition and scheduling;
- 3. Hardware, software, and user interface descriptions;
- 4. Performance criteria (e.g., timing, sizing, speed, capacity, accuracy, precision, safety, and security);
- 5. Critical configuration data;
- 6. System, device, and software control (e.g., initialization, transaction and state monitoring, and self-testing).
All documents released by the Nuclear Delivery in the implementation phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP)
[Ref. 10.2.8]
There are no technical issues associated with the implementation phase outputs or the phase completion.
9.0 RECOMMENDATIONS Nuclear IV&V recommends exiting the implementation phase.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
26 of 31 Date:
01/15/2016
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Software Requirements Specification (SRS), 993754-11-809 10.2.10 Validation Test Plan (VTP), 993754-1-813 10.2.11 Protection Set II-IV Software Design Description (SDD), 993754-12-810 10.2.12 Hardware Design Description (HDD), 993754-11-917 10.2.13 Project Traceability Matrix (PTM), 993754-1-804 10.2.14 Software Verification Test Specification (SVTS), 993754-1-869 10.2.15 Software Verification Test Plan (SVTP), 993754-1-861 10.2.16 Validation Test Specification (VTS), 993754-1-812 10.2.17 Software Requirements Specification (SRS), 993754-11-809 10.2.18 Technical Requirements List (TRL), 993754-1-801 10.2.19 Software Safety Plan (SSP), 993754-1-911 10.2.20 Software Configuration Management Plan (SCMP), 993754-1-909
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
27 of 31 Date:
01/15/2016 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: PTM Software Quality Metrics Data Appendix 2: SDD Software Quality Metrics Data
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
28 of 31 Date:
01/15/2016 Appendix 1: PTM Software Quality Metrics Data 11.1a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.2b PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
29 of 31 Date:
01/15/2016 APPENDIX 2: SDD SOFTWARE QUALITY METRICS DATA 11.3a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.3b SDD Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: SDD OFE(1) = # of Pages Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
30 of 31 Date:
01/15/2016 11.3c SDD V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
31 of 31 Date:
01/15/2016 12.0 ATTACHMENTS None.
I n v e. n s*.~ s*
i f'"'! v ' e. f'"'! S *.~ s*
1M Triconex PG&E PROCESS PROTECTION SYSTEM REPLACEMENT Purchase Order No.:
3500880752 Project Sales Order:
993754 Author:
PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT v &V IMPLEMENTATION PHASE
SUMMARY
REPORT PPSII-IV Document No. 993754-12-862 Revision 1 January 15,2016 Name M la Montellano Reviewers:
Son Phan Michael Sh u A
roval:
Kevin Vu
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
2 of 31 Date:
01/15/2016 Document Change History Revision Date Change Author 0
11/19/14 Initial Issuance M. Montellano 1
01/15/16 Minor editorial and formatting corrections.
-Table of Contents - fixed the section heading number
-Section 1.0 - Fixed the page break to move 1.0 to page 3.
-Remove duplicate Section 6.1.1.2 and changed to 6.1.2
-Table 6-4b title - corrected typo, subscript (2) to (1)
-Corrected Section number 6.2.2.3 to 6.2.2.2 M. Montellano
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
3 of 31 Date:
01/15/2016 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 4 2.0 SCOPE.......................................................................................................... 4 3.0 TASKS PERFORMED............................................................................... 4 3.1 PHASE INPUTS.......................................................................................................... 4 3.2 PHASE TASKS........................................................................................................... 5 4.0 TASK RESULTS......................................................................................... 7 4.1 RESULTS
SUMMARY
.............................................................................................. 7 4.2 TASK AND PHASE ITERATION............................................................................ 9 4.3 PHASE OUTPUTS................................................................................................... 10 5.0 IMPLEMENTATION PHASE ANOMALIES....................................... 11 6.0 SOFTWARE QUALITY........................................................................... 18 6.1 SOFTWARE QUALITY METRICS...................................................................... 18 6.2 V&V EFFECTIVENESS METRICS...................................................................... 20 6.3 QUALITY EVALUATION
SUMMARY
............................................................... 22 7.0 SOFTWARE SAFETY METRICS.......................................................... 24 8.0 TECHNICAL AND MANAGEMENT RISKS....................................... 25 9.0 RECOMMENDATIONS.......................................................................... 25
10.0 REFERENCES
........................................................................................... 26 10.1 INDUSTRY DOCUMENTS..................................................................................... 26 10.2 INVENSYS TRICONEX DOCUMENTS............................................................... 26 11.0 APPENDICES............................................................................................ 27 12.0 ATTACHMENTS...................................................................................... 31
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
4 of 31 Date:
01/15/2016 1.0 PURPOSE The purpose of this V&V Implementation Phase Summary Report is to summarize the results of the V&V tasks performed for the implementation phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting implementation specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
2.0 SCOPE The PPSII-IV V&V implementation phase activities include the verification software coding and testing.
The objectives of the V&V efforts are to ensure that the design requirements are clear, consistent, complete, readable, testable, and correctly translate and trace to the design.
Per Project Conditional Release (CR No. 993754-07), the V&V implementation phase activities also include the following tasks for Protection Set II-IV:
- 1. Generate, verify, and issue Hardware Validation Test (HVT) Procedure.
- 2. Perform HVT procedure dry-run and HVT.
- 3. Generate and verify the Factory Acceptance Test (FAT) Procedure.
- 4. Perform the FAT procedure dry-run.
- 5. Continue development of downstream Test Phase documentation.
3.0 TASKS PERFORMED 3.1 PHASE INPUTS The table below contains the implementation phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] which lists document revisions, deletions or additions are considered as customer inputs during the entire implementation phase and all project iteration phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report represents the project inputs applicable only for PPSII-IV V&V implementation phase tasks.
Table 3-1 Implementation Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 17 07/10/2014 993754-1-803 Master Configuration List (MCL) 54 09/15/2014
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
5 of 31 Date:
01/15/2016 993754-1-804 Project Traceability Matrix (PTM) 6 04/11/2014 993754-12-807 Protection Set II Hardware Requirements Specification (HRS) 3 06/19/2014 993754-13-807 Protection Set III Hardware Requirements Specification (HRS) 3 06/19/2014 993754-14-807 Protection Set IV Hardware Requirements Specification (HRS) 3 06/19/2014 993754-1-802 Software Verification and Validation Plan (SVVP) 6 08/28/2014 993754-1-915 Design Phase Safety (Criticality/Hazard/Risk/Interface)
Analysis 7
08/27/2014 993754-11-809 Software Requirements Specification (SRS) 6 06/26/2014 993754-12-810 Protection Set II-IV Software Design Description (SDD) 2 04/02/2014 993754-11-917 Hardware Design Description (HDD) 3 06/11/2014 993754-1-811 Failure Modes and Effects Analysis (FMEA) 1 02/24/2014 993754-1-907 Software Development Plan Coding Guidelines 4
05/23/2014 993754-1-817 Maximum TSAP Scan Time 1
04/09/2012 993754-1-868 Software Verification Test Plan (SVTP) 2 05/20/2014 993754-1-869 Software Verification Test Specification (SVTS) 4 08/21/2014 993754-1-813 Validation Test Specification (VTS) 3 06/11/2014 993754-12-806 Protection Set II Input / Output (I/O) List 0
12/19/2013 993754-13-806 Protection Set III Input / Output (I/O) List 0
12/19/2013 993754-14-806 Protection Set IV Input / Output (I/O) List 0
12/19/2013 993754-12-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-12-700 PGE DCPP PPS.PT2 0.0 8/29/2014 993754-13-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-12-700 PGE DCPP PPS.PT2 0.0 8/29/2014 993754-14-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-12-700 PGE DCPP PPS.PT2 0.0 8/29/2014 3.2 PHASE TASKS The following are phase tasks associated with the PPSII-IV implementation phase. Official document reviews are demonstrated in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
6 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
7 of 31 Date:
01/15/2016 4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The table below contains a summary of PPSII-IV implementation phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the implementation phase documents during the implementation phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-1 as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
8 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
9 of 31 Date:
01/15/2016 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2 outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is no iteration since the start implementation phase.
Table 4-2: Implementation Phase Task and Phase Iteration Iteration #
Date Reason Tasks Performed None N/A N/A N/A
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
10 of 31 Date:
01/15/2016 4.3 PHASE OUTPUTS The following table lists the final output documents for the Implementation Phase.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
11 of 31 Date:
01/15/2016 5.0 IMPLEMENTATION PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). For test procedure discrepancies, an Interim Change Notice (ICN) will be generated and the appropriate level of review/approval as defined in PPMs 6.0 and 10.0. Corrective Action Reports (CARs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. The summary of anomalies or deficiencies that were identified during the completion of implementation phase tasks are listed in Table 5-1, 5-2, and 5-3.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
12 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
13 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
14 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
15 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
16 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
17 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
18 of 31 Date:
01/15/2016 6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the implementation phase.
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the PTM and SDD during IV&V review
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
19 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
20 of 31 Date:
01/15/2016 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
6.2.1 V&V Review Completeness
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
21 of 31 Date:
01/15/2016 6.2.2 V&V Reporting Accuracy
%Accuracy = Number of Software Product Defects (Actual) / Number of Reported Product Defects (Potential Defects) x 100 6.2.2.1PTM V&V Reporting Accuracy 6.2.2.2 SDD V&V Reporting Accuracy
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
22 of 31 Date:
01/15/2016 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9 below lists the quality evaluation summary for the implementation phase.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
23 of 31 Date:
01/15/2016
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
24 of 31 Date:
01/15/2016 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the implementation phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
25 of 31 Date:
01/15/2016 8.0 TECHNICAL AND MANAGEMENT RISKS All of the PPSII-IV implementation phase tasks are complete, most of the deficiencies are resolved except for SIDRs 917, 920, 923 (which required FAT execution for closure) 959 (replacement part is already ordered), and output documents are issued. Issued CARs 2507 and 2923 does not pose any technical or management risk to PPSI-IV project. CAR 2507 corrective actions implementation has been completed and CAR 2923 action plan will be implemented during the FAT.
Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set II-IV, Revisions 1Aand 2A based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following are satisfied:
- 1. Functionality (e.g., algorithms, state/mode definitions, input/output validation, exception handling, reporting and logging);
- 2. Process definition and scheduling;
- 3. Hardware, software, and user interface descriptions;
- 4. Performance criteria (e.g., timing, sizing, speed, capacity, accuracy, precision, safety, and security);
- 5. Critical configuration data;
- 6. System, device, and software control (e.g., initialization, transaction and state monitoring, and self-testing).
All documents released by the Nuclear Delivery in the implementation phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP)
[Ref. 10.2.8]
There are no technical issues associated with the implementation phase outputs or the phase completion.
9.0 RECOMMENDATIONS Nuclear IV&V recommends exiting the implementation phase.
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
26 of 31 Date:
01/15/2016
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Software Requirements Specification (SRS), 993754-11-809 10.2.10 Validation Test Plan (VTP), 993754-1-813 10.2.11 Protection Set II-IV Software Design Description (SDD), 993754-12-810 10.2.12 Hardware Design Description (HDD), 993754-11-917 10.2.13 Project Traceability Matrix (PTM), 993754-1-804 10.2.14 Software Verification Test Specification (SVTS), 993754-1-869 10.2.15 Software Verification Test Plan (SVTP), 993754-1-861 10.2.16 Validation Test Specification (VTS), 993754-1-812 10.2.17 Software Requirements Specification (SRS), 993754-11-809 10.2.18 Technical Requirements List (TRL), 993754-1-801 10.2.19 Software Safety Plan (SSP), 993754-1-911 10.2.20 Software Configuration Management Plan (SCMP), 993754-1-909
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
27 of 31 Date:
01/15/2016 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: PTM Software Quality Metrics Data Appendix 2: SDD Software Quality Metrics Data
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
28 of 31 Date:
01/15/2016 Appendix 1: PTM Software Quality Metrics Data 11.1a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.2b PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
29 of 31 Date:
01/15/2016 APPENDIX 2: SDD SOFTWARE QUALITY METRICS DATA 11.3a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.3b SDD Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: SDD OFE(1) = # of Pages Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
30 of 31 Date:
01/15/2016 11.3c SDD V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-12-862
Title:
V&V Implementation Phase Summary Report PPSII-IV Revision:
1 Page:
31 of 31 Date:
01/15/2016 12.0 ATTACHMENTS None.