ML16061A465
Text
I n v e. n s*.~ s*
i r] v' e. r] S *.!::l s*
Triconex Project:
PG&E PROCESS PROTECTION SYSTEM REPLACEMENT Purchase Order No.:
3500880752 Project Sales Order:
993754 PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT V &V IMPLEMENTATION PHASE
SUMMARY
REPORT PPSI Document No. 993754-11-862 Revision 1 August 07, 2014
~
Name Signature
\\
Title Author:
Myla Montellano
.~~,;y.-~
IV & V Engineer Reviewers:
Hoan Nguyen V.:r~ -------
IV & V Engineer Michael Shyu
~.~ 9-L-Project QA Engineer Approval:
Kevin Vu
/~~--
IV & V Manager
~
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
2 of 55 Date:
08/07/14 Document Change History Revision Date Change Author 0
05/28/14 Initial Issuance M. Montellano 1
08/07/14 Update this phase report to address NRC audit open items 121, 122, and 123. Reference DCPP PPS Open Item Summary Table.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
3 of 55 Date:
08/07/14 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 5 2.0 SCOPE.......................................................................................................... 5 3.0 TASKS PERFORMED............................................................................... 5 3.1 PHASE INPUTS.......................................................................................................... 5 3.2 PHASE TASKS........................................................................................................... 6 4.0 TASK RESULTS......................................................................................... 8 4.1 RESULTS
SUMMARY
.............................................................................................. 8 4.2 TASK AND PHASE ITERATION.......................................................................... 10 4.3 PHASE OUTPUTS................................................................................................... 10 5.0 IMPLEMENTATION PHASE ANOMALIES....................................... 13 6.0 SOFTWARE QUALITY........................................................................... 37 6.1 SOFTWARE QUALITY METRICS...................................................................... 37 6.1.1 Defect Count37 6.1.1.1 PTM Defect Count.......................................................................................... 37 6.1.1.2 SDD Defect Count........................................................................................... 38 6.1.2 Defect Density. 38 6.1.2.1 PTM Defect Density........................................................................................ 39 6.1.2.2 SDD Defect Density........................................................................................ 40 6.2 V&V EFFECTIVENESS METRICS...................................................................... 41 6.2.1 V&V Review Completeness41 6.2.2 V&V Reporting Accuracy... 42 6.3 QUALITY EVALUATION
SUMMARY
............................................................... 43 7.0 SOFTWARE SAFETY METRICS.......................................................... 45 8.0 TECHNICAL AND MANAGEMENT RISKS....................................... 46 9.0 RECOMMENDATIONS.......................................................................... 46
10.0 REFERENCES
........................................................................................... 47 11.0 APPENDICES............................................................................................ 48 Appendix 1: PTM Software Quality Metrics Data. 48 Appendix 2: SDD Software Quality Metrics Data. 48
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
4 of 55 Date:
08/07/14 12.0 ATTACHMENTS...................................................................................... 55
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
5 of 55 Date:
08/07/14 1.0 PURPOSE The purpose of this V&V Implementation Phase Summary Report is to summarize the results of the V&V tasks performed for the implementation phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting implementation specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
2.0 SCOPE The PPSI V&V implementation phase activities include the verification software coding and testing. The objectives of the V&V efforts are to ensure that the design requirements are clear, consistent, complete, readable, testable, and correctly translate and trace to the design.
Per Project Conditional Release (CR No. 993754-06), the V&V implementation phase activities also include the following tasks for Protection Set I:
- 1. Generate, verify, and issue Hardware Validation Test (HVT) Procedure, HVT dry-run, performance of HVT.
3.0 TASKS PERFORMED 3.1 PHASE INPUTS The table below contains the implementation phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] which lists document revisions, deletions or additions are considered as customer inputs during the entire implementation phase and all project iteration phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report represents the project inputs applicable only for PPSI V&V implementation phase tasks.
Table 3-1 Implementation Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 12 08/26/2011 993754-1-803 Master Configuration List (MCL) 39 04/11/2014 993754-1-804 Project Traceability Matrix (PTM) 6 04/11/2014 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 3 01/16/2014 993754-1-802 Software Verification and Validation Plan (SVVP) 4 01/28/2014
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
6 of 55 Date:
08/07/14 993754-1-915 Design Phase Safety (Criticality/Hazard/Risk/Interface)
Analysis 4
04/03/2014 993754-11-809 Software Requirements Specification (SRS) 4 01/21/2014 993754-11-810 Protection Set I Software Design Description (SDD) 2 04/02/2014 993754-11-917 Hardware Design Description (HDD) 2 02/14/2014 993754-1-819 Failure Modes and Effects Analysis (FMEA) 1 02/24/2014 993754-1-907 Software Development Plan Coding Guidelines 2
01/28/2014 993754-1-817 Maximum TSAP Scan Time 1
04/09/2012 993754-1-868 Software Verification Test Plan (SVTP) 1 04/03/2014 993754-1-869 Software Verification Test Specification (SVTS) 1 04/03/2014 993754-1-813 Validation Test Specification (VTS) 1 04/04/2014 993754-11-806 Protection Set I Input / Output (I/O) List 2
12/19/2013 993754-11-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-11-700 PGE DCPP PPS.PT2 0.0 4/11/2014 3.2 PHASE TASKS The following are phase tasks associated with the PPSI implementation phase. Official document reviews are demonstrated in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
7 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
8 of 55 Date:
08/07/14 4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The table below contains a summary of PPSI implementation phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the implementation phase documents during the implementation phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-1 as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
9 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
10 of 55 Date:
08/07/14 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2 outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is no iteration since the start implementation phase on 4/08/2014.
Table 4-2: Implementation Phase Task and Phase Iteration Iteration #
Date Reason Tasks Performed None N/A N/A N/A 4.3 PHASE OUTPUTS
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
11 of 55 Date:
08/07/14 The following table lists the final output documents for the Implementation Phase.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
12 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
13 of 55 Date:
08/07/14 5.0 IMPLEMENTATION PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). For test procedure discrepancies, an Interim Change Notice (ICN) will be generated and the appropriate level of review/approval as defined in PPMs 6.0 and 10.0. Corrective Action Reports (CARs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of implementation phase tasks are listed in Table 5-1, 5-2, and 5-3.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
14 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
15 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
16 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
17 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
18 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
19 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
20 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
21 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
22 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
23 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
24 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
25 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
26 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
27 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
28 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
29 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
30 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
31 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
32 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
33 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
34 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
35 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
36 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
37 of 55 Date:
08/07/14 6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the implementation phase.
Defect density is the defect count divided by the number of opportunities for error (OFE).
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the PTM and SDD during IV&V review
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
38 of 55 Date:
08/07/14 6.1.2 Defect Density Defect Density = Defects (PTM, SDD / Opportunities for Error (OFE)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
39 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
40 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
41 of 55 Date:
08/07/14 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
6.2.1 V&V Review Completeness
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
42 of 55 Date:
08/07/14 6.2.2 V&V Reporting Accuracy
%Accuracy = Number of Software Product Defects (Actual) / Number of Reported Product Defects (Potential Defects) x 100 6.2.2.1PTM V&V Reporting Accuracy 6.2.2.3 SDD V&V Reporting Accuracy
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
43 of 55 Date:
08/07/14 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9 below lists the quality evaluation summary for the implementation phase.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
44 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
45 of 55 Date:
08/07/14 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the implementation phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
46 of 55 Date:
08/07/14 8.0 TECHNICAL AND MANAGEMENT RISKS All of the PPSI implementation phase tasks are complete, deficiencies are resolved, and output documents are issued. Issued CAR 2507, TS1131 Emulator Test Driver (ETD) Rev 4.4.0 issue with "Result Check Value Function" to track the ETD anomaly. This ETD anomaly does not pose any technical or management risk to PPSI-IV project.
Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set I, Revisions 2A, 3, and 4 based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following are satisfied:
- 1. Functionality (e.g., algorithms, state/mode definitions, input/output validation, exception handling, reporting and logging);
- 2. Process definition and scheduling;
- 3. Hardware, software, and user interface descriptions;
- 4. Performance criteria (e.g., timing, sizing, speed, capacity, accuracy, precision, safety, and security);
- 5. Critical configuration data;
- 6. System, device, and software control (e.g., initialization, transaction and state monitoring, and self-testing).
All documents released by the Nuclear Delivery in the implementation phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP)
[Ref. 10.2.8]
There are no technical issues associated with the implementation phase outputs or the phase completion.
9.0 RECOMMENDATIONS Nuclear IV&V recommends exiting the implementation phase.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
47 of 55 Date:
08/07/14
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Software Requirements Specification (SRS), 993754-11-809 10.2.10 Deleted 10.2.11 Deleted 10.2.12 Deleted 10.2.13 Master Configuration List (MCL), 993754-1-803 10.2.14 Validation Test Plan (VTP), 993754-1-813 10.2.15 Protection Set I Software Design Description (SDD), 993754-11-810 10.2.16 Hardware Design Description (HDD), Protection Set I, 993754-11-917 10.2.17 Project Traceability Matrix (PTM), 993754-13-804 10.2.18 Software Verification Test Specification (SVTS), 993754-13-869 10.2.19 Software Verification Test Plan (SVTP), 993754-1-861 10.2.20 Validation Test Specification (VTS), 993754-13-813
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
48 of 55 Date:
08/07/14 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: PTM Software Quality Metrics Data Appendix 2: SDD Software Quality Metrics Data
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
49 of 55 Date:
08/07/14 Appendix 1: PTM Software Quality Metrics Data 11.1a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Traces Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
50 of 55 Date:
08/07/14 11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: PTM OFE(2) = # of Cells Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2) 11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
51 of 55 Date:
08/07/14 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
52 of 55 Date:
08/07/14 APPENDIX 2: SDD SOFTWARE QUALITY METRICS DATA 11.3a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.3b SDD Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Requirements Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
53 of 55 Date:
08/07/14 11.3c SDD Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: SDD OFE(2) = # of Pages Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
54 of 55 Date:
08/07/14 11.3d SDD V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
55 of 55 Date:
08/07/14 12.0 ATTACHMENTS None.
I n v e. n s*.~ s*
i r] v' e. r] S *.!::l s*
Triconex Project:
PG&E PROCESS PROTECTION SYSTEM REPLACEMENT Purchase Order No.:
3500880752 Project Sales Order:
993754 PACIFIC GAS & ELECTRIC COMPANY NUCLEAR SAFETY-RELATED PROCESS PROTECTION SYSTEM REPLACEMENT DIABLO CANYON POWER PLANT V &V IMPLEMENTATION PHASE
SUMMARY
REPORT PPSI Document No. 993754-11-862 Revision 1 August 07, 2014
~
Name Signature
\\
Title Author:
Myla Montellano
.~~,;y.-~
IV & V Engineer Reviewers:
Hoan Nguyen V.:r~ -------
IV & V Engineer Michael Shyu
~.~ 9-L-Project QA Engineer Approval:
Kevin Vu
/~~--
IV & V Manager
~
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
2 of 55 Date:
08/07/14 Document Change History Revision Date Change Author 0
05/28/14 Initial Issuance M. Montellano 1
08/07/14 Update this phase report to address NRC audit open items 121, 122, and 123. Reference DCPP PPS Open Item Summary Table.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
3 of 55 Date:
08/07/14 TABLE OF CONTENTS 1.0 PURPOSE..................................................................................................... 5 2.0 SCOPE.......................................................................................................... 5 3.0 TASKS PERFORMED............................................................................... 5 3.1 PHASE INPUTS.......................................................................................................... 5 3.2 PHASE TASKS........................................................................................................... 6 4.0 TASK RESULTS......................................................................................... 8 4.1 RESULTS
SUMMARY
.............................................................................................. 8 4.2 TASK AND PHASE ITERATION.......................................................................... 10 4.3 PHASE OUTPUTS................................................................................................... 10 5.0 IMPLEMENTATION PHASE ANOMALIES....................................... 13 6.0 SOFTWARE QUALITY........................................................................... 37 6.1 SOFTWARE QUALITY METRICS...................................................................... 37 6.1.1 Defect Count37 6.1.1.1 PTM Defect Count.......................................................................................... 37 6.1.1.2 SDD Defect Count........................................................................................... 38 6.1.2 Defect Density. 38 6.1.2.1 PTM Defect Density........................................................................................ 39 6.1.2.2 SDD Defect Density........................................................................................ 40 6.2 V&V EFFECTIVENESS METRICS...................................................................... 41 6.2.1 V&V Review Completeness41 6.2.2 V&V Reporting Accuracy... 42 6.3 QUALITY EVALUATION
SUMMARY
............................................................... 43 7.0 SOFTWARE SAFETY METRICS.......................................................... 45 8.0 TECHNICAL AND MANAGEMENT RISKS....................................... 46 9.0 RECOMMENDATIONS.......................................................................... 46
10.0 REFERENCES
........................................................................................... 47 11.0 APPENDICES............................................................................................ 48 Appendix 1: PTM Software Quality Metrics Data. 48 Appendix 2: SDD Software Quality Metrics Data. 48
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
4 of 55 Date:
08/07/14 12.0 ATTACHMENTS...................................................................................... 55
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
5 of 55 Date:
08/07/14 1.0 PURPOSE The purpose of this V&V Implementation Phase Summary Report is to summarize the results of the V&V tasks performed for the implementation phase of the application life cycle for the Nuclear Safety-Related (Class 1E) Diablo Canyon Power Plant (DCPP) Process Protection System (PPS) Replacement Project in order to fulfill the V&V reporting implementation specified in the Software Verification and Validation Plan (SVVP) [Ref.10.2.2].
2.0 SCOPE The PPSI V&V implementation phase activities include the verification software coding and testing. The objectives of the V&V efforts are to ensure that the design requirements are clear, consistent, complete, readable, testable, and correctly translate and trace to the design.
Per Project Conditional Release (CR No. 993754-06), the V&V implementation phase activities also include the following tasks for Protection Set I:
- 1. Generate, verify, and issue Hardware Validation Test (HVT) Procedure, HVT dry-run, performance of HVT.
3.0 TASKS PERFORMED 3.1 PHASE INPUTS The table below contains the implementation phase inputs, the associated document numbers, revisions, approval dates. The Technical Requirements List (TRL) [Ref.10.2.6] which lists document revisions, deletions or additions are considered as customer inputs during the entire implementation phase and all project iteration phases. It provides a complete listing of the customer input documents while the SVVP provides a generic guidance. Therefore, this report is not intended to duplicate those documents listed in the TRL. This report represents the project inputs applicable only for PPSI V&V implementation phase tasks.
Table 3-1 Implementation Phase Inputs Document ID Document Name Revision Date 993754-1-808 Technical Requirements List (TRL) 12 08/26/2011 993754-1-803 Master Configuration List (MCL) 39 04/11/2014 993754-1-804 Project Traceability Matrix (PTM) 6 04/11/2014 993754-11-807 Protection Set I Hardware Requirements Specification (HRS) 3 01/16/2014 993754-1-802 Software Verification and Validation Plan (SVVP) 4 01/28/2014
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
6 of 55 Date:
08/07/14 993754-1-915 Design Phase Safety (Criticality/Hazard/Risk/Interface)
Analysis 4
04/03/2014 993754-11-809 Software Requirements Specification (SRS) 4 01/21/2014 993754-11-810 Protection Set I Software Design Description (SDD) 2 04/02/2014 993754-11-917 Hardware Design Description (HDD) 2 02/14/2014 993754-1-819 Failure Modes and Effects Analysis (FMEA) 1 02/24/2014 993754-1-907 Software Development Plan Coding Guidelines 2
01/28/2014 993754-1-817 Maximum TSAP Scan Time 1
04/09/2012 993754-1-868 Software Verification Test Plan (SVTP) 1 04/03/2014 993754-1-869 Software Verification Test Specification (SVTS) 1 04/03/2014 993754-1-813 Validation Test Specification (VTS) 1 04/04/2014 993754-11-806 Protection Set I Input / Output (I/O) List 2
12/19/2013 993754-11-700-SDC-0 Software Development Checklist for TriStation Application Program 993754-11-700 PGE DCPP PPS.PT2 0.0 4/11/2014 3.2 PHASE TASKS The following are phase tasks associated with the PPSI implementation phase. Official document reviews are demonstrated in section 4.1 Results Summary and reported by the Document Review/Release (DRR) process. Various other document verifications listed in the SVVP are inherent to the documentation process.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
7 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
8 of 55 Date:
08/07/14 4.0 TASK RESULTS 4.1 RESULTS
SUMMARY
The table below contains a summary of PPSI implementation phase task results, the documents associated with each task, the revision of each document reviewed, the DRR number where each review was documented, and the date each document was issued. The Document Review/Release (DRR) processes documented all required review for both initial releases and approved changes to the implementation phase documents during the implementation phase. Required V&V reviews of certain task outputs are noted in the task descriptions of Table 4-1 as per section 3.2 of this report.
Note: The changes on the MCL do not require the use of a DRR. The revisions processed without DRRs are reported for information purposes.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
9 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
10 of 55 Date:
08/07/14 4.2 TASK AND PHASE ITERATION When phase inputs are changed, or issues with phase outputs are discovered, tasks are required to be re-executed. Table 4-2 outlines the iterations, the date iterations were initiated, the reason the iterations were initiated and the repeated tasks and/or revised phase outputs. There is no iteration since the start implementation phase on 4/08/2014.
Table 4-2: Implementation Phase Task and Phase Iteration Iteration #
Date Reason Tasks Performed None N/A N/A N/A 4.3 PHASE OUTPUTS
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
11 of 55 Date:
08/07/14 The following table lists the final output documents for the Implementation Phase.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
12 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
13 of 55 Date:
08/07/14 5.0 IMPLEMENTATION PHASE ANOMALIES Anomalies or deficiencies are documented on System Integration Deficiency Report (SIDR). For test procedure discrepancies, an Interim Change Notice (ICN) will be generated and the appropriate level of review/approval as defined in PPMs 6.0 and 10.0. Corrective Action Reports (CARs) are used to document internal programmatic deficiencies associated with project activities that needs corrective or preventive action. Anomalies or deficiencies that were identified during the completion of implementation phase tasks are listed in Table 5-1, 5-2, and 5-3.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
14 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
15 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
16 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
17 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
18 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
19 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
20 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
21 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
22 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
23 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
24 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
25 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
26 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
27 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
28 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
29 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
30 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
31 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
32 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
33 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
34 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
35 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
36 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
37 of 55 Date:
08/07/14 6.0 SOFTWARE QUALITY There are two categories of metrics considered in the phase summary reports to evaluate the software quality:
- 1. Software Quality Metrics-provides the metrics for evaluating the software development processes and products. These metrics are calculated using the defect count and defect density methods. Per the Invensys PPM [Ref.10.2.1], software documentation defects are classified as technical, editorial, or procedural.
Defect count is the number of defects in software products identified during the implementation phase.
Defect density is the defect count divided by the number of opportunities for error (OFE).
- 2. V&V Effectiveness Metrics-provide the metrics for evaluating the degree of completeness and accuracy of the work executed by the IV&V organization. Note that effectiveness does not include efficiency and productivity, which relate to the resources consumed to execute the activity. These metrics measures the V&V review completeness and reporting accuracy.
V&V Review Completeness is the percent of the phase software products that Nuclear IV&V reviews.
V&V Reporting Accuracy is the number of proven defects divided by the number of reported defects.
6.1 SOFTWARE QUALITY METRICS 6.1.1 Defect Count Defect Count = number of defects identified in the PTM and SDD during IV&V review
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
38 of 55 Date:
08/07/14 6.1.2 Defect Density Defect Density = Defects (PTM, SDD / Opportunities for Error (OFE)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
39 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
40 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
41 of 55 Date:
08/07/14 6.2 V&V EFFECTIVENESS METRICS Software V&V effectiveness metrics are calculated using two methods:
- 1. V&V Review Completeness - the amount (%) of contained material in the document that has undergone software V&V review.
- 2. V&V Reporting Accuracy - the measure of the number of reported defects that are ultimately proven to be defects.
6.2.1 V&V Review Completeness
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
42 of 55 Date:
08/07/14 6.2.2 V&V Reporting Accuracy
%Accuracy = Number of Software Product Defects (Actual) / Number of Reported Product Defects (Potential Defects) x 100 6.2.2.1PTM V&V Reporting Accuracy 6.2.2.3 SDD V&V Reporting Accuracy
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
43 of 55 Date:
08/07/14 6.3 QUALITY EVALUATION
SUMMARY
Table 6-9 below lists the quality evaluation summary for the implementation phase.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
44 of 55 Date:
08/07/14
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
45 of 55 Date:
08/07/14 7.0 SOFTWARE SAFETY METRICS The software safety effort and its effectiveness in the implementation phase are assessed in the metrics below. The safety metrics were outlined in the Software Safety Plan (SSP) [Ref. 10.2.7]. The safety metrics are divided into four categories to show the collected safety numbers and their justifications.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
46 of 55 Date:
08/07/14 8.0 TECHNICAL AND MANAGEMENT RISKS All of the PPSI implementation phase tasks are complete, deficiencies are resolved, and output documents are issued. Issued CAR 2507, TS1131 Emulator Test Driver (ETD) Rev 4.4.0 issue with "Result Check Value Function" to track the ETD anomaly. This ETD anomaly does not pose any technical or management risk to PPSI-IV project.
Nuclear IV&V reviewed the Software Design Description (SDD) for Protection Set I, Revisions 2A, 3, and 4 based on IEEE 1012-1998 software requirements evaluation criteria and determined that the following are satisfied:
- 1. Functionality (e.g., algorithms, state/mode definitions, input/output validation, exception handling, reporting and logging);
- 2. Process definition and scheduling;
- 3. Hardware, software, and user interface descriptions;
- 4. Performance criteria (e.g., timing, sizing, speed, capacity, accuracy, precision, safety, and security);
- 5. Critical configuration data;
- 6. System, device, and software control (e.g., initialization, transaction and state monitoring, and self-testing).
All documents released by the Nuclear Delivery in the implementation phase have been correctly developed and updated in the MCL described in the Software Configuration Management Plan (SCMP)
[Ref. 10.2.8]
There are no technical issues associated with the implementation phase outputs or the phase completion.
9.0 RECOMMENDATIONS Nuclear IV&V recommends exiting the implementation phase.
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
47 of 55 Date:
08/07/14
10.0 REFERENCES
10.1 INDUSTRY DOCUMENTS 10.1.1 IEEE 1012-1998, Standard for Software Verification and Validation 10.2 INVENSYS TRICONEX DOCUMENTS 10.2.1 Invensys Project Procedures Manual (PPM) 10.2.2 Software Verification and Validation Plan (SVVP), 993754-1-802 10.2.3 Project Management Plan (PMP), 993754-1-905 10.2.4 Software Quality Assurance Plan (SQAP), 993754-1-801 10.2.5 Safety Analysis (Criticality/Hazards/Risks/Interfaces), 993754-1-915 10.2.6 Technical Requirements List (TRL), 993754-1-801 10.2.7 Software Safety Plan (SSP), 993754-1-911 10.2.8 Software Configuration Management Plan (SCMP), 993754-909 10.2.9 Software Requirements Specification (SRS), 993754-11-809 10.2.10 Deleted 10.2.11 Deleted 10.2.12 Deleted 10.2.13 Master Configuration List (MCL), 993754-1-803 10.2.14 Validation Test Plan (VTP), 993754-1-813 10.2.15 Protection Set I Software Design Description (SDD), 993754-11-810 10.2.16 Hardware Design Description (HDD), Protection Set I, 993754-11-917 10.2.17 Project Traceability Matrix (PTM), 993754-13-804 10.2.18 Software Verification Test Specification (SVTS), 993754-13-869 10.2.19 Software Verification Test Plan (SVTP), 993754-1-861 10.2.20 Validation Test Specification (VTS), 993754-13-813
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
48 of 55 Date:
08/07/14 11.0 APPENDICES The following appendices include supporting data used in the software quality metrics calculations.
Appendix 1: PTM Software Quality Metrics Data Appendix 2: SDD Software Quality Metrics Data
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
49 of 55 Date:
08/07/14 Appendix 1: PTM Software Quality Metrics Data 11.1a PTM Defect Count Data Defect Count for the PTM = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.2b PTM Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Traces Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
50 of 55 Date:
08/07/14 11.2c PTM Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: PTM OFE(2) = # of Cells Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2) 11.2d PTM V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
51 of 55 Date:
08/07/14 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
52 of 55 Date:
08/07/14 APPENDIX 2: SDD SOFTWARE QUALITY METRICS DATA 11.3a SDD Defect Count Data Defect Count for the SDD = Incorporated comments in the DRCS Uninc = Unincorporated comments in the DRCS Nedit = number of editorial defects Nproc = number of procedural defects Ntech = number of technical defects Ntot = total number of defects = Ntech +Nedit+Nproc 11.3b SDD Defect Density (1) Data Defect Density (1) = Defects/Opportunities for Error(1)
Where: PTM OFE(1) = # of Requirements Density of Editorial defects (DDedit1) = Nedit / OFE(1)
Density of Procedural defects (DDproc1) = Nproc / OFE(1)
Density of Technical defects (DDtech1) = Ntech / OFE(1)
Density of all defects (DDtot1) = Ntot / OFE(1)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
53 of 55 Date:
08/07/14 11.3c SDD Defect Density (2) Data Defect Density (2) = Defects/Opportunities for Error(2)
Where: SDD OFE(2) = # of Pages Density of Editorial defects (DDedit2) = Nedit / OFE(2)
Density of Procedural defects (DDproc2) = Nproc / OFE(2)
Density of Technical defects (DDtech2) = Ntech / OFE(2)
Density of all defects (DDtot2) = Ntot / OFE(2)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
54 of 55 Date:
08/07/14 11.3d SDD V&V Reporting Accuracy Data Accuracy (%) = Ndefects/ Nrep x 100 Ndefects = Number of software product defects (actual defects)
Nrep = Number of reported product defects (potential defects)
Document:
993754-11-862
Title:
V&V Implementation Phase Summary Report PPSI Revision:
1 Page:
55 of 55 Date:
08/07/14 12.0 ATTACHMENTS None.