ML19233A215: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
 
Line 15: Line 15:


=Text=
=Text=
{{#Wiki_filter:Description of Preliminary Staff Evaluation for Use of IEC 60880 and IEC 61513 for Software Development Steven Arndt, Deanna Zhang, Richard Stattel and Paul RebstockAugust 29, 2019 Agenda*Background
{{#Wiki_filter:Description of Preliminary Staff Evaluation for Use of IEC 60880 and IEC 61513 for Software Development Steven Arndt, Deanna Zhang, Richard Stattel and Paul Rebstock August 29, 2019
*Issues To Be Addressed
 
*Preliminary Evaluations of Software Regulatory GuidesReview methodologyRG 1.168RG 1.169RG 1.171 2  
Agenda
* Background
* Issues To Be Addressed
* Preliminary Evaluations of Software Regulatory Guides Review methodology RG 1.168 RG 1.169 RG 1.171 2


===Background===
===Background===
*Original plan was to develop guidance that endorses specific IEC standards (e.g., IEC 61513) as an alternative to the IEEE standards incorporated by reference (IBR) in 10 CFR 50.55a(h)Endorse IEC standards as an alternative to IBR IEEE Standards in accordance with 10 CFR 50.55a(z)Evaluate such endorsement's impact current regulatory guidance, including the current regulatory guides, with respect to other IEC and/or IEEE standards
* Original plan was to develop guidance that endorses specific IEC standards (e.g., IEC 61513) as an alternative to the IEEE standards incorporated by reference (IBR) in 10 CFR 50.55a(h)
*Initial evaluation determined:Endorsing IEC standards as alternatives to IBR standards may not be useful for operating reactorsSeveral digital I&C platforms that have been developed using IEC standards were previously approved by the NRC for use in safety applications of nuclear power plantsPotential to endorse, via most likely a regulatory guide, a subset of IEC standards with focus on software development as one potential way to satisfy regulatory requirements related to quality 3
Endorse IEC standards as an alternative to IBR IEEE Standards in accordance with 10 CFR 50.55a(z)
Evaluate such endorsements impact current regulatory guidance, including the current regulatory guides, with respect to other IEC and/or IEEE standards
* Initial evaluation determined:
Endorsing IEC standards as alternatives to IBR standards may not be useful for operating reactors Several digital I&C platforms that have been developed using IEC standards were previously approved by the NRC for use in safety applications of nuclear power plants Potential to endorse, via most likely a regulatory guide, a subset of IEC standards with focus on software development as one potential way to satisfy regulatory requirements related to quality 3
 
Issues to be Addressed
Issues to be Addressed
*Software development standards are the focus for endorsementThis effort evaluates whether IEC 60880 could be endorsed as one way to meet regulations related to qualityIdentify which other IEC standards need to be endorsed if there are gaps within IEC 60880 for a certain topicSignificant differences in the classification of systems in IEC and IEEE nuclear standards will need to be addressed oUse IEC 61266 to categorize functions performed and corresponding safety class; or oLimit application to "safety
* Software development standards are the focus for endorsement This effort evaluates whether IEC 60880 could be endorsed as one way to meet regulations related to quality Identify which other IEC standards need to be endorsed if there are gaps within IEC 60880 for a certain topic Significant differences in the classification of systems in IEC and IEEE nuclear standards will need to be addressed o Use IEC 61266 to categorize functions performed and corresponding safety class; or o Limit application to safety-related systems that is equivalent to Class 1 systems performing Category A functions 4
-related" systems that is equivalent to Class 1 systems performing Category A functions 4
 
IEC 61513 and IEC Software StandardsIEC 61513IEC 60880 SoftwareIEC 61500 CommunicationIEC 62340 Common Cause FailureIEC 62566 HDL/FPGAIEC 60987 HardwareIEC 62645    CyberIEC 60780 QualificationIEC 61000 EMI/RFIIEC 61266Classification 5
IEC 61513 and IEC Software Standards IEC 61513 IEC 61266 Classification IEC 60880       IEC 62566 IEC 60987             IEC 62645 IEC 60780   IEC 61000 Software      HDL/FPGA    Hardware              Cyber  Qualification  EMI/RFI IEC 61500 Communication IEC 62340 Common Cause Failure 5
6IEC 60880 software for category A functions Preliminary Evaluations of Software Regulatory Guides
 
*How have we done the evaluation?Review of the underlying regulationsReview of the what is needed to demonstrate that the regulations are being meet and the system is safeEvaluation of the method by which the regulatory guides and endorsed IEEE standards provide that assuranceReview the IEC 60880 criteria Determine whether IEC 60880 provides equivalent demonstration of safety and/or if additional criteria are needed 7
IEC 60880 software for category A functions 6
 
Preliminary Evaluations of Software Regulatory Guides
* How have we done the evaluation?
Review of the underlying regulations Review of the what is needed to demonstrate that the regulations are being meet and the system is safe Evaluation of the method by which the regulatory guides and endorsed IEEE standards provide that assurance Review the IEC 60880 criteria Determine whether IEC 60880 provides equivalent demonstration of safety and/or if additional criteria are needed 7
 
Preliminary Review of Software Verification and Validation 8
Preliminary Review of Software Verification and Validation 8
Software Verification and Validation
Software Verification and Validation
*RG 1.168 endorses IEEE Std1012-2004 for meeting the NRC's requirements for verification and validation of safety
* RG 1.168 endorses IEEE Std 1012-2004 for meeting the NRCs requirements for verification and validation of safety-related system software.
-related system software.
* IEEE Std 1012-2004 is a computer society standard.
*IEEE Std1012-2004 is a computer society standard.It establishes a minimum set of V&V activities for software based on a Software Integrity Level (SIL) classification. RG 1.168 states that software used in nuclear power plant safety systems should be assigned integrity level 4 or the equivalent, as demonstrated by a mapping between the applicant or licensee approach and integrity level 4.
It establishes a minimum set of V&V activities for software based on a Software Integrity Level (SIL) classification.
9 Software Verification and Validation
RG 1.168 states that software used in nuclear power plant safety systems should be assigned integrity level 4 or the equivalent, as demonstrated by a mapping between the applicant or licensee approach and integrity level 4.
*IEC 60880 includes software verification (Section 8) and software integration (Section 10) criteria for nuclear software applications used to perform safety functions with the highest category of safety (i.e., Category A).  
9
*NRC staff evaluated IEC 60880 to determine if the criteria provided in this standard meet the underlying V&V regulatory requirements cited in RG 1.168.10 Examples of Evaluation  
 
*Examples of criteria within IEC 60880 that support compliance with underlying regulatory requirements for software verification:IEC 60880, Section 8 provides criteria for establishing independence between a verification team and the development team oThese criteria provide an acceptable way of meeting the quality requirements in IEEE 603 Section 5.3, "Quality" oThese criteria are consistent with the criterion of IEEE Std 7
Software Verification and Validation
-4.3.2-2003, Section 5.3.4, "V&V Independence Requirements" and are consistent with IEEE 1012 Section 5 and Section C.3 of RG 1.168Section 8 also includes requirements for performing verification of outputs from each of the development life cycle phases oThese activities confirm the adequacy of the software requirements specification in fulfilling the system requirements oThese requirements are compatible with IEEE Std 1012-2004, Section 5, "Software V&V Process," and thus provide an acceptable means of establishing quality in safety I&C systems as required by IEEE 603
* IEC 60880 includes software verification (Section
-1991, Section 5.3 11 Examples of Evaluation Contd.  
: 8) and software integration (Section 10) criteria for nuclear software applications used to perform safety functions with the highest category of safety (i.e., Category A).
*Example of criteria within IEC 60880 that meets the underlying regulatory requirements for software validation:Section 10 provides criteria for validating design of an integrated system These criteria provide an acceptable way of meeting the objectives of the validation activities prescribed in IEEE Std 1012-2004, Table 2 12 Examples of Evaluation Contd.
* NRC staff evaluated IEC 60880 to determine if the criteria provided in this standard meet the underlying V&V regulatory requirements cited in RG 1.168.
*Examples where criteria in IEC 60880 ar enot sufficient to provide one acceptable way of meeting underlying regulatory requirements for verification and validation:Rather than prescribe a minimum set of V&V activities to be performed for each phase of development, IEC 60880 is results oriented Criteria within this standard are the resulting characteristics of the safety system Users of this standard is allowed to decide which V&V activities would be needed to achieve the required results The IEC standard does not include tables of V&V activities or any equivalent list of activities to be performed for safety software Mapping of activities performed to activities prescribed by this standard cannot be performedDocumentation focuses on system design performance results rather than completion of specified V&V activities 13 Software Verification and Validation(IEC 61513)  
10
*IEC 61513 includes criteria for deriving the I&C requirements from the plant safety design base and design of the entire I&C architecture and assignment of the I&C functions Several V&V activities are specified within this criterion. Examples include:Performance of Requirements TraceabilityAssignment of Software Integrity LevelsVerifying requirements specifications to ensure correct assignment of functions to systems
 
*NRC staff is evaluating IEC 61513 to determine if the criteria provided in this standard can be used as a means of meeting the underlying V&V regulatory requirements cited in RG 1.168 14 Verification Definition Differences
Examples of Evaluation
*Verification (IEEE Std1012-2004)Process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phaseProcess of providing objective evidence that the software and its associated products:
* Examples of criteria within IEC 60880 that support compliance with underlying regulatory requirements for software verification:
oConform to requirements (e.g., for correctness, completeness, consistency, accuracy) for all life cycle activities during each life cycle process (acquisition, supply, development, operation, and maintenance);
IEC 60880, Section 8 provides criteria for establishing independence between a verification team and the development team o These criteria provide an acceptable way of meeting the quality requirements in IEEE 603 Section 5.3, Quality o These criteria are consistent with the criterion of IEEE Std 7-4.3.2-2003, Section 5.3.4, V&V Independence Requirements and are consistent with IEEE 1012 Section 5 and Section C.3 of RG 1.168 Section 8 also includes requirements for performing verification of outputs from each of the development life cycle phases o These activities confirm the adequacy of the software requirements specification in fulfilling the system requirements o These requirements are compatible with IEEE Std 1012-2004, Section 5, Software V&V Process, and thus provide an acceptable means of establishing quality in safety I&C systems as required by IEEE 603-1991, Section 5.3 11
oSatisfy standards, practices, and conventions during life cycle processes; and successfully complete each life cycle activity; and oSatisfy all the criteria for initiating succeeding life cycle activities (e.g., building the software correctly)
 
*Verification (IEC 60880)Confirmation by examination and by provision of objective evidence that the results of an activity meet the objectives and requirements defined for this activity 15 Validation Definition Differences
Examples of Evaluation Contd.
*Validation (IEEE Std1012-2004)Process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirementsProcess of providing evidence that the software and its associated products:
* Example of criteria within IEC 60880 that meets the underlying regulatory requirements for software validation:
oSatisfy system requirements allocated to software at the end of each life cycle activity; oSolve the right problem (e.g., correctly model physical laws, implement business rules, use the proper system assumptions); and oSatisfy intended use and user needs Validation (IEC 60880) Confirmation by examination and provision of other evidence that a system fulfils in its entirety the requirement specification as intended (e.g., functionality, response time, fault tolerance, robustness) 16 Preliminary Review of Software Configuration Management 17 Software Configuration Management
Section 10 provides criteria for validating design of an integrated system These criteria provide an acceptable way of meeting the objectives of the validation activities prescribed in IEEE Std 1012-2004, Table 2 12
*RG 1.169 endorses IEEE Std 828-2005 for meeting the NRC's requirements for configuration management for safety
 
-related system software
Examples of Evaluation Contd.
*IEEE Std 828-2005 is a computer society standard.No minimum set of activities established for configuration management for nuclear safety software applications.RG 1.169 identifies the minimum set of activities for nuclear safety software applications
* Examples where criteria in IEC 60880 are not sufficient to provide one acceptable way of meeting underlying regulatory requirements for verification and validation:
*IEC 60880 provides criteria for software configuration management for nuclear software applications used to perform safety functions with the highest category of safety (i.e. Category A) *Staff evaluated IEC 60880 to determine if the criteria provided in this standard meetthe underlying configuration management regulatory requirements cited in RG 1.169 18 Examples of Evaluation  
Rather than prescribe a minimum set of V&V activities to be performed for each phase of development, IEC 60880 is results oriented Criteria within this standard are the resulting characteristics of the safety system Users of this standard is allowed to decide which V&V activities would be needed to achieve the required results The IEC standard does not include tables of V&V activities or any equivalent list of activities to be performed for safety software Mapping of activities performed to activities prescribed by this standard cannot be performed Documentation focuses on system design performance results rather than completion of specified V&V activities 13
*Examples where criteria within IEC 60880 that meet the underlying regulatory requirements for configuration management include Sections 5.6.4, 5.6.5, 5.6.7, and 5.6.9 These sections provide criteria for identifying software such as the following:
 
oEach produced version of each software entity shall be uniquely identified oIt shall be possible to identify the relevant versions of all software documentation associated with each software entity oIt shall be possible to identify the versions of all software entities which together constitute a complete version of the final productThese criteria provide an acceptable way of meeting:
Software Verification and Validation (IEC 61513)
o10 CFR Part 50, Appendix A, GDC 1 oIEEE Std603-1991, Clause 5.3 requirements for quality 19 Examples of Evaluation Contd.
* IEC 61513 includes criteria for deriving the I&C requirements from the plant safety design base and design of the entire I&C architecture and assignment of the I&C functions Several V&V activities are specified within this criterion.
*Examples where criteria within IEC 60880 that meet the underlying regulatory requirements for configuration management include Sections 6.4.2, 7.1.1.9, 7.1.4.3, and 7.4.1These sections provide criteria to ensure configuration management of software documentation such as the following:
Examples include:
oThe software requirements specification shall be presented according to a standard whose formality should not preclude readability oComprehensive and clearly written documentation shall be provided oThe configuration data shall be documented These criteria provide an acceptable way of meeting:
Performance of Requirements Traceability Assignment of Software Integrity Levels Verifying requirements specifications to ensure correct assignment of functions to systems
o10 CFR Part 50, Appendix A, GDC 1 oIEEE Std603-1991, Clause 5.3 requirements for quality 20 Examples of Evaluation Contd.
* NRC staff is evaluating IEC 61513 to determine if the criteria provided in this standard can be used as a means of meeting the underlying V&V regulatory requirements cited in RG 1.168 14
*Examples where criteria in IEC 60880 were insufficient to provide one acceptable way of meeting underlying regulatory requirements for configuration management:Section 5.6, including its subsections references the system level configuration management requirements in IEC 61513, including provision of a configuration management plan or the quality assurance plan Control of software vendors supplying safety systems software not addressed in IEC 60880 Software configuration audits not addressed in IEC 60880 21 Preliminary Review of Software Unit Testing 22 Provisions for Software Unit Testing
 
*Software Unit Testing is addressed in RG 1.171RG 1.171 endorses, with clarifications and additions, IEEE 1008-1987 "IEEE Standard for Software Unit Testing"RG 1.171 endorses the 2008 edition of IEEE Std829 "IEEE Standard for Software Test Documentation", in lieu of the version cited in IEEE 1008-1987 23 Regulatory Requirements for Software Unit Testing
Verification Definition Differences
*The regulatory requirements cited in RG 1.171 include:Requirements related to quality and documentationRequirements for periodic testingRequirements related to the development and testing of design changesRequirements for documentation of test plans and resultsNo explicit requirements for unit testing 24 Provisions for Software Unit Testing in IEC 60880*Annex E (nonmandatory) presents general recommendations for system testing
* Verification (IEEE Std 1012-2004)
*Annex B (mandatory) presents considerations for unit and integration tests (B4.g), and for testing related to modifications (B1.d)
Process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase Process of providing objective evidence that the software and its associated products:
*Various sections within IEC 60880 contain provisions for: Testing modifications;Selection of test cases; andDocumentation of test plans and resultsRequirements are generally not very specific and consensus must be attained as to whether they are adequate to meet the regulatory requirements cited in RG 1.171 25 NextStep s 26 Stakeholder Engagement
o Conform to requirements (e.g., for correctness, completeness, consistency, accuracy) for all life cycle activities during each life cycle process (acquisition, supply, development, operation, and maintenance);
*NRC would like to work with an industry working groupHelp identify a subset of the suite of IEC standards that would be of most use to industry Provide early feedback on NRC strategy for endorsementProvide feedback on overlaps, gaps, and possible challenges to endorsement
o Satisfy standards, practices, and conventions during life cycle processes; and successfully complete each life cycle activity; and o Satisfy all the criteria for initiating succeeding life cycle activities (e.g.,
*Table top/example review Identify a specific system or platform to evaluate the new process to ensure the guidance is practical and well understoodMake a selection as soon as possibleReview will be done over the course of a few months, so will need to be appropriately scoped 27 Proposed Project Plan(Key Milestones)
building the software correctly)
*Public meeting to engage stakeholders on the proposal, identify participants for the working group, and identify a subset of the suite of IEC standards that would be of most use to industry (Done)*Formalize project plan and select IEC standards to include (Done)*Work with OGC to determine appropriate guidance document (Done)*Coordinate with IEEE and IEC (Done)*Conduct review of IEC nuclear (45a) standards to determine overlaps, gaps and possible challenges with endorsement of IEC 61513 suite of standards (in-progress)*Select system or platform for example review (Fall 2019)
* Verification (IEC 60880)
*Develop possible solutions to previously identified concerns associated with endorsement of IEC standards (Fall 2019)
Confirmation by examination and by provision of objective evidence that the results of an activity meet the objectives and requirements defined for this activity 15
*Complete example review (Early 2020)
 
*Brief ACRS (Early 2020)
Validation Definition Differences
*Publish draft guidance for public comment (Spring 2020) 28 Discussion 29 Acronyms*ACRS: Advisory Committee on Reactor Safeguards
* Validation (IEEE Std 1012-2004)
*CFR: Code of Federal Regulations
Process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements Process of providing evidence that the software and its associated products:
*GDC: General Design Criterion
o Satisfy system requirements allocated to software at the end of each life cycle activity; o Solve the right problem (e.g., correctly model physical laws, implement business rules, use the proper system assumptions); and o Satisfy intended use and user needs Validation (IEC 60880)
*I&C: Instrumentation and Controls
Confirmation by examination and provision of other evidence that a system fulfils in its entirety the requirement specification as intended (e.g., functionality, response time, fault tolerance, robustness) 16
*IBR: Incorporated by Reference
 
*IEC: International Electrotechnical Commission
Preliminary Review of Software Configuration Management 17
*IEEE: Institute of Electrical and Electronics Engineering
 
*NRC: Nuclear Regulatory Commission
Software Configuration Management
*RG: Regulatory Guide
* RG 1.169 endorses IEEE Std 828-2005 for meeting the NRCs requirements for configuration management for safety-related system software
*SIL: Software Integrity Level
* IEEE Std 828-2005 is a computer society standard.
*V&V: Verification and Validation 30}}
No minimum set of activities established for configuration management for nuclear safety software applications.
RG 1.169 identifies the minimum set of activities for nuclear safety software applications
* IEC 60880 provides criteria for software configuration management for nuclear software applications used to perform safety functions with the highest category of safety (i.e. Category A)
* Staff evaluated IEC 60880 to determine if the criteria provided in this standard meet the underlying configuration management regulatory requirements cited in RG 1.169 18
 
Examples of Evaluation
* Examples where criteria within IEC 60880 that meet the underlying regulatory requirements for configuration management include Sections 5.6.4, 5.6.5, 5.6.7, and 5.6.9 These sections provide criteria for identifying software such as the following:
o Each produced version of each software entity shall be uniquely identified o It shall be possible to identify the relevant versions of all software documentation associated with each software entity o It shall be possible to identify the versions of all software entities which together constitute a complete version of the final product These criteria provide an acceptable way of meeting:
o 10 CFR Part 50, Appendix A, GDC 1 o IEEE Std 603-1991, Clause 5.3 requirements for quality 19
 
Examples of Evaluation Contd.
* Examples where criteria within IEC 60880 that meet the underlying regulatory requirements for configuration management include Sections 6.4.2, 7.1.1.9, 7.1.4.3, and 7.4.1 These sections provide criteria to ensure configuration management of software documentation such as the following:
o The software requirements specification shall be presented according to a standard whose formality should not preclude readability o Comprehensive and clearly written documentation shall be provided o The configuration data shall be documented These criteria provide an acceptable way of meeting:
o 10 CFR Part 50, Appendix A, GDC 1 o IEEE Std 603-1991, Clause 5.3 requirements for quality 20
 
Examples of Evaluation Contd.
* Examples where criteria in IEC 60880 were insufficient to provide one acceptable way of meeting underlying regulatory requirements for configuration management:
Section 5.6, including its subsections references the system level configuration management requirements in IEC 61513, including provision of a configuration management plan or the quality assurance plan Control of software vendors supplying safety systems software not addressed in IEC 60880 Software configuration audits not addressed in IEC 60880 21
 
Preliminary Review of Software Unit Testing 22
 
Provisions for Software Unit Testing
* Software Unit Testing is addressed in RG 1.171 RG 1.171 endorses, with clarifications and additions, IEEE 1008-1987 IEEE Standard for Software Unit Testing RG 1.171 endorses the 2008 edition of IEEE Std 829 IEEE Standard for Software Test Documentation, in lieu of the version cited in IEEE 1008-1987 23
 
Regulatory Requirements for Software Unit Testing
* The regulatory requirements cited in RG 1.171 include:
Requirements related to quality and documentation Requirements for periodic testing Requirements related to the development and testing of design changes Requirements for documentation of test plans and results No explicit requirements for unit testing 24
 
Provisions for Software Unit Testing in IEC 60880
* Annex E (nonmandatory) presents general recommendations for system testing
* Annex B (mandatory) presents considerations for unit and integration tests (B4.g), and for testing related to modifications (B1.d)
* Various sections within IEC 60880 contain provisions for:
Testing modifications; Selection of test cases; and Documentation of test plans and results Requirements are generally not very specific and consensus must be attained as to whether they are adequate to meet the regulatory requirements cited in RG 1.171 25
 
Next Steps 26
 
Stakeholder Engagement
* NRC would like to work with an industry working group Help identify a subset of the suite of IEC standards that would be of most use to industry Provide early feedback on NRC strategy for endorsement Provide feedback on overlaps, gaps, and possible challenges to endorsement
* Table top/example review Identify a specific system or platform to evaluate the new process to ensure the guidance is practical and well understood Make a selection as soon as possible Review will be done over the course of a few months, so will need to be appropriately scoped 27
 
Proposed Project Plan (Key Milestones)
* Public meeting to engage stakeholders on the proposal, identify participants for the working group, and identify a subset of the suite of IEC standards that would be of most use to industry (Done)
* Formalize project plan and select IEC standards to include (Done)
* Work with OGC to determine appropriate guidance document (Done)
* Coordinate with IEEE and IEC (Done)
* Conduct review of IEC nuclear (45a) standards to determine overlaps, gaps and possible challenges with endorsement of IEC 61513 suite of standards (in-progress)
* Select system or platform for example review (Fall 2019)
* Develop possible solutions to previously identified concerns associated with endorsement of IEC standards (Fall 2019)
* Complete example review (Early 2020)
* Brief ACRS (Early 2020)
* Publish draft guidance for public comment (Spring 2020) 28
 
Discussion 29
 
Acronyms
* ACRS: Advisory Committee on Reactor Safeguards
* CFR: Code of Federal Regulations
* GDC: General Design Criterion
* I&C: Instrumentation and Controls
* IBR: Incorporated by Reference
* IEC: International Electrotechnical Commission
* IEEE: Institute of Electrical and Electronics Engineering
* NRC: Nuclear Regulatory Commission
* RG: Regulatory Guide
* SIL: Software Integrity Level
* V&V: Verification and Validation 30}}

Latest revision as of 00:06, 8 October 2019

Description of Preliminary Staff Evaluation for Use of Iec 60880 and Iec 61513 for Software Development - Iec Project Meeting Presentation 2 - Clean (Aug 2019)
ML19233A215
Person / Time
Issue date: 08/29/2019
From: Paul Kallan
NRC/NRO/DLSE
To:
Kallan P, NRR/DLP, 415-2809
References
Download: ML19233A215 (30)
v • d • e


Text

Description of Preliminary Staff Evaluation for Use of IEC 60880 and IEC 61513 for Software Development Steven Arndt, Deanna Zhang, Richard Stattel and Paul Rebstock August 29, 2019

Agenda

  • Background
  • Issues To Be Addressed
  • Preliminary Evaluations of Software Regulatory Guides Review methodology RG 1.168 RG 1.169 RG 1.171 2

Background

Endorse IEC standards as an alternative to IBR IEEE Standards in accordance with 10 CFR 50.55a(z)

Evaluate such endorsements impact current regulatory guidance, including the current regulatory guides, with respect to other IEC and/or IEEE standards

  • Initial evaluation determined:

Endorsing IEC standards as alternatives to IBR standards may not be useful for operating reactors Several digital I&C platforms that have been developed using IEC standards were previously approved by the NRC for use in safety applications of nuclear power plants Potential to endorse, via most likely a regulatory guide, a subset of IEC standards with focus on software development as one potential way to satisfy regulatory requirements related to quality 3

Issues to be Addressed

  • Software development standards are the focus for endorsement This effort evaluates whether IEC 60880 could be endorsed as one way to meet regulations related to quality Identify which other IEC standards need to be endorsed if there are gaps within IEC 60880 for a certain topic Significant differences in the classification of systems in IEC and IEEE nuclear standards will need to be addressed o Use IEC 61266 to categorize functions performed and corresponding safety class; or o Limit application to safety-related systems that is equivalent to Class 1 systems performing Category A functions 4

IEC 61513 and IEC Software Standards IEC 61513 IEC 61266 Classification IEC 60880 IEC 62566 IEC 60987 IEC 62645 IEC 60780 IEC 61000 Software HDL/FPGA Hardware Cyber Qualification EMI/RFI IEC 61500 Communication IEC 62340 Common Cause Failure 5

IEC 60880 software for category A functions 6

Preliminary Evaluations of Software Regulatory Guides

  • How have we done the evaluation?

Review of the underlying regulations Review of the what is needed to demonstrate that the regulations are being meet and the system is safe Evaluation of the method by which the regulatory guides and endorsed IEEE standards provide that assurance Review the IEC 60880 criteria Determine whether IEC 60880 provides equivalent demonstration of safety and/or if additional criteria are needed 7

Preliminary Review of Software Verification and Validation 8

Software Verification and Validation

  • RG 1.168 endorses IEEE Std 1012-2004 for meeting the NRCs requirements for verification and validation of safety-related system software.

It establishes a minimum set of V&V activities for software based on a Software Integrity Level (SIL) classification.

RG 1.168 states that software used in nuclear power plant safety systems should be assigned integrity level 4 or the equivalent, as demonstrated by a mapping between the applicant or licensee approach and integrity level 4.

9

Software Verification and Validation

  • IEC 60880 includes software verification (Section
8) and software integration (Section 10) criteria for nuclear software applications used to perform safety functions with the highest category of safety (i.e., Category A).
  • NRC staff evaluated IEC 60880 to determine if the criteria provided in this standard meet the underlying V&V regulatory requirements cited in RG 1.168.

10

Examples of Evaluation

  • Examples of criteria within IEC 60880 that support compliance with underlying regulatory requirements for software verification:

IEC 60880, Section 8 provides criteria for establishing independence between a verification team and the development team o These criteria provide an acceptable way of meeting the quality requirements in IEEE 603 Section 5.3, Quality o These criteria are consistent with the criterion of IEEE Std 7-4.3.2-2003, Section 5.3.4, V&V Independence Requirements and are consistent with IEEE 1012 Section 5 and Section C.3 of RG 1.168 Section 8 also includes requirements for performing verification of outputs from each of the development life cycle phases o These activities confirm the adequacy of the software requirements specification in fulfilling the system requirements o These requirements are compatible with IEEE Std 1012-2004, Section 5, Software V&V Process, and thus provide an acceptable means of establishing quality in safety I&C systems as required by IEEE 603-1991, Section 5.3 11

Examples of Evaluation Contd.

  • Example of criteria within IEC 60880 that meets the underlying regulatory requirements for software validation:

Section 10 provides criteria for validating design of an integrated system These criteria provide an acceptable way of meeting the objectives of the validation activities prescribed in IEEE Std 1012-2004, Table 2 12

Examples of Evaluation Contd.

  • Examples where criteria in IEC 60880 are not sufficient to provide one acceptable way of meeting underlying regulatory requirements for verification and validation:

Rather than prescribe a minimum set of V&V activities to be performed for each phase of development, IEC 60880 is results oriented Criteria within this standard are the resulting characteristics of the safety system Users of this standard is allowed to decide which V&V activities would be needed to achieve the required results The IEC standard does not include tables of V&V activities or any equivalent list of activities to be performed for safety software Mapping of activities performed to activities prescribed by this standard cannot be performed Documentation focuses on system design performance results rather than completion of specified V&V activities 13

Software Verification and Validation (IEC 61513)

  • IEC 61513 includes criteria for deriving the I&C requirements from the plant safety design base and design of the entire I&C architecture and assignment of the I&C functions Several V&V activities are specified within this criterion.

Examples include:

Performance of Requirements Traceability Assignment of Software Integrity Levels Verifying requirements specifications to ensure correct assignment of functions to systems

  • NRC staff is evaluating IEC 61513 to determine if the criteria provided in this standard can be used as a means of meeting the underlying V&V regulatory requirements cited in RG 1.168 14

Verification Definition Differences

Process of evaluating a system or component to determine whether the products of a given development phase satisfy the conditions imposed at the start of that phase Process of providing objective evidence that the software and its associated products:

o Conform to requirements (e.g., for correctness, completeness, consistency, accuracy) for all life cycle activities during each life cycle process (acquisition, supply, development, operation, and maintenance);

o Satisfy standards, practices, and conventions during life cycle processes; and successfully complete each life cycle activity; and o Satisfy all the criteria for initiating succeeding life cycle activities (e.g.,

building the software correctly)

  • Verification (IEC 60880)

Confirmation by examination and by provision of objective evidence that the results of an activity meet the objectives and requirements defined for this activity 15

Validation Definition Differences

Process of evaluating a system or component during or at the end of the development process to determine whether it satisfies specified requirements Process of providing evidence that the software and its associated products:

o Satisfy system requirements allocated to software at the end of each life cycle activity; o Solve the right problem (e.g., correctly model physical laws, implement business rules, use the proper system assumptions); and o Satisfy intended use and user needs Validation (IEC 60880)

Confirmation by examination and provision of other evidence that a system fulfils in its entirety the requirement specification as intended (e.g., functionality, response time, fault tolerance, robustness) 16

Preliminary Review of Software Configuration Management 17

Software Configuration Management

  • RG 1.169 endorses IEEE Std 828-2005 for meeting the NRCs requirements for configuration management for safety-related system software

No minimum set of activities established for configuration management for nuclear safety software applications.

RG 1.169 identifies the minimum set of activities for nuclear safety software applications

  • IEC 60880 provides criteria for software configuration management for nuclear software applications used to perform safety functions with the highest category of safety (i.e. Category A)
  • Staff evaluated IEC 60880 to determine if the criteria provided in this standard meet the underlying configuration management regulatory requirements cited in RG 1.169 18

Examples of Evaluation

  • Examples where criteria within IEC 60880 that meet the underlying regulatory requirements for configuration management include Sections 5.6.4, 5.6.5, 5.6.7, and 5.6.9 These sections provide criteria for identifying software such as the following:

o Each produced version of each software entity shall be uniquely identified o It shall be possible to identify the relevant versions of all software documentation associated with each software entity o It shall be possible to identify the versions of all software entities which together constitute a complete version of the final product These criteria provide an acceptable way of meeting:

o 10 CFR Part 50, Appendix A, GDC 1 o IEEE Std 603-1991, Clause 5.3 requirements for quality 19

Examples of Evaluation Contd.

  • Examples where criteria within IEC 60880 that meet the underlying regulatory requirements for configuration management include Sections 6.4.2, 7.1.1.9, 7.1.4.3, and 7.4.1 These sections provide criteria to ensure configuration management of software documentation such as the following:

o The software requirements specification shall be presented according to a standard whose formality should not preclude readability o Comprehensive and clearly written documentation shall be provided o The configuration data shall be documented These criteria provide an acceptable way of meeting:

o 10 CFR Part 50, Appendix A, GDC 1 o IEEE Std 603-1991, Clause 5.3 requirements for quality 20

Examples of Evaluation Contd.

  • Examples where criteria in IEC 60880 were insufficient to provide one acceptable way of meeting underlying regulatory requirements for configuration management:

Section 5.6, including its subsections references the system level configuration management requirements in IEC 61513, including provision of a configuration management plan or the quality assurance plan Control of software vendors supplying safety systems software not addressed in IEC 60880 Software configuration audits not addressed in IEC 60880 21

Preliminary Review of Software Unit Testing 22

Provisions for Software Unit Testing

  • Software Unit Testing is addressed in RG 1.171 RG 1.171 endorses, with clarifications and additions, IEEE 1008-1987 IEEE Standard for Software Unit Testing RG 1.171 endorses the 2008 edition of IEEE Std 829 IEEE Standard for Software Test Documentation, in lieu of the version cited in IEEE 1008-1987 23

Regulatory Requirements for Software Unit Testing

  • The regulatory requirements cited in RG 1.171 include:

Requirements related to quality and documentation Requirements for periodic testing Requirements related to the development and testing of design changes Requirements for documentation of test plans and results No explicit requirements for unit testing 24

Provisions for Software Unit Testing in IEC 60880

  • Annex E (nonmandatory) presents general recommendations for system testing
  • Annex B (mandatory) presents considerations for unit and integration tests (B4.g), and for testing related to modifications (B1.d)
  • Various sections within IEC 60880 contain provisions for:

Testing modifications; Selection of test cases; and Documentation of test plans and results Requirements are generally not very specific and consensus must be attained as to whether they are adequate to meet the regulatory requirements cited in RG 1.171 25

Next Steps 26

Stakeholder Engagement

  • NRC would like to work with an industry working group Help identify a subset of the suite of IEC standards that would be of most use to industry Provide early feedback on NRC strategy for endorsement Provide feedback on overlaps, gaps, and possible challenges to endorsement
  • Table top/example review Identify a specific system or platform to evaluate the new process to ensure the guidance is practical and well understood Make a selection as soon as possible Review will be done over the course of a few months, so will need to be appropriately scoped 27

Proposed Project Plan (Key Milestones)

  • Public meeting to engage stakeholders on the proposal, identify participants for the working group, and identify a subset of the suite of IEC standards that would be of most use to industry (Done)
  • Formalize project plan and select IEC standards to include (Done)
  • Work with OGC to determine appropriate guidance document (Done)
  • Coordinate with IEEE and IEC (Done)
  • Conduct review of IEC nuclear (45a) standards to determine overlaps, gaps and possible challenges with endorsement of IEC 61513 suite of standards (in-progress)
  • Select system or platform for example review (Fall 2019)
  • Develop possible solutions to previously identified concerns associated with endorsement of IEC standards (Fall 2019)
  • Complete example review (Early 2020)
  • Brief ACRS (Early 2020)
  • Publish draft guidance for public comment (Spring 2020) 28

Discussion 29

Acronyms

  • CFR: Code of Federal Regulations
  • GDC: General Design Criterion
  • I&C: Instrumentation and Controls
  • IEC: International Electrotechnical Commission
  • IEEE: Institute of Electrical and Electronics Engineering
  • NRC: Nuclear Regulatory Commission
  • RG: Regulatory Guide
  • SIL: Software Integrity Level
  • V&V: Verification and Validation 30
Retrieved from "https://kanterella.com/w/index.php?title=ML19233A215&oldid=1892290"