ML24023A619
| ML24023A619 | |
| Person / Time | |
|---|---|
| Site: | National Bureau of Standards Reactor |
| Issue date: | 10/23/2023 |
| From: | US Dept of Commerce, National Institute of Standards & Technology (NIST) |
| To: | Office of Nuclear Reactor Regulation |
| Shared Package | |
| ML24023A617 | List: |
| References | |
| Download: ML24023A619 (1) | |
Text
NIST Center for Neutron Research Reactor Operations and Engineering (NCNR -ROE )
Assessment of NCNR -ROE Procedure s Pro gram
October 23, 2023
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
Table of Contents
EXECUTIVE
SUMMARY
................................................................................................................... 4
BACKGROUND INFORMATION...................................................................................................... 5 The Event.................................................................................................................................................................. 5
THE TEAM........................................................................................................................................ 6
METHODOLOGY.............................................................................................................................. 6
ACKNOWLEDGEMENTS................................................................................................................. 7
ASSESSMENT RESULTS................................................................................................................ 7
PERFORMANCE -BASED AND COMPLIANCE -BASED ASSESSMENTS..................................... 7 Performance -Based Assessment........................................................................................................................... 8 Compliance -Based Assessment............................................................................................................................. 8 NCNR -ROE Administrative Procedure Program Review...................................................................................... 8 Procedure Quality Review (Objective 1)................................................................................................................ 9 Current Status...................................................................................................................................................... 9 Gaps.................................................................................................................................................................. 10 Vulnerability....................................................................................................................................................... 10 Review the Engineering Modification Process and How Impact on Procedures is Communicated (Objective 2)............................................................................................................................................................ 10
ASSESSMENT FINDINGS.............................................................................................................. 10 Positive Observations (PO) for Performance -Based Assessment.................................................................... 10 Positive Observations (PO) for Compliance -Based Assessment...................................................................... 11 Area in Need of Attention (ANA)........................................................................................................................... 11 Area for Improvement (AFI)................................................................................................................................... 11
ADDITIONAL IMPROVEMENTS ITEMS........................................................................................ 12 Establish Site Wide Procedure Change Request (PCR) System....................................................................... 12 Issue.................................................................................................................................................................. 12 Recommendations............................................................................................................................................. 12 Establish Qualification Process for Procedure Writers...................................................................................... 14 Issue.................................................................................................................................................................. 14 Recommendations............................................................................................................................................. 14 Additional Items Identified Based on Discussions with Site Personnel........................................................... 15
Page 2 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
Attachment 1: Technical Procedure Checklist.................................................................................................... 16 Attachment 2: Administrative Procedure Checklist............................................................................................ 18 Attachment 3: Assessment Team......................................................................................................................... 20 Attachment 4: Contacts and Interviews............................................................................................................... 24 Attachment 5: Gap Analysis.................................................................................................................................. 25 Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures........................ 52 Attachment 7: Procedures that Should Not be Implemented in the Field as Written..................................... 69 Attachment 8: Reference Documents.................................................................................................................. 74
Page 3 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
EXECUTIVE
SUMMARY
This report provides the conclusions of an independent and comprehensive assessment of the Procedures P rogram at the NIST Center for Neutron Re search Reactor Operations and Engineering (NCNR -ROE ), which contributed to the event occurring February 3, 2021, at NCNR -
ROE. The assessment will be used to identify areas for improvement, corrective actions, and other improvement opportunities. The assessment identified areas for improvement and corrective actions, and other improvement opportunities related to the NCNR -ROE Procedures Program.
The assessment was performed by a team of five independent nuclear consultants, with operations, training, and nuclear procedures development and revision experience, including procedure upgrade programs. The team members experience includes extensive involvement in the implementation and management in o perations of Naval reactors, Department of Energy (DOE) nuclear facilities, and commercial nuclear power plants.
The primary objectives of the assessment were to:
- First, perform a review of NCNR -ROE procedures using checklists with criteria derived from industry norms.
- Second, assess the NCNR -ROE process for changing to procedures that involve the use of engineered items and how those changes are communicated throughout the organization,
and provide recommendations to address any weaknesses and opportunities for improvement identified through the assessment.
The assessment team reviewed the NCNR -ROE Procedures Program against industry standards.
The team then conducted collegial reviews of the results to determine final conclusions and to recommend actions to correct the identified findings.
The assessment team noted the NCNR -ROE Operations staff has implemented actions to date, such as
- NCNR -ROE modified Administrative Requirement (AR) 5.0, Procedure Use and Adherence, which guides conduct of operations, to strengthen the oversight role supervisors must play and to require that all personnel be trained. The required procedure use and adherence training has been completed.
- Drafted and approved AR 5.1, Procedure Writers Guide, related to procedure writing, which includes addressing improvements in procedure quality to comply with PPA AP -907 -
005, Procedure Writers Manual.
- Identified procedures that will be updated according to new guidance in AR 5.1 prior to reactor startup.
- Implemented AR 1.1, Human Performance Tools, related to improving training on a continuous basis.
- Modified Operating Instruction (OI) 1.1, Reactor Startup, to provide detailed guidance on evaluation of abnormal fluctuations in nuclear instrumentation.
NCNR -ROE has made p rogress upgrading the P rocedure P rogram, but the lack of resources is impacting the ability to complete these actions in their entirety in a timely manner.
Page 4 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
O verall, the assessment concluded, while the NCNR -ROE staff has a vision for a Procedure Program, the lack of dedicated resources and lack of knowledge of Procedure Program requirement s prevents the implementation of an effective Procedure Program. This presents a vulnerability to NCNR -ROE as the program does not contain the necessary rigor to ensure the safe operation of the facility for the long term. The assessment team recommend s NCNR -ROE obtain external resources to support implementing Procedure Program improvements and upgrading procedures based on the results of this assessment in the near term which will allow NCNR -ROE personnel to maintain these procedures long term.
BACKGROUND INFORMATION
The Event
On February 3, 2021, the NBSR reactor was conducting a normal start up after completing a refueling on January 4, 2021. Upon approach to full power, the plant experienced a sudden drop in power level and rapid increases on several radiation monitors, including fission products monitor (RM 3 -2) and the stack monitor (RM 4 -1). A major scram was automatically initiated when the stack monitor reached its setpoint of 50,000 counts per minute, and an immediate activation of the confinement isolation system sealed off confinement to prevent or limit release of radioactive material from the facility. NCNR -ROE and NIST staff took emergency actions, including declaring an Alert condition, notifying the Nuclear Regulatory Commission (NRC) as required by regulations, and stabilizing the plant. The Alert declaration was subsequently downgraded based on analysis of radiological sampling revealed that the radiological conditions at the Site Boundary and at the reactor confinement exhaust stack did not meet the conditions fo r the Alert declaration. The emergency was terminated later that day, and the staff began recovery actions.
A post event root cause analysis was conducted and two of the root causes identified w ere :
- Procedures as written do not capture necessary steps in assuring elements are latched.
- Procedural compliance was not enforced.
The Confirmatory Order of August 1, 2022, directed the following actions pertaining to procedures:
- Prior to any restart, NCNR -ROE will develop, implement, and maintain a written procedure that covers procedural use and adherence in accordance with the most recent version of INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence.
- Procedures referenced in paragraph 1.d of Section III of this CO will be subject to inspection prior to any restart.
- Within 30 months of issuance of the CO, NCNR -ROE will develop, implement, and maintain all procedures involving reactor operations activities in accordance with the most recent version of PPA AP -907 -005, Procedure Writers Manual.
The assessment validated the conclusions reached by the post -event NCNR -ROE root cause evaluation regarding the Procedures Program at NCNR -ROE.
Page 5 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
THE TEAM
The team was comprised of five independent consultants with commercial nuclear industry, Naval Reactors, and Department of Energy (DOE) experience and backgrounds in senior management as well as backgrounds in operations, nuclear operating procedures including normal, alarm response procedures, abnormal and emergency operating procedures, and operations training.
This experience includes the implementation of multiple procedure development and up -grade programs in both government and commercial nuclear industr ies, which were developed and implemented in accordance with standards set forth by the Procedure Professional Association (PPA), Nu clear Regulatory Commission (NRC), and the Institute of Nuclear Power Operations (INPO). Refer to Attachment 3 for a description of the team members background and experience.
METHODOLOGY
In accordance with DOE Guide 414.1 -1c (Management and Independent Assessments Guide),
Section 3.7, this assessment was both Compliance -Based and Performance -Based.
First, Procedure Quality Checklist s (Attachments 1 and 2 ), derived from accepted industry norms, were developed for use in the reviews. App roximately 300 procedures which have been upgraded to the AR 5.1 requirements were pr ovided by NCNR -ROE for the team to review. Each procedure was reviewed against the criteria in the checklist. To assist with the potential procedure upgrade, each procedure was graded according to the following set of criteria:
- Level 1 - The procedure is technically adequate. It needs to have format and Human Error (HU ) Error Prevention Tools inserted.
- Level 2 - The procedure is technically adequate. It needs acceptance criteria research and then inserted, format and HU Error Prevention Tools required.
- Level 3 - The procedure cannot be implemented without tribal knowledge, but sufficient documentation is available (Technical manuals, Safety Analysis Report ( SAR ), etc.) to rewrite with a Verification and Validation ( V&V ).
- Level 4 - The procedure is not implementable in the field.
- Level 5 - Critical Operation Procedures where a number of operators need to be involved but a procedure writer could be a great help.
- Level 6 - Procedures that have been changed to facilitate continued operation with material deficiencies (Level 1 Ops Work Around).
- Level 7 - Procedure is Acceptable and meets current industry and facility standards. (ACC)
Second, assess the NCNR -ROE process for changing procedures that involve the use of engineered items and how those changes are communicated throughout the organization, and provide recommendations on actions to address any weaknesses and opportunities for improvement identified through the assessment.
Page 6 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
ACKNOWLEDGEMENTS
The team received excellent support from NCNR -ROE staff. NCNR -ROE staff questioning and openness to understand the team perspective is positive.
ASSESSMENT RESULTS
The assessment identified numerous gaps between the NCNR -ROE Procedure Program and industry standard documents. The following sections of this report provide the detailed results of this assessment and recommendations, as appropriate, for each of the NCNR -ROE Procedure Program documents identified above. The procedures that have been upgraded to date that do not fully meet industry standards should be updated to comply with industry standards.
The following actions are recommended to be completed prior to upgrading any further procedures:
- Obtain n ew procedure template s prior to upgrading any procedures.
PERFORMANCE -BASED AND COMPLIANCE -BASED ASSESSMENTS
The performance -based assessment is based on interviews with facility staff and observations of work being performed. The compliance -based assessment is based on review of upgraded procedures against applicable industry guidance. The Assessment Report uses the following definitions to classify the observations of each of the categories:
- Area of Strength - This term is used to characterize demonstrated performance in a program or process element within an area being assessed that is exceptionally effective in achieving its desired results. An Area of Strength is a program, process, or activity of such a high quality that it could serve as an example for other similar elements.
- Positive Observation (PO) - This term refers to a noteworthy observation that does not rise to the level where it would be considered as a strength.
- Area in Need of Attention (ANAs) - This term is used to identify a performance, program, or process element that is sufficient to meet its basic intent; however, management attention is required to achieve full effectiveness and consistency. ANAs are not normally identified or addressed in action plans submitted to the NRC but are brought to management attention for consideration and possible entry into the NCNR -ROE Corrective Action Program or alternative tracking process.
- Area For Improvement (AFIs) - This term is used to characterize an identified performance, program, or process element that requires significant improvement to obtain the desired results in a consistent and effective manner. AFIs identified in the Assessment Report will be addressed by the Action Plan Submitted to the NRC.
Page 7 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
Performance -Based Assessment
The assessment team i nterview ed eight Operations personnel (SRO/RO/Equipment Operator) and Procedures Program Owner. The team observed a reactor startup and shutdown. D uring the Nuclear Safety Culture Assessment, a Pre -Job brief was observed prior to a reactor startup.
Compliance -Based Assessment
This assessment consisted of two elements:
- 1. One element reviewed the NCNR -ROE Procedure Program against the industry standard documents. The industry standard documents used during this review were:
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence
NCNR -ROE Administrative Procedure Program Review
An assessment was performed of Procedure Program Procedures; AR 5.1, Procedure Writers Guide (Rev. 2), AR 5.0, Procedure Use and Adherence (Rev. 2), AR 5.2, Document Routing Policy (Rev. 1), and the Procedure Templates. The intent of the assessment was to review the NCNR -ROE Procedure Program against industry standard documents that include the necessary attributes required to develop a Procedure Program that has the necessary rigor to ensure safe operation of the facility for the long term. Document s used during this review were:
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence A detailed review of the AR 5.0 series Procedure Program documents is provided in Attachment 6, Assessment of NCNR -ROE Administrative Procedure Program Procedures, and it identified many gaps when compared to PPA and INPO guidance documents. These gaps represent a vulnerability that procedures may not be written in a manner that minimizes the potential for error during performance. Specific suggestions are provided in Attachment 6; however, the overall suggestion is that AR 5.0, AR 5.1, and AR 5.2 be upda ted to align with the requirements of the PPA and INPO guidance.
If it is determined that a Procedure Upgrade Project is needed, the AR 5.0 series Procedure Program documents will need to be updated prior to upgrading any procedures. Additionally, new templates should be developed to support upgrading procedures per the updated AR 5.0 series guidance.
Page 8 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
Procedure Quality Review (Objective 1)
- Procedure Quality Checklists ( Attachments 1 and 2) were developed from accepted industry norms, PPA AP -907 -005, Procedure Writers Manual, and PPA AP 907 -001, Procedure Process Description, Attachment 2, Error Tra ps, for use in the reviews. Each procedure was reviewed against the criteria in the checklist. The completed checklist s will be provided separately from the report.
- It is not practical to review the procedures against AR 5.1, Procedure Writers Guide, as AR 5.1 is currently not aligned with PPA AP -907 -005.
- The procedures reviewed were provided by NCNR -ROE.
Current Status
Of 282 procedures reviewed, all except 1 7 were deemed technically adequate for use, although most require additional measures to be incorporated, such as human error prevention tools, acceptance criteria and formatting before implementation. Only four were assessed as being in accordance with industry standards, see table below. Attachment 7, Procedures that Should Not Be Implemented in the Field as Written, provides details on the 1 7 procedures assessed as not implementable in the field.
Table 1, Results of Procedure Reviews Using the Procedure Quality Checklists Type of Procedure Total Reviewed Level 1 Level 2 Level 3 Level 4 Level 7 OI 89 86 82 0 8 0 4 AP 52 45 44 0 1 0 0 IC 44 44 2 10 25 10 0 MP 40 34 2 29 26 2 0 TSP 34 31 0 28 19 2 0 RP 14 11 7 4 3 1 0 HP 8 8 2 1 5 0 0 EI 5 4 3 0 1 0 0 TP 3 3 3 0 0 0 0 AR 24 16 16 0 1 2 0 313 282 161 72 89 1 7 4
- Level 1 - The procedure is technically adequate. It needs to have format and HU Error Prevention Tools inserted.
- Level 2 - The procedure is technically adequate. It needs acceptance criteria research and then inserted, format and HU Error Prevention Tools required.
- Level 3 - The procedure cannot be implemented without tribal knowledge, but sufficient documentation is available (Technical manuals, SAR, etc.) to rewrite with a V&V.
- Level 4 - The procedure is not implementable in the field.
- Level 7 - Procedure is Acceptable on meets current industry and facility standards. (ACC)
Page 9 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
Gaps
- Attachment 4, Gap Analysis, provide s the Current Status and the Gap between the PPA AP -907 -005 and the procedures that have been developed using the AR 5.1.
Vulnerability
The facility has revised many procedures since the new revision to AR 5.1 Rev. 2 was approved.
The Review Team reviewed approximately 300 of the revised procedures, and all but four were found to have some type of deficiency. These procedures lack Human Performance tools/ guidance to reduce the potential of performance errors. AR 5.1, Procedure Writers Guide is not aligned with PPA AP -907 -005, Procedure Writers Manual and it is evident in the approximately 300 procedures that have been revised, since the AR 5.1 revision in November 2021.
Review the Engineering Modification Process and How Impact on Procedures is Communicated (Objective 2)
The team reviewe d engineering guidelines and found they clearly define procedure changes as required elements of an engineering design change {Engineering Change Notification (ECN)}
package, that must be made as a part of ECN implementation. (Reference NBSR -0003 -DOC -06 Guidelines For Engineering Changes).
The processes appear whole and during NCNR -ROE team meetings, a high level of personnel awareness of procedure changes as a result of ECNs, is observable. Based on the signatories to an ECN, the Chiefs are responsible to communicate the impactful changes.
Two minor Enhancements:
- Enhancement - The nuclear power industry developed IP -ENG -001, Standard Design Change, which provides a streamlined and graded approach to the design change process.
This process may be helpful as a benchmark for engineering procedure upgrades in the future.
ASSESSMENT FINDINGS
Positive Observations (PO) for Performance -Based Assessment
- (PO -1) - Operators understand and are able to articulate the difference between Continuous Use, References Use, and Information Use as defined in AR 5.0.
- (PO -2) - Operators understand the requirement to stop the activity and contact supervision if problems are encountered during procedure execution.
- (PO -3) - SharePoint reports are available to all personnel to see status of procedures in the review and approval process.
- (PO -4) - In addition, during the Training Program Assessment the team observed one reactor startup and shutdown in preparation for the upcoming NRC exam (License
Page 10 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
candidate practice for upcoming NRC exam). The observation received a Positive grading result with implementing those continuous use procedures.
- (PO -5) - The Pre -Job brief observed prior to a reactor startup o n April 4, 2023, was thorough. The Chief, Reactor Operations ( CRO ) was in attendance. Procedure use was good. Issues were identified and discussed by the CRO. One occasion of background noise by others in the Control Room ( CR ) was addressed and use/expectation of Alarm Response Procedures ( ARPs ) was also communicated. The CRO also prompted the reperformance of the Estimated Critical Position due to a slight temperature change.
Positive Observations (PO) for Compliance -Based Assessment
- (PO -6 ) - Several of the Operating Instructions (OIs) procedures have information that provides a brief description of the system and some pictures of the controls. Although having this information in an OI may not be the proper place for this type of information, the Assessment Team found it helpful during the review of the associated procedures.
Examples are OI 4.1.0, Helium Sweep System Checklist, and OI 4.3, Ventilation System Information.
- (PO -7 ) - The engineering guidelines clearly define procedure changes as required elements of an engineering design change {Engineering Change Notification (ECN)} package, that must be made as a part of ECN implementation. The processes appear whole and during NCNR -ROE team meetings, a high level of personnel awareness of procedure changes as a result of ECNs, is observable. Based on the signatories to an ECN, the Chiefs are responsible to communicate the impactful changes.
Area in Need of Attention (ANA)
- (ANA -1) - The Procedure template does not provide automation/macros to simplify use of the template by providing single click macros that perform a multi -step function. This results in writers having to spend significantly more time formatting the document when inserting content/text. This becomes a burden to the writer that can lead to errors in documents and can impact performance of the document. Automation/macros will also help enforce consistency in format per the writers manual across all the documents. Procedure template s needs to include automation/macros and align with a revised AR 5.1, Procedure Writers Guide.
Area for Improvement (AFI)
- (AFI -1) - The Writers Guide and Procedure Use & Adherence Process Procedures do not reflect the full applicable scope of PPA/INPO 11 -003, and therefore, procedures written to those standards lack both structure and technical detail to effectively prevent or mitigate human error.
o GAP: there are 30 discrete gaps and differences between the standards (PPA/INPO 11 -003) and the facilitys documentation, (ranging from layout, actions in notes, Reference and Branching, Required Sections of Procedures, etc.). The issues identified by the assessors were frequently validated by interviews. These have been previously discussed with the process owner and will be included in the final report.
Page 11 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
- (AFI -2) - Formal Training for both writers and users is insufficient to ensure compliance and error prevention.
o GAP: Writer Training o GAP: PU&A Training o GAP: 50.59 Training o GAP: Use of SharePoint for procedure revision process
- (AFI -3) - Observation Guidance - The observation program checklist focuses more on procedure content rather than the behaviors of good PU&A.
o GAP: AR 5.4, Observation Program and Checklist, does not contain any specific criteria related to PU&A (e.g., revision check prior to procedure use, place keeping being used properly, steps performed in specified sequence, etc. )
- (AFI -4) - 1 7 Newly Published Procedures are recommended to not be used until rewritten.
They are listed in Attachment 7.
- (AFI -5) - Conflicting personnel responsibilities and availability of consistent resources is affecting the quality and quantity of procedure throughput.
- (AFI -6) - None of the reviewed TSPs/ICPs include a reference to calculational basis for acceptance criteria (typically as a developmental reference). This is required to ensure that the facility is operated within the boundaries of the design and licensing basis. If no such calculations exist, then a different issue with the facility design and licensing basis is identified.
ADDITIONAL IMPROVEMENTS ITEM S
Establish Site Wide Procedure Change Request (PCR) System
Issue
Currently NCNR -ROE has no method or process that allows the procedure user to feedback issues related to procedures. Procedure s will likely never improve without feedback from the procedure users.
The feedback system should include a mechanism to provide feedback to the initiator on the status of the feedback.
During INPO evaluations, the evaluators review Procedure Change Requests (PCRs) to determine the health of the facilitys procedures. T hree main areas are examined :
- Number of open feedbacks
- Age of open feedbacks
- Priority system for feedbacks
Recommendations
Develop an electronic process that provides any procedure user the ability to submit a Procedure Change Request (PCR) to include the following:
Page 12 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
- Available to all site personnel.
- Available on the facility intranet or other electronic method.
- Priority system is used for each PCR.
- Each PCR should only include activities associated with one procedure.
- A process to feedback to initiator that PCR has been closed and the reason for closing.
- All work performed on a procedure should require a PCR.
Establish a Priority s ystem for each Procedure Change Request (PCRs) using something similar to T -E -A -M.
T - Technical E - Enhancement A - Administrative Correction M - Modification/Technical Specification Change
- Technical (T) o Procedure cannot be performed as written.
o Procedure works but has caused significant identifiable technical or functional challenges.
o Involves Nuclear/Personnel Safety issue.
- Enhancement (E) o A better way to perform a task.
o Does the proposed change provide a Cost Benefit:
§ Reduce Radiological dose.
§ Involve Human Performance issues.
§ Reduce the amount of time equipment is out of service.
§ Reduce outage duration.
§ Shorten amount of time to perform evolution.
§ Close PCRs that do not have a cost benefit.
- Administrative Correction (A) o Obvious typographical errors (e.g., obvious step numbering errors or changes of referenced steps, sections, attachments, forms, etc.), if adequate information is available to verify correct numbers.
o Incorrect personnel/position titles due to approved organizational changes.
o Obvious discrepancies between component tag and the equipment identification if adequate information is available to verify correct component/equipment identification.
o Incorrect References or missing References which should have been listed.
o Updates to Commitment information or adding commitment source noting which should have been previously incorporated.
Page 13 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
o Updates to or modifications of format when it does not affect performance of the instructions.
- Modifications/Technical Specifications Change (M) o Procedure changes that are dependent on another change taking place before they can be implemented.
- Other documents containing information on Process Change Request include:
o PPA AP -907 -001, Procedure Program Description o PPA AP -907 -001 -001, Procedure Performance Metrics
Establish Qualification Process for Procedure Writers
Issue
Currently a Facility Qualification Program for Procedure Writers does not exist, and the process does not provide the tools the writer will need to be successful.
The current process allows anyone to make a procedure change or revision but does not require any qualifications or training. During the review of the facilities procedures, the team observed that procedure writers are not equipped to properly use the procedure writing template, many are not proficient in the use of Microsoft Word, many are not aware of human performance tools that are necessary for a quality procedure, and many are unaware of the requirements of AR 5.1, Procedure Writer's Guide.
Recommendations
Procedure Writer Certification is currently available and endorsed by the Procedure Professionals Association (PPA) and can be used for the basics for procedure writers, planners, procedure reviewers and management.
This course is available both remotely and onsite for 10 to 15 individuals at a time. It is approximately a 16 - to 18 -hour course and is provided over several days.
Establish a Procedure Writer Qualifications Program. The qualification program at a minimum should include the following modules:
- Administrative Procedures to Review
- Electronic Support Tools (SharePoint)
- Procedure Writing (Procedure Writers Guide & PU&A procedure should be part of this)
- Types of Procedure Changes (Procedure Process) o Major Revision o Minor Revision o Administrative Revision o Temporary Change
Page 14 of 74
Independent Nuclear Safety Consultant Task 0 5 - Procedures Program Independent Assessment Report
o Cancellation
- Software Proficiency (How to use the template)
Additional Items Identified Based on Discussions with Site Personnel
- There is no document hierarchy prescribed in the Procedures program documents.
Organizations typically have a formal document hierarchy showing the relationship between Policies, Program Documents, and Procedures.
- There is no official document numbering scheme prescribed in the Procedures program documents. Organizations typically have a document numbering scheme to maintain consistency in numbering of various document types.
- The only Change Management action in the procedure change process is for affected parties to acknowledge (via a signoff sheet) that they have reviewed the newly published procedure. Other typical change management actions (besides the communication action) should be considered such as:
o E ffect on training o Impact to scheduled work o Impact to other departments o Impact to other documents o Impact to routinely used products or services (e.g., forms, databases, software)
- Interviewees stated no formal training is provided on Procedure Use and Adherence (PU&A). They are expected to read and understand AR 5.0, Procedure Use and Adherence, which does not meet the requirements of formal training. While training was provided in September 2021, employees hired after that have not received training. There does not seem to be a plan of initial training or continuing training for PU&A going forward.
Page 15 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report A ttachment 1: Technical Procedure Checklist
Procedure Number and
Title:
Reviewer: Rev:
No. Topic/Checklist Question Yes No N/A
- 1. Is the Level of Use - Continuous or Reference Use?
Comments:
- 2. Does the Table of Contents list major section headings, and attachments with page numbers?
Comments:
- 3. Does Purpose & Scope, References, Definitions, Precautions & Limitations and Records section have no actions in steps?
Comments:
- 4. Does that Purpose / Scope adequately describe the work performed and any applicability of the procedure?
Comments:
- 5. Does the Reference section include reference number and name with revision date/number for Developmental References? With NO Revision date/number for Implementing References?
Comments:
- 6. Are Definitions listed in alphabetical order?
Comments:
- 7. Do Precautions & Limitations apply to the entire procedure?
Comments:
- 8. Do Prerequisites describe the activities to be completed and requirements to be met prior to procedure performance?
Comments:
- 9. Are In -field decisions without clear guidance avoided?
Comments:
- 10. Does conditional logic (IF/THEN, WHEN/THEN) specify conditions and required actions?
Comments:
- 11. Do steps have only one Action Verb or related actions (Press & Hold) per step?
Comments:
- 12. Steps with multiple components are they listed as substeps or bullets?
Comments:
- 13. Does Precautions, Limitations, Notes, Cautions, and Warnings have no actions either explicit or implicit?
Comments:
- 14. Are Notes, Cautions and Warnings always prior to the affected step?
Comments:
- 15. Are Notes, Cautions and Warnings presented with Notes first and Warnings prior to the affected step?
Comments:
- 16. Are acronyms spelled out the first time they are used in the procedure or section?
Comments:
- 17. Does procedure minimize the use of Referencing and Branching?
Comments:
- 18. Does the procedure use only Verification Techniques when required?
Comments:
Page 16 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report A ttachment 1: Technical Procedure Checklist
- 19. Are simple word words and short concise sentences used in the procedure?
Comments:
No. Topic/Checklist Question Yes No N/A
- 20. Is the format, layout and writing style the same throughout the procedure?
Comments:
- 21. Are non -value added information in Precautions, Prerequisites, Notes, Cautions, Warnings, and Steps avoided? (e.g. Review the Safety Manual)
Comments:
- 22. Are all cross references accurate?
Comments:
- 23. Are procedures referenced identified by both the procedure number and the procedure title? Are these procedures listed in the Reference section?
Comments:
- 24. Are functional & acceptance criteria identified?
Comments:
- 25. Is a calculational basis provided for all acceptance criteria?
Comments:
- 26. Is a second check used for all calculations?
Comments:
- 27. Do attachments/appendices include Page X of Y of the overall procedure and include Page x of y of the individual attachment/appendix?
Comments:
- 28. Are all Attachments/Appendices referenced within the body of the procedure.?
Comments:
- 29. Grading criteria. Check all that applies:
Level 1 - The procedure is technically adequate. It needs to have format and HU Error Prevention Tools inserted. (HU)
Level 2 - The procedure is technically adequate. It needs acceptance criteria research and then inserted, format and HU Error Prevention Tools required. (HU, LOD)
Level 3 - The procedure cannot be implemented without tribal knowledge, but sufficient documentation is available (Technical manuals, SAR, etc.) to rewrite with a V&V. (LOD, Tech)
Level 4 - The procedure should not be implemented in the field. (HU, LOD, Tech)
Level 5 - Critical Operation Procedures where a number of operators need to be involved but a procedure writer could be a great help. (Tech)
Level 6 - Procedures that have been changed to facilitate continued operation with material deficiencies (Level 1 Ops Work Around).
Level 7 - Acceptable - meets or exceeds standards. (ACC)
- 30. Additional Comments:
Page 17 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Att a chment 2: Administrative Proce d ure Checklist
Procedure Number and
Title:
Reviewer: Rev:
No. Topic/Checklist Question Yes No N/A
- 1. Is the Level of Use - Information Use?
Comments:
- 2. Does the Table of Contents list major section headings, and attachments with page numbers?
Comments:
- 3. Does Purpose & Scope, Responsibilities, References, Definitions, and Records section have no actions in steps?
Comments:
- 4. Does that Purpose / Scope clearly state the primary objective of the procedure and what the procedure covers?
Comments:
- 5. Does the Reference section include reference number and name with revision date/number for Developmental References? With NO Revision date/number for Implementing References?
Comments:
- 6. Are Definitions listed in alphabetical order?
Comments:
- 7. Does the Responsibilities section identify the general responsibilities of performers, reviewers, and groups by task, position and organization title?
Comments:
- 8. Are Responsibilities limited to those who have responsibilities within the procedure scope?
Comments:
- 9. Do Notes have no actions either explicit or implicit?
Comments:
- 10. Are Notes always prior to the affected step?
Comments:
- 11. Are acronyms spelled out the first time they are used in the procedure or section?
Comments:
- 12. Does procedure minimize the use of Referencing and Branching?
Comments:
- 13. Is the format, layout and writing style the same throughout the procedure?
Comments:
- 14. Is non -value added information in Notes and Steps avoided? (e.g. Review the Safety Manual)
Comments:
- 15. Are all cross references accurate?
Comments:
- 16. Are procedures referenced identified by both the procedure number and the procedure title? Are these procedures listed in the Reference section?
Comments:
Page 18 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Att a chment 2: Administrative Proce d ure Checklist
- 17. Do attachments/appendices include Page X of Y of the overall procedure and include Page x of y of the individual attachment/appendix?
Comments:
- 18. Are all Attachments/Appendices referenced within the body of the procedure.?
Comments:
No. Topic/Checklist Question Yes No N/A
- 19. Grading criteria. Check all that applies:
Level 1 - The procedure is technically adequate. It needs to have format and HU Error Prevention Tools inserted. (HU)
Level 2 - The procedure is technically adequate. It needs acceptance criteria research and then inserted, format and HU Error Prevention Tools required. (HU, LOD)
Level 3 - The procedure cannot be implemented without tribal knowledge, but sufficient documentation is available (Technical manuals, SAR, etc.) to rewrite with a V&V. (LOD, Tech)
Level 4 - The procedure should not be implemented in the field. (HU, LOD, Tech)
Level 5 - Critical Operation Procedures where a number of operators need to be involved but a procedure writer could be a great help. (Tech)
Level 6 - Procedures that have been changed to facilitate continued operation with material deficiencies (Level 1 Ops Work Around).
Level 7 - Acceptable - meets or exceeds standards. (ACC)
- 20. Additional Comments:
Page 19 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 3: Assessment Team
Rear Admiral Rear Admiral Coyle has over 58 years of experience in the maintenance and Team M.T. Coyle operation of Naval and commercial nuclear power, including significant positions Lead USN (Ret) of leadership. After completing the Navy Nuclear Propulsion and Submarine training programs he served in nuclear submarines before transitioning to the Navys Engineering Duty Officer program where his duties included extensive experience in naval shipyards that speciali zed in nuclear submarine and surface ship maintenance and repair. He eventually comman ded the Pearl Harbor, Hawaii and Mare Island, California Naval Shipyards. After selection for Rear Admiral his Flag Officer assignments were Deputy Commander for Submarines at the Naval Sea Systems Command, Maintenance Officer for the Pacific Fleet, and De puty Commander for Engineering at the Naval Sea Systems Command.
Following retirement from the Navy in 1998 he was the Site Vice President at the Clinton, Illinois Nuclear Power Station and later, the Site Vice President at the Cooper Nuclear Plant in Brownville, Nebraska. From 2004 to 2006 he was on loan to the Nuclear Energy Institute (NEI) in Washington, DC. From 2006 to 2009 he was Vice President, Special Projects for Exelon, the largest operator of commercial nuclear plants in the United States. From 2009 he has been an employee and consultant to several contractors performing work for the Department of Energy. RADM Coyle received a Bachelors degree from the U.S.
Naval Academy where he graduated with distinction in 1965, and a Master of Science degree in Mechanical Engineering from the U.S. Naval Postgraduate School in Monterey, California where he received the Naval Sea Systems Command award for Naval Engineering.
Dave Linders Mr. Linders has o ver 40 years experience in nuclear power, including over 30 Analyst years of procedure program and procedure writing experience. Provided oversight for Diablo Canyon, Palo Verde, Robinson, FPL (Turkey Point/St. Lucie) and Ginna Nuclear Plants for Procedure Upgrade Projects. Previous Steering Committee Member of Procedure Professional Association (PPA). NRC Licensed Reactor Operator from Waterford 3 Nuclear Plant. Qualified as PPA Writer Certi fication Instructor. Developed and implemented Project Plan for the various Procedure Upgrade projects. Conversion Project Manager, Beaver Valley Nuclear Plant: Performed as the Conversion Project Manager for Operations procedures from MS Word to VE PROMS. Responsible for deve loping project schedule, providing work assignments, editing of procedures, reviewing of converted procedures prior to delivering finished product to the client, provided overall status of the project and any issues at Weekly Status Meeting. Wo rked with Software developers to identify and provide solutions for software related issues. Technical Assistant to Procedure Manager at Palo Verde during 95003 Recovery effort. Performed procedure assessments and Improvement Plans for numerous nuclear facilities. Developed Procedure Writers Guide, Procedure Process and Procedure Use & Adherence Procedures for several nuclear utilities. Working group on all Procedure Professional Association (PPA) related documents. Conversion Project Manager, Barakah Nuclear Power Plant (UAE):
Performed as the Conversion Project Manager for Operations procedures from MS Word to VE PROMS. Responsible for developing project schedule, providing work assignments, editing of procedures, reviewing of converted procedures prior to delivering finished product to the client, provided overall status of the project and any issues at Weekly Status Meeting. Worked with Software developers to identify and provide solutions for software related iss ues. Business Development Manager - Procedures, Westinghouse Electric Company:
Page 20 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 3: Assessment Team
Responsible for business development, customer support, and employee relations related to procedures in the nuclear industry. Perform assessments of utility Procedure Programs to ensure compliance with NRC, INPO upper tiered documents as well industry best practices. Provide consultation to various clients as industry expert for procedure related issues. Lead Procedure Writer, Barakah Nuclear Power Plant (UAE): Incorporated Procedure Change Request for Abnormal Operating Procedures (AOPs) while writing and supervising a remote group of procedure writers. Procedure Consultant, Fermi Nuclear Power Plant:
Performed procedure assessment to ensure compliance with Regulatory Guide 1.33, Quality Assurance Program Requirements (Operation), ANSI N18.7/ANS 3.2, "Administrative Controls and Quality Assurance for t he Operational Phase of Nuclear Power Plants, and applicable NUREGs for EOP development.
Procedure Consultant, Idaho National Laboratory: Performed procedure assessment to ensure compliance with applicabl e INPO and Procedure Professional Association (PPA) documents. Procedure Writer, South Texas Project: Developed procedures related to plant modifications for upcoming outage, provided procedure backlog and outage support. Project Manager -
Procedures, Barakah Nuclear Power Plant (UAE): Provided project oversight and direction to 35 procedure writers for development of Normal Operating Procedures and Alarm Response Procedures to support upcoming fuel load.
Fleet Procedure Writer, Entergy Fleet: Developed n ew Fleet procedures using existing procedures from each site. This included placing the existing procedures in a new Entergy Fleet procedure template, retaining information that was common at each site and developing a new procedure incorporating industry Human Performance guidelines. Lead Procedure Writer, Waterford 3: Performed Pilot Project for a Human Performance Upgrade of Operations Procedures as part of the 10 Year IST Update. This included placing the procedures in a new Entergy Fleet procedure tem plate, incorporating guidance in the Writers Guide, rewriting steps to incorporate industry Human Performance guidelines and incorporated changes associated with the 10 Year IST Update.
Carl Kuester Mr. Kuester has more than 35 years of experience in the commercial Nuclear Analyst industry. He spent 22 years at Perry Nuclear Power Plant, and held positions in Operations, as a Non -Licensed and Licensed Operator, in System Engineering as an NSSS Engineer and NS SS Supervisor, and in Work Management as a Work Week Manager and Cycle Manager. While at Perry Mr. Kuester took part in two regulatory recoveries. After leaving Perry Mr. Kuester held various consulting roles in six regulatory recoveries, and managed a num ber of projects, including Fukushima modifications, procedure upgrades and several Innovation and Efficiency improvement projects. Mr. Kuester holds a Project Management Professional Certification (#1533860) and a Bachelors of Science in Nuclear Engineer ing Technology.
Page 21 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 3: Assessment Team
Kevin Neal Mr. Neal has over 30 years of experience in the maintenance and operation of Analyst Naval and commercial nuclear power facilities. After completing the Navy Nuclear Propulsion training program, he served on a nuclear surface ship where his duties included operation of mechanical equipment, maintaining chemistry controls of power plant water systems, and managing radiological controls for the nuclear plant. After completing his service in the Navy, he became a procedure writer supporting projects at various Department of E nergy facilities, commercial nuclear power plants, non -nuclear power plants, refineries, and pharmaceutical facilities for the last 25+ years. Kevin has been a Procedure Professionals Association (PPA) steering committee member since 2015 and certified to teach the PPA writer certification course since 2014. He has taught the writer certification course to students from a wide range of industries including power plants, the Department of Energy, and oil & gas.
Ron Stevens Mr. Stevens has over 45 years of experience in Nuclear Quality Assurance, Analyst Nuclear and Environmental Regulatory compliance and support, Nuclear Safety and Licensing, Project Management, Contract Management, Nuclear Safety Culture, Change Management, Nuclear and Mechanical Engineer ing, and Senior Management of Commercial Nuclear organizations and U.S. Department of Energy (DOE) Contractor organizations. As an NQA -1 certified Lead Auditor, Mr.
Stevens conducted numerous audits and surveillances, including supplier audits, QA program implementation and effective implementation of quality procedures.
Mr. Stevens has developed both NQA -1 and 10 CFR 50, Appendix B QA programs, including preparation of QA program procedures, and he has performed oversight of QA program implementation. Mr. Stevens has also performed numerous management assessments, self -assessments, and management observations, having authored the procedure for management observations for the Lead Laboratory at the Yucca Mountain project. Mr.
Stevens has significant experti se in Nuclear Safety Basis and Authorization Basis for DOE nuclear facilities, and he was a key developer of the Documented Safety Analysis for the Hanford Tank Farms. Mr. Stevens also has extensive experience interacting with regulatory agencies and in implementation of U.S. Nuclear Regulatory Commission (NRC) regulations and DOE regulations. Mr. Stevens has had many years of experience working with behavioral scientists and developing programs to instill a strong Nuclear Safety Culture at several commerci al nuclear facilities, and he spearheaded CB&Is Nuclear Safety Culture journey, including development of policies and procedures, the Nuclear Safety Culture Council and the Councils charter, as well as development of the Nuclear Safety Culture survey too l. Mr. Stevens is a registered professional engineer (inactive), and he holds a DOE Q clearance. Mr. Stevens retired from full -time consulting July 31, 2022, but remains active in the industry on an as -needed basis.
John Ettien Mr. Ettien brings more than 43 years of experience in program/project/facility Oversight management, procedure development and implementation, training, and qualification, conduct of operations (CONOPS), and regulatory compliance to include over 21 years of experien ce directly supporting the U.S. Department of Energy (DOE) and its prime contractors. His relevant experience includes:
Waste Disposition Operations Manager; Oak Ridge Reservation Landfill (ORRLF) Operations Manager; Operations Manager for the K -25 Decont amination and Decommissioning (D&D) Project; and Deputy Project Manager (DPM) for the Molten Salt Reactor Experiment (MSRE) chemical defueling project. When Mr. Ettien was the ORRLF Manager and the Waste
Page 22 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 3: Assessment Team
Disposition Operations Manager, he maintained all of the roads necessary for operation of the facilities. He was also required to ensure all maintenance was performed to U.S. Department of Transportation (USDOT) standards for a large variety of equipment ( e.g., bulldozers, pan scrapers, dump trucks, backhoes, straight -frame trucks, mobile cranes, forklifts, tractor trailers, etc.) Mr. Ettien never missed an annual inspection for the equipment for which he was responsible. He served as the Surveillance and M aintenance (S&M) Project Manager for all Bechtel Jacobs Company LLC (BJC) managed facilities at the Y -
12 National Security Complex (Y -12) and the Oak Ridge National Laboratory (ORNL). Mr. Ettien led the effort that was responsible for the reduction of S&M requirements and field activities, exceeding $100M. He managed the document revisions necessary (e.g., Documented Safety Analysis [DSA], Technical Safety Requirements [TSR], etc.) to reflect the reduced level of S&M activities. He was the Project Manager f or various DOE projects, including the development of the Standards/Requirements Identification Documents (S/RIDS) at the Savannah River Site (SRS) for both the Site and Defense Waste Processing Facilities. He served as the Program Manager for URS l CH2M O ak Ridge LLC (UCOR)
Facility Management, training all UCOR Facility Managers (FMs) and implementing CONOPS organization wide for UCOR. Mr. Ettien also served as the Lockout/Tagout Subject Matter Expert (SME) and the Hazardous Energy Program Manager for UCO R. On the Tower Shield Reactor Project, Mr. Ettien ensured the development, review, and approval of all defueling procedures, developed the working schedule, and interfaced with the various organizations responsible for the oversight of this project. He pr ovided oversight to the BJC and subcontractors CONOPS programs, mentoring and qualifying facility managers, interfacing with training organizations to identify training requirements, and reviewing training courses for implementation of applicable standards. He was also the DPM for the U.S. Spent Fuel Team in the DPRK and mentor to the Operations Managers at the Comanche Peak, Point Beach, and Sizewell B Nuclear Plants. Mr. Ettien is trained in Tap Root Cause Analysis and has participated in performing Root Cause Analysis utilizing other methodologies. Mr.
Ettien has an active DOE "Q" clearance.
Page 23 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 4: Contacts and Interviews
Randy Strader, Chief of Reactor Operations Michelle Dudley, QA Program Manager Jacob Seiter, Training Program Manager Brandy Frie, Operations Planner In addition, onsite interviews were conducted with 8 individuals from Operations.
Page 24 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Page Numbering for Attachments Step 4.6 maintain the page numbering Appendices do not include the page sequence and a separate internal numbering for the procedure, but only attachment page number format as include the page numbering for Page X of Y. appendix.
Step 4.13.3.2 display unique attachment page number and total attachment page count.
Table of Contents (TOC) Step 4.12 states the Table of Many procedures lack a Table of Contents serves as overview for major Contents, but it is required per AR 5.1, activities is in the procedure. Procedure Writers Guide, Appendix A.
Should be automatically generated and systematically maintained by the procedure template.
Include the Table of Contents before the procedure instructions.
Page 25 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Procedure Sections - Structure Step 4.13.1, Table 1, Procedure Current procedures do not use the Structure provides Required and recommended Procedure Structure.
Optional Procedure sections for each
- Purpose and Scope type of procedure.
- Definitions
- Personnel Requirements
- Purpose and Scope
- Tools, Equipment, Reagents
- References and Commitments
- Limitation
- Definitions
- Precautions
- Responsibilities
- Initial Conditions
- Precautions and Limitation
- Lock Out Tag Out
- Prerequisites
- Procedure
- Instructions
- Restoration
- Acceptance Criteria
- Shutdown
- Retention of Records
- Signature
- Attachments
- References
Page 26 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Procedure Sections - Required Step 4.13.1, Table 1, Procedure NCNR -ROE has maintained some Sections Structure provides Required and sections as optional, therefore the Optional Procedure sections for each procedure writer has the option to not type of procedure. include the section. This choice should have been made by the Procedure Once it is determined what sections Process Owner during the will be included in each type of development of the Writers Manual.
procedure it is a Required section and This results in some procedures within not an Optional section. the same type having 6 sections and others 7 or 8 sections.
Step 4.13.1.3 states do not omit required sections. Use None or N/A when no information is needed for a required section.
Page 27 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Scope Section Step 4.13.3 describes the Most of the Scope statements that requirements for a Scope statement. were reviewed consisted of various The applicable statement is as types of information, but not actual follows: Scope.
Substep 4. For technical procedures, use the Scope section to describe the activities covered by the procedure and, if necessary, to address the limitations or boundaries of the procedure (see Figure 15, Sample Technical Procedure Scope Statement). Include the following types of items in this section:
- The extent to which any technical requirements are fulfilled.
Any exceptions or specific boundaries in the applicability of the procedure, including personnel and types of equipment.
Precautions and Limitations - Actions Step 4.13.7.2 states do not provide Limitations and Precautions section direction in the P&L section. includes actions or implied actions in most procedures.
Many of the Limitations and Precautions are Prerequisites.
Page 28 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Precautions and Limitations - Specific Step 4.13.7.3 do not repeat Caution Specific Cautions are listed in the Cautions statements in the P&L that are listed Precautions section and should be as a Caution prior to the to which they moved prior to the step to which they apply. apply.
Prerequisites Step 4.13.8 shall identify the activities NCNR -ROE does not include a to be completed by the performer and Prerequisite section. The bullet items requirements that shall be met prior to below would be listed as subsections procedure performance. per the PPA under Prerequisites. In some cases, the items below are prior Place the Prerequisites section to the prior to the Limitation and immediately before the Instruction Precautions section.
section.
- Personnel Requirements
- Tools, Equipment, Reagents
- Initial Conditions
- Lock Out Tag Out
AR 5.0, Procedure Use and Adherence refers to the Prerequisite section, so there is a disconnect between AR 5.0 & AR 5.1.
Instructions - Content Step 4.13.9 general rules for writing Some procedures include instruction steps include step by step informational steps and action steps.
instructions for performing required Many of these informational steps tasks and subtasks. should be Notes prior to the step.
Page 29 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Acceptance Criteria Section Step 4.13.10 describes the None of the reviewed TSPs /ICPs requirements for the Acceptance identified acceptance criteria overtly, Criteria section. There are two which indicates that the SRO applicable statements, listed below: reviewing the procedure must identify The Acceptance Criteria section shall the acceptance criteria for each provide a basis for determining the performance. Many of the MPs success or failure of an activity. It is included vague acceptance criteria, the parameter against which collected such as verify no significant wear, data is compared to determine which leave the actual determination satisfactory completion of or of acceptability with the user and their adherence to the following: interpretation of significant.
- Equipment operability conditions Lack of any referenced calculational
- Conditions for functionality basis for acceptance criteria. This
- Technical Specifications would be considered a significant gap.
- Regulatory requirements None of the reviewed TSPs included
- Design bases criteria the calculational basis for acceptance
- Post maintenance testing criteria (typically as a developmental reference). For instance, the T SP for Reactor Coolant System Relief Valve testing was reviewed, with the acceptance criteria listed as 50 -
55 PSIG.
Page 30 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Acceptance Criteria Section Typically, a calculation that supports (continued) the value would be identified in the references, with the calculation taking instrument inaccuracy into account and specifying the margin to protect the Reactor Coolant System pressure boundary. No such calculation wa s referenced. If no calculation exists, then a different issue with the facility design basis is identified.
Notes - Location Step 4.15.5, 7th bullet Notes should Notes follow the steps to which they be placed prior to the steps to which apply.
they apply.
Notes /Cautions/Warnings - Actions or Step 4.15.5, 4th bullet states Notes, Actions or implied actions in Notes Implied Actions Cautions or Warning should not and Cautions.
contain an implied instruction or action step.
Page 31 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Conditional Logic Step 4.16 provides information on the Conditional Logic statements have not use of terms IF/THEN and been developed in AR 5.1, Procedure WHEN/THEN, plus the use of AN or Writers Guide, therefore they are OR when part of the logic statement. formatted incorrectly in all procedures reviewed.
Includes requirement to emphasis and when 3 or more conditions use a list.
Commitment Referencing Step 4.19.1 describes the None of the procedures reviewed requirements for Commitment included any commitment referencing.
Referencing: The result is that any changes made due to Corrective Action Program Commitments should be referenced at actions can be changed with any the step level and uniquely identified subsequent revision.
so they will not be inadvertently removed (see Figure 38, Sample Commitment Reference).
Page 32 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Methods of Emphasis Step 4.20 emphasis should be used No methods of emphasis or for Technical Procedures in inconsistent method of emphasis for Prerequisites, Instructions and Action Action Verbs and Conditional Logic Steps in Attachments terms, plus no directions that emphasis should only be used in Emphasis should be used on the Prerequisite, Instructions and following items: Attachments (Appendices).
- Action verbs Some a ction verbs are presented in
- Conditional and logic terms Upper Case and others are in Title
- Component positions and nouns Case. An emphasis technique should names be used for action verbs and it should
- Notes, Cautions and Warnings be consistent throughout the
- Computer Terms procedure.
PPA AP -907 -001 - Attachment 2, Error Traps Item 17 Use of Italics Step 4.20.6 avoid using emphasis Many procedures use italics as a such as italics, shading, highlighting method of emphasis.
and quotes.
Acronyms and Abbreviations Step 4.22.2 acronyms and Many acronyms are not spelled out in abbreviations should be spelled out the procedure s.
the first time they are used in a section of the procedure.
Page 33 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Vague Terms Step 4.23 states avoid terms that Vague terms such as approximately, required judgment on the part of the adequate, if required, and as desired user: are often used. This provides the
- Approximately procedure user an opportunity to make
- Frequently a field decision without discussion with
- As (if) required supervision.
- As (if) applicable
- As (if) needed
- As (if) desired Increase/Decrease Step 4.23.7 states to avoid words that Both increase/decrease are in the should similar when read aloud such procedures.
as increase and decrease.
Action verbs Step 4.23.8 maintain a list of Action NCNR -ROE does not have a list of Verbs. Action Verbs.
Page 34 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Symbols Step 4.28.2.3 for procedure text write Symbols are use d in the procedure out the following terms: text.
- Greater than or equal to
- Less than or equal to
- Greater than
- Less than
- Equal to
- Does not equal Symbols associated with calculations, formula, equation, table or data sheet may be used.
Level of Detail Step 4.29 states to provide a level of Some procedures are missing level of detail that takes into account detail or refer the user to a technical experience/qualification of user, skill manual.
of the craft, complexity, frequency, Missing level of detail can result in consistency, and consequence of errors due to the user now knowing error. how to perform a task. Referring the user to a technical manual can present challenges to the user when the technical manual is not written to the PPA standard (which is typically the case). Additionally, the technical manual has likely not received the same reviews as the proce dure.
Page 35 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Consistency Step 4.30 maintain consistency in Procedures are not consistent with procedure style, form, and each other and lack the guidance that organization both within and among would be provided by a industry the procedures. standard procedure template and a Writers Guide that is detailed to address procedure writing activities.
Referencing and Branching Step 4.31.1.3 states if information Many times, the user is directed back required for a step can be easily to the Limitation and Precautions to incorporated in the step it should be obtain information. This information included at the step where it is should be place d where it is needed needed and not refer the procedure by the procedure user.
user back to a previous step. Some procedures have steps that reference to an incorrect step (such as referring to a step number that doesn't exist).
Page 36 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Referencing and Branching Step 4.31.3.2/4.31.3.2 - Branching Referencing and Branching are not directs the user t o another procedure defined in AR 5.1, Procedure Writers or step in the same procedure, but the Guide, therefore they are not used or user does not return to the original not properly applied when they are step. used.
AR 5.0, Procedure Use and Terms used Branching are Go To and Adherence discusses Referencing and Proceed To. Branching, but if they are not provided in the Writers Manual they will not be Referencing directs the user to used or incorrectly used.
another step or action within the same procedure or another document and the user returns to the original step.
Terms used for Referencing are Refer To, Per, See, Use, Repeat.
Referencing and Branching - Step 4.31.4 states the first time a Procedure number and Attachment Referencing Procedures procedure or procedure attachment is number are used, but procedure title referenced in a section, include both or attachment title is very seldom the procedure or attachment number used.
and title.
Page 37 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Reference Section Step 4.31.3 states the first time a Reference section does not include procedure or procedure attachment is titles in most cases.
referenced in a section include both the procedure or attachment number and title.
Action Verbs Step 4.32.8(1) Steps contain a single Steps throughout the procedure sets action verb. contained multiple actions in a step which could result in Human Performance issues when place -
keeping.
Page 38 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Action Verbs Attachment 1, Action Verbs terms P rocedure s uses all three terms and such as Ensure, Check and Verify are with no Action Verb list to define the defined. Two of these terms will mean terms the procedure writer provides an the same thing. The PPA standard interpretation, and the procedure user states facilities should select the provides another interpretation.
combination of two verbs that best fits the facilities historical use of these verbs and apply those two verbs consistently.
PPA AP -907 -001 - Att. 2, Error Traps Item 16 two words that mean the same thing can make the document harder to understand, which can lead to error.
Action Verbs Attachment 1, Action Verbs provides a Steps are written as OPEN or CHECK list of action verbs and their OPEN, which means the procedure definitions. writer is not aware of the terms Ensure or Verify. Ensure or Verify are not defined, but may mean (depending on facility definition ) to take action to place the component in the required position. The step should be written ENSURE valve is open.
Page 39 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
General Items (Affects All Procedure Sets)
Multiple Components in Same Step PPA AP -907 -001 - Att. 2, Error Traps Multiple components in a step which Item 5 steps with one action verb and could result in Human Performance two objects affecting configuration are issues when place -keeping.
also an error trap.
Vague Steps or Steps Missing Critical PPA AP -907 -001 - Att. 2, Error Traps Providing generic statements that all Detail Item 4 vague steps or inadequate Technical Specifications must be detail can put the worker in knowledge adhered to does not add value to the base performance mode with a procedure. This is a requirement per corresponding higher error rate. the Operating License for the facility and does not need to be included as a Precaution and Limitation.
Page 40 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set
AR - Administrative Rules AR - Content None AR 1.1, Human Performance Tools are based on the Exelon procedures and not the INPO guidance. There are too many HU Tools provided, and the user cannot possibly be proficient in all of them. Suggest aligning the procedure with the INPO standards and reduce to number of HU Tools to a number that is beneficial to the facility.
OI - Operating Instructions OI - No Unique Items Identified. Items are provided in in the General Items Review Section.
Page 41 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set
MP - Maintenance Procedures MP - Procedure Sections Section 4.13 is called Procedure The TSPs do not have an Acceptance Sections. Step 4.13.1, General Criteria section.
Requirements, includes a table that designates the required (R) and optional (O) sections for each procedure type and lists these sections in the recommended order.
The required sections for TSP (Testing) and MP (Maintenance) are as follows:
- Purpose
- Scope
- References and Commitments
- Precautions and Limitations
- Prerequisites
- Instructions
Additionally, TSP (Testing) also requires Acceptance Criteria.
Page 42 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set MP - Purpose Step 4.13.2 describes the Both the TSPs and MPs have requirements for a Purpose appropriate Purpose statements, but statement. The two applicable there were definitely procedures that statements are as follows: included incorrect Purpose Substep 4. If writing the Purpose statements.
section in a technical procedure, then clearly describe the objectives to be achieved by performance of the procedure.
Substep 5. Clearly state the primary objective of the procedure without simply repeating the title of the procedure. This is accomplished by answering what, when, and why as appropriate.
Page 43 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set
TSP - Technical Specification Procedures TSP - Procedure Section Section 4.13 is called Procedure The TSPs do not have an Acceptance Sections. Step 4.13.1, General Criteria section.
Requirements, includes a table that designates the required (R) and optional (O) sections for each procedure type and lists these sections in the recommended order.
The required sections for TSP (Testing) and MP (Maintenance) are as follows:
- Purpose
- Scope
- References and Commitments
- Precautions and Limitations
- Prerequisites
- Instructions
Additionally, TSP (Testing) also requires Acceptance Criteria.
Page 44 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set TSP - Purpose Section Step 4.13.2 describes the Both the TSPs and MPs have requirements for a Purpose appropriate Purpose statements, but statement. The two applicable there were definitely procedures that statements are as follows: included incorrect Purpose Substep 4. If writing the Purpose statements.
section in a technical procedure, then clearly describe the objectives to be achieved by performance of the procedure.
Substep 5. Clearly state the primary objective of the procedure without simply repeating the title of the procedure. This is accomplished by answering what, when, and why as appropriate.
Page 45 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set TSP - References and Commitments Step 4.13.4 describes the Some of both TSP and MP have Section requirements for the References and references, but none of them tag the Commitments section. The applicable references as either Developmental or statements are as follows: Implementing/Performance.
The References and Commitments section shall identify the documents used in the development of the procedure or required for procedure performance. For example:
- Regulatory documents
- Operating experience (internal and external)
TSP - Instructions Section 4.13.9 describes the requirements for Almost no TSPs or MPs include any Instructions section. There are two means of place -keeping. The few applicable statements, listed below: procedures that required data General rules for writing instructional recording typically used data sheets, steps are: which is a practice that has not been
- Include any provision necessary to used in commercial nuclear for several facilitate recording data, signoffs, decades.
place -keeping marks, and remarks (as applicable).
Page 46 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set TSP - Restoration Section S te p 4.13.9 describes the Most of the restoration steps reviewed requirements for the Restoration were very high level, lacking section. specificity, with no discussion of the When writing restoration steps, return of tools or disposal/storage of include the following: consumables. Very few procedures
- Steps required to return the included notifications of facility affected structures, systems, or personnel after the task was equipment to the configuration completed.
appropriate for plant conditions.
- Return of tools and equipment.
- Disposal or storage of consumables.
- Notification of appropriate personnel of system return to service
Page 47 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set TSP - Attachments Step 4.13.13 describes the The labeling of attachments of TSPs requirements for the Attachments and MPs was uneven at best, and section. The applicable statements non -existent at worst.
are listed below:
Each attachment shall be:
- Uniquely identified by title and attachment identifier.
- Ordered with a sequential identifier.
- Displayed using the unique attachment page number and total attachment page count.
- Referenced within the body of the procedure.
TSP - Tolerances Section 4.28 includes the The level of inconsistency in the TSPs requirements for Numerals, Symbols, and MPs was noteworthy. Tolerances Units and Tolerances. were annotated differently and varied significantly on a procedure by procedure basis. This is a challenge to the user because they have to adjust to the different standards that wer e utilized for various procedures.
I&CP - Instrumentation and Control Procedures
Page 48 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set I&CP - No Unique Items Identified.
Items are provided in in the General Items Review Section.
HPP - Health Physics Procedures HPP - No Unique Items Identified.
Items are provided in in the General Items Review Section.
HPI - Health Physics Instructions HPI - No Unique Items Identified.
Items are provided in in the General Items Review Section.
AP - Annunciator Instructions AP - Immediate Actions Immediate Actions are not discussed NCNR -ROE has numerous Immediate in the PPA Standard, but they are Action Steps in the AP procedures.
discussed by the NRC. This is The operator does not want to be discussed in ANSI N18.7 -1976/ANS - burdened with memorizing these 3.2, as actions that are performed Immediate Actions.
from memory and response and indication available in the Control Room. The NRC endorsed the ANSI standard in Regulatory Guide 1.33.
Page 49 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set AP - Format INPO 90 -021, Alarm Response The A P s are not written in the Procedures provides several formats consistent format and do not follow the that may used in the development of guidance of AR 5.1, A pp A, Document A Ps (ARPs). Section Requirements.
AP - Document Hierarchy None Several of the procedures reviewed should be Abnormal Operating Procedures (AOPs) instead of APs.
AP 0.7, Notification of Duress or Security Threat would be an example of something that would typically be an AOPs. The facility can either change AR 5.0 (Step 2.1.2) to include these procedures as APs or create a new category for AOPs.
Page 50 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 5: Gap Analysis
Topic Current Status Gap (PPA AP -907 -005)
Specific Items for Each Procedure Set
EI - Emergency Instructions EI - Content None EI 4.4 inspects HP equipment that may be needed during an emergency.
This should not be an EI procedure, but rather an Operations o r Health Physics Preventative Maintenance Task Order where the inventory of equipment is available when an emergency situation exists.
EI - Content None EI 3.6, Essential Personnel Evacuation does not provide a method of Accountability during the evacuation.
Page 51 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- 1. Introduction 1.1. Developing an effective Procedure Program is the first step in developing procedures and procedure users that will ensure activities are performed as planned and provide error free results.
1.1.1. Three main administrative procedures are developed and maintained. These procedures include the following:
- Procedure Writers Manual - provides directions to the procedure writer on how to write procedures.
- Procedure Process - provides direction on development, review, and approval of procedures.
- Procedure Use & Adherence - provides directions to the procedure user on how to use procedures.
1.1.2. The procedure template is a tool used by the procedure writer to ensure the following:
- Ensures procedures are properly formatted.
- Provides a consistent presentation throughout the entire procedure set.
- Includes Human Performance tools to reduce the potential of performance errors.
- Includes many time saving features which reduces the time spent on writing procedures, therefore reduces the cost of maintaining procedures.
1.1.3. The intent of the assessment is review against industry standard documents that include the necessary attributes to develop a Procedure Program that has the necessary rigor to ensure the safe operation of the facility for the long term. Documents used during this review were:
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence
Page 52 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- 2. Summary 2.1. The detailed review of the AR 5.0 series Procedure Program documents as discussed in the subsequent sections identified a number of gaps when compared to the PPA and INPO guidance documents. These gaps represent a vulnerability that procedures may not be written in a manner that minimizes the potential for error during performance.
Specific suggestions are provided in the detailed review below; however, the overall suggestion is that AR 5.0, AR 5.1, and AR 5.2 be updated to more closely align with the requi rements of the PPA and INPO guidance.
2.2. If it is determined that a Procedure Upgrade Project is needed, the AR 5.0 series Procedure Program documents will need to be updated prior to upgrading any procedures. Additionally, new templates should be developed to support upgrading procedures per the updated AR 5.0 series guidance.
- 3. Review of Administrative Rule (AR) 5.1, Procedure Writers Guide (Rev. 2)
The review was performed using the following industry standard documents:
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence 3.1. General Review 3.1.1. NCNR -ROE has not fully implemented the PPA Procedure Writer's Manual (e.g., PPA standard requires an Acceptance Criteria section for testing procedures but AR 5.1 does not require an Acceptance Criteria section per Appendix A). NCNR -ROE has also chosen to implement something different than the PPA standard in some instances (e.g., per the PPA standard, Notes are place prior to the applicable step; per AR 5.1, Notes are placed after the affected step). If not followed and no justification provided for the de viation from the standard, over time additional items may be changed moving the facility farther from the industry standard.
3.1.2. The title of the procedure includes the term Guide, which may be interpreted as merely a recommendation or suggestion.
- This Writers Guide is an approved procedure and is required to be followed when developing procedures. These are not recommendations or suggestions, but the method that has been chosen for writing procedures.
Suggest changing the title from Writers Guide to Writers Manual.
Page 53 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures 3.1.3. AR 5.1 does not include all information from the PPA Procedure Writers Manual.
- In EA -21 -148, Confirmatory Order, On Page 12 of Enclosure A it states:
Updated procedure standards to comply with PPA AP -907 -005, Procedure Writers Manual. T here are significant gaps between the NCNR -ROE Procedure Writers Guide and the PPA Standard.
3.1.4. Procedure lacks Examples to clarify the information written in the text.
Examples help the user understand the principles that are described in the text and are essential in the development of a Writers Manual.
- PPA Procedure Writers Manual includes over 60 examples to clarify and aid the user in understanding the principles that are being described.
Suggest adding examples similar to the PPA Writers Manual.
3.2. Specific Review 3.2.1. Procedure Cover Page includes for Level of Use the letter I. Users may not know that I means Information Use and what the Usage Criteria is for an Information Use procedure.
- Develop procedures that are written for the least or newly qualified procedure user, which in this case would mean spelling out Information.
- Procedure Cover Page included a Table with the Authors and various reviewers. This information is captured as part of the review and approval process and does not need to be included on the procedure cover page.
3.2.2. Purpose Step 1.1, first sentence uses acronym NCNR.
- Step 6.1 of the NCNR -ROE Writers Guide states that acronyms should be defined or spelled out the first time that are used. This was not done in NCNR -ROE Writers Guide. The facility has the option to either spell out all acronyms in first use per AR 5.1 or create a facility list of commonly understood acronyms.
3.2.3. Purpose Step 1.1, second sentence states, It includes consolidated guidance from government, commercial and research entities for effective procedure writing.
- EA -21 -148, Confirmatory Order, On Page 12 of Enclosure A does not list other sources that NCNR -ROE states as consolidated guidance, but rather uses one document (PPA AP -907 -005, Procedure Writers Manual) that was not followed to correct the issue.
- PPA AP -907 -005, Step 4.13.2.3, states clearly and briefly the primary objective of the procedure. The information in the first bullet is not part of the Purpose of the procedure, but is the various sources used to develop the document. Suggest deleting this sentence from the Purpose; applicable documents should be retained in the Reference section.
Page 54 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures 3.2.4. Scope, Step 2.1 states ROE documents with an I level of use
- PPA AP -907 -005, Step 4.29.1, first bullet states procedures should be written for the inexperienced, qualified user to successfully complete the task with no direct supervision. The I in the Scope statement should be spelled out as Information Use.
3.2.5. The Definition section in the NCNR -ROE Writers Guide includes one defined term. For comparison, PPA AP -907 -005 includes 35 defined terms. Although some of the Definitions included in PPA AP -907 -005 may not apply, definitions such as Action Verb, Administrative Procedure, Branching, Commitment, Conditional Step, Emphasis, Hold Point, Level of Detail, Procedure, Referencing, Supplemental Infor mation, Technical Procedure and several others should be included.
- This disparity in the number of definitions between the two documents indicates a lack of detail in the NCNR -ROE Writers Guide.
- PPA AP -907 -005, Step 4.13.5.5, states terms or phrases which have a special or limited meaning when applied to the context of the procedure or are unique should be defined.
3.2.6. Section 5, Font Face and Size, the NCNR -ROE Writers Guide uses Lucida Bright font and an 11 pt size. Suggest using one of the standard fonts specified in PPA AP 907 -005, Step 4.4.2:
- Arial 11 or 12 point
- Times New Roman 12 point
- Verdana 12 point 3.2.7. Step 6.1 states to define Acronyms on first use. The PPA standard requires abbreviations and acronyms to be spelled out the first time they are used in a section.
3.2.8. Step 6.2 has the action verb Use and has substeps such as:
Consistent word meanings Action verbs Concrete description
- These items are all statements and provide no guidance to the writer on how to do these items.
Page 55 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- The first item is Consistent word meanings. Earlier it was noted that one item is defined in the Definitions section of the NCNR -ROE Writers Guide compared to 35 items in the PPA AP -907 -005. Adding appropriate definitions promotes consistency.
- Action verbs are typically found as an attachment to a Writers Manual.
They provide guidance on the definition of verbs to ensure the correct verb is used. NCNR -ROE Writers Guide does not provide a list of Action Verbs.
PPA AP -907 -005 provides a list of over 500 Action Verbs and their meaning.
- Concrete description lacks detail to determine what it is in reference to:
components, locations, action verbs, etc. Additional information needs to be added to clarify the step.
3.2.9. Step 6.3.3 states to avoid using words that are ambiguous but does not provide a listing of suggested words that may be vague.
- PPA AP -907 -005, Step 4.23.6 and 4.23.7 provides a list of vague terms such as increase/decrease, approximately, frequently, gradually, as required, if needed, as applicable and several others. This list should be incorporated into the NCNR -ROE Writers Guide.
3.2.10. Step 7.2, Level of Use states the Author decides on the Level of Use during document creation. This step does not provide a reference to what the Author is using as a tool to determine the correct Level of Use. Typically, the Author/Procedure Writer would recommend a Level of Use during procedure development, and this would be Approved by the Procedure Process Owner when the procedure is approved. Suggest adding a reference to AR 5.0, Procedure Use and Adherence, to the step.
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence - Appendix B, Level of Use Determination Flowchart provides a flowchart to determine the Level of Use for procedures.
Recommend incorporating this type of flowchart into the Procedure Process.
3.2.11. Step 7.3 provides the definitions of Shall, Should and May from TR -5 License Section 1.3.29. The definition of Should includes the following:
- Should is used to denote a recommendation. This is most often the best course of action but is not a requirement.
- The first sentence is from TR -5, but the second sentence is not part of the definition. It is not the best course of action and when developing Technical Procedures the word should needs to be avoided. Should is a soft/vague term that allows the user to make in field decisions on their own. This is an error trap per PPA AP -907 -001, Procedure Process Description -
Page 56 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures Attachment 2, Error Traps. The second sentence should be deleted, which would make the definition of Should consistent with TR -5.
- A procedure represents the best most efficient way to perform a task, so that there is a better chance for a successful outcome. Adding soft terms, such as should can place the procedure user into Knowledge -based performance space and the outcome will no longer be successful.
3.2.12. Step 7.8, discusses Caution, Warning and Danger. All three of these items are for potential injury.
- Step 7.8.5.1, Caution is used to avoid potential injury.
- Step 7.8.6.1, Warning is used to avoid loss of life, personal injury, and other health hazards.
- Step 7.8.7.1, Danger is used to avoid loss of life.
- Step 3.11, Caution - equipment damage, plant transients or conditions that may adversely affect plant operation. (Equipment)
- Step 3.35, Warning - personnel injury, loss of life or health hazards.
(Personnel)
- Recommend deleting Danger and align with the PPA AP -907 -005 regarding use of Cautions and Warnings.
3.2.13. Step 7.8.8.1.1 - requires Notes to be placed after the affected step.
3.2.14. Step 8.2.4 through 8.2.6 are for Reviewers, Approvers and Reason for Changes
- This information should be moved to the Procedure Review and Approval Procedure since this procedure is for how to write/develop procedures.
3.2.15. Step 8.3.1.1 - states that Table of Contents should be removed wherever possible from working documents. PPA AP -907 -005 requires a Table of Contents for both Administrative and Technical procedures. Suggest deleting the statement.
3.2.16. Step 8.3.1.2 - states the Table of Contents may be used in Information Use procedures.
Page 57 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures 3.2.17. Step 8.7, 8.8, 8.11, 8.12 are all Prerequisites per the PPA standard:
8.7, Personnel Requirements 8.8, Tools, Equipment and Reagents 8.11, Initial Conditions 8.12, Lock Out Tag Out
- PPA AP -907 -005, Section 4.13.8.3 has a Prerequisites section, and the listed items above are subsections. These are items that are required to be completed prior to starting the Instructions section of the procedure.
Suggest aligning with the PPA standard and aligning Appendix A, Document Section Requirements with PPA AP -907 -005, Section 4.13.1.2.
3.2.18. Steps 8.7, 8.8, and 8.11 are all listed as requiring no action.
- PPA AP 907 -005, states to only write actions steps in Prerequisites, Instructions and Attachments. The above sections are all Prerequisites sections meaning they should have actions and need to be completed before the Instructions section can be started. Suggest changing AR 5.1 to align with the PPA AP 907 -005.
3.2.19. Page 21 of 31 is blank.
3.2.20. Step 8.13.2.2.1 states The verb that controls a physical action should be capitalized. The word should is a vague term and only a recommendation and provides an option for the terms not to be capitalized. The sentence should be re -written to The verb that controls a physical action is written as all capital letters.
3.2.21. Step 8.12.2.2.1 is a fifth level step and starts in the middle of the page, which leaves a lot of unnecessary white space on the left side of the page. The procedure template should be modified to have sections placed on the left margin of the page and rather than using numbers to use a capital letter. For example, Section 5.0, Procedure and the first subsection would be A, Starting Pump A. Both would be left justified.
3.2.22. Step 8.13.8, Conditional Step discusses what conditional steps are and when to use them, but it does not provide the key terms IF/THEN or WHEN/THEN.
3.2.23. Step 8.13.11, Continuous steps are steps that may be performed throughout the entirety or a majority of the procedure, but never discusses key terms that are used for Continuous steps.
- PPA AP -907 -005, Action Verb List - While, IF AT ANY TIME, Monitor, Maintain. Suggest incorporating these terms as part of Step 8.13.11.
Page 58 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures 3.2.24. Step 8.16.2.2 states the References should include the Revision number and date.
Development and Performance References. The Development References should include the Revision number or date. The Performance References, which are the procedures that may be used when the procedure is performed, should not include a Revision number. Procedures are constantly being revised and adding the revision number adds little value.
3.2.25. Section 10.1.1 - PPA AP -907 -005 is listed under a Government Reference.
The Procedure Professional Association is not a government agency, but an organization composed of members from commercial nuclear power plants, DOE facilities and non -nuclear faciliti es such as oil and gas.
3.2.26. Appendix A, Document Section Requirements does not include a Responsibilities section. A Responsibilities section is included in a similar table of PPA AP 907 005, Section 4.13.1. AR 5.0 has a Responsibilities section and AR 5.1 does not. In PPA 907 -005 it is an optional section for both Technical and Administrative procedures. Most Technical procedures do not have a Responsibilities section, but most Administrative procedures do have them.
The intent of the table is to allow facilities to choose what secti ons are best for each procedure type and then it is not an optional section, but a requirement.
It is not the writers decision to determine what sections are required for each procedure type, but rather the Procedure Process Owner for the procedure type. The Instructions section may be Section 6.0 in the procedure, but it would be Section 6.0 for all procedures of that type, such as the OIs. If no content is needed for a required section such as the Definition section, the writer would include the Definiti on section and write the word None in the first line after the section.
3.2.27. Items that are missing from the NCNR -ROE Writers Guide:
- Detailed Definitions Section.
- Information on Commitments or Corrective Actions and how to ensure procedure changes will not alter or remove these Commitments or Corrective Actions in the future.
- Information on Acceptance Criteria for Test.
- Detailed information about Signoffs and Place -keeping.
- Information on Hold Points that prevents the user from going further in the procedure without approval.
- Use of Tables - alignment, headers and when to use.
- Use of Checklist to establish a lineup or data collection.
Page 59 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- Use of symbols - spell out in the body the procedure and use the symbol where space is limited (tables).
- Tolerances - how to correctly write tolerances for upper and lower limits.
Incorrect 400 psig + 2% Correct 400 psig (392 to 408 psig).
- Level of Detail - considerations for the writer to use when determining level of detail in a procedure.
- Discussion on Reference and Branching:
- Referencing - go to another step, attachment or procedure and return to the original step. (Refer To, per, see, use)
- Branching - go to another step, attachment or procedure and do not return to the original step. (Go To)
- Writing steps for multiple components.
- Discussion on Punctuation - when and what method of punctuation will be used.
3.3. Recommendation 3.3.1. Administrative Rule (AR) 5.1, Procedure Writers Guide should be re -written to more closely align with PPA AP -907 -005, Procedure Writers Manual and by someone knowledgeable in the Procedure Professional Association (PPA) documents. During the development additional level of detail should be added and examples provided to clarify what is written in the text. Interim measures should be put in place to ensure compliance with PPA standards prior to AR 5.1 being updated to align with the PPA standards.
- 4. Review of Administrative Rule (AR) 5.0, Procedure Use and Adherence (Rev. 2 )
The review was performed using the following industry standard documents:
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence 4.1. General Review 4.1.1. The procedure does not follow the flow path of INPO 11 -003 and overall does not flow well due to how content is organized with the sections and subsections ( e.g., Section 11.3 Use of N/A is a subsection under Section 11 Procedure Alterations, but in the INPO sta ndard Use of N/A is a stand -alone section.
- In EA -21 -148, Confirmatory Order, On Page 25 of Enclosure A it states, Prior to any restart, NCNR will develop, implement, and maintain a written
Page 60 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures procedure that covers procedural use and adherence in accordance with the most recent version of INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence. With sections missing or key items that were not included or modified in the NCNR -ROE Procedure Use & Adherence procedure there are significant gaps between the NCNR -ROE Procedure Use and Adherence procedures and the INPO Standard.
4.1.2. Suggest adding how to verify the most current revision of the procedure.
4.1.3. Revision 2 of this document replaced many numbered steps with bullets steps where you have bullet underneath other bullet s. Three different types of bullets are used in the document and no explanation is provided for what each means or the significance. A closed bullet, open bullet and arrow are now used and some at the same level. This should not be an option per an industry standard template. This is an inconsistent method of writing. Using step number s provides structure to a document, especially when st eps may be several pages long.
4.1.4. Key items are discussed in the Procedure Use and Adherence procedure that are not part of the Procedure Writers Guide. It shows a disconnect between the two documents. Both items are discussed in PPA AP -907 -005 and INPO 11 -003. Examples include:
- Referencing and Branching
- Prerequisites 4.2. Specific Review 4.2.1. Section 2, S cope lists a detailed description of each procedure type. The purpose of the list is to provide a listing of procedures that are within the scope of this procedure.
- Recommend deleting the additional information and just listing the procedure types that are within the scope of the Procedure Use &
Adherence document.
- Suggest adding an additional step to list procedure types that are not within the scope of the Procedure Use & Adherence document.
- The detailed description of each procedure type should be described in a separate procedure defining the NCNR -ROE document hierarchy.
4.2.2. Section 3.2 0, Signal Words for Danger, Warning and Caution are not consistent with AR 5.1, Procedure Writers Guide - Section 7.8. For example, in AR 5.1 a Danger is for a loss of life and in AR 5.0 a Danger is for death or serious injury.
Page 61 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- Step 3.11, Caution - equipment damage, plant transients or conditions that may adversely affect plant operation. (Equipment)
- Step 3.35, Warning - personnel injury, loss of life or health hazards.
(Personnel)
- Recommend deleting Danger and align with the PPA AP -907 -005 and ensure the definitions are consistent between all documents.
4.2.3. Section 4, Roles and Responsibilities is a total of 8 pages in a 37 -page procedure.
- The information in this section is too detailed and many of the items should be moved to other documents, such as the Procedure Process (Review &
Approval).
- In PPA AP -907 -005, Step 4.13.6 states this section should be a high -level summary rather than a definitive list and should not repeat procedural requirements.
- The Responsibility of the Safety Evaluation Committee does not provide any responsibilities on Procedure Use and Adherence, but rather for Review & Approval of procedures. These types of items should be moved to another procedure.
- Many of the responsibilities involve items that should be procedural requirements and need to be discussed in the Instructions section of this procedure or another procedure. For example, Step 4.4.7 is to Verify that a hazard review has been completed. This task should be in the Review &
Approval procedure.
005, Step 4.32.1. Actions are only allowed in Prerequisites, Instructions and Attachments. AR 5.0, Step 4.4.7 uses the action verb Verify.
4.2.4. Step 5.2.1, 1 st bullet under 4 th bullet states the user will review and understand the precautions, limitations and prerequisites section prior to performing the procedure.
- AR 5.1, Procedure Writers Guide - Appendix A, Document Section Requirements does not include a Prerequisite section.
- When developing AR 5.1 the facility deviated from PPA AP -907 -005 and deleted the Prerequisites section, therefore items that should be applicable, such as Personnel Requirements, Tools, Equipment, Reagents, Initial Conditions and Lock Out Tagout are not be ing review as required by INPO 11 003.
Page 62 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures 4.2.5. Step 7. 7.7.1 states Critical Steps typically occur anytime an action involves a substantial hazard to equipment or personnel.
- This is not consistent with the Definition 3.5, An action that may trigger immediate and irreversible harm to personnel or equipment, or could significantly impact reactor facility operation, if that action or a preceding action is performed improperly.
- INPO 11 -003 defines Critical Steps as a procedure step, series of steps, or action that, if performed improperly, will cause irreversible harm to plant equipment or people or will significantly impact plant operation.
4.2.6. Step 7. 7.7.2 states Critical steps are indicated by Caution, Warning or Danger statements. The discussions of Critical Steps in INPO 11 -003 and PPA AP -
907 -005 do not suggest a Critical Step should be identified as part of a Caution, Warning or Danger. Critical Steps are most often identified during the Prejob Brief and may be identified with a unique identifier when identified in the Writers Manual.
4.2.7. Step 7 7.9, Branching and Referencing between procedures. Referencing and Branching are not discussed in AR 5.1, Procedure Writers Manual, so it would never be written into a procedure because there is no guidance for how to use it or what it means. This information needs to be added to AR 5.1
- Branching is discussed where the user would transition to another procedure and not return to the original procedure.
- Referencing sends the user to other steps, sections or supplemental information or another procedure and then the user returns the original step.
- Referencing - go to another step, attachment or procedure and return to the original step. (Refer To, per, see, use)
- Branching - go to another step, attachment or procedure and do not return to the original step. (Go To) 4.2.8. Step 7. 7.9.3 states when transitioning to another procedure, the initial conditions, precautions, and limitations of the new procedure must be met.
- INPO 11 -003, Step 5.5, last paragraph includes that the Prerequisites section requirement must met when transitioning to another document.
- AR 5.1, Procedure Writers Guide - Appendix A, Document Section Requirements does not include a Prerequisite section.
Page 63 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- When developing AR 5.1 the facility deviated from PPA AP -907 -005 and deleted the Prerequisites section, therefore items that should be applicable, such as Personnel Requirements, Tools, Equipment, Reagents, Initial Conditions and Lock Out Tagout are not being review as required by INPO 11 -003.
4.2.9. INPO 11 -003, Step 5.3 discusses the use and place -keeping of Notes, Cautions and Warnings where they are located on the same page and immediately before the step to which they apply.
- AR 5.0 does not discuss the place -keeping of Notes, Cautions and Warnings.
- AR 5.1, Step 7.8.8.1.1 requires Notes to be placed after the step it pertains to.
- Suggest adding information to AR 5.0 for place -keeping of Notes, Cautions, and Warnings and in AR 5.1 to move the Note prior to the step to be consistent with the PPA and INPO documents.
4.2.10. Section 11, Procedure Alteration allows minor modifications or temporary deviation to be made with the supervisors approval.
- This is not addressed in INPO 11 -003, and the reviewer feels what is stated should be addressed through the normal procedure review and approval process. Section 11 is asking the supervisor to approve a change to the procedure which does not affect reactor safety and a change of intent. How do they know that without going through the procedure review and approval process. There is also no mention of the 10CFR 50.59 screening process.
- The pen & ink changes that are made in the field with the supervisors initials should include editorial items that would automatically screen out during a 50.59 screening.
- Typically, what is seen is a change to a procedure in the field without going through an approved procedure review would be:
If the procedure change qualifies as any of the following:
- Obvious typographical errors (e.g., obvious step numbering errors or changes of referenced steps, sections, attachments, forms, etc.), if adequate information is available to verify correct numbers.
- Incorrect personnel/position titles due to approved organizational changes.
Page 64 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- Obvious discrepancies between component tag and the equipment identification, if adequate information is available to verify correct component/equipment identification.
- Incorrect References or missing References which should have been listed.
- Updates to or modifications of format when it does not affect performance of the instructions.
Then perform the following:
- Note the deficiency on the Working Copy.
- Obtain your supervisors review and initial adjacent to the step indicating their concurrence that this is an administrative correction.
- Continue the activity in progress.
- Initiate a Procedure Change Request after completion of the activity.
4.3. Recommendation 4.3.1. Administrative Rule (AR) 5.0, Procedure Use and Adherence is much closer to the industry standard (INPO 11 -003) than the Procedure Writers Guide but still has deficiencies as identified above. This document should be re -written by someone knowledgeable in the Procedure Professional Association (PPA) and INPO documents. The Procedure Use and Adherence and the Procedure Writers Manual should be written by the same person to ensure consistency between the two documents.
- 5. Review of Administrative Rule (AR) 5.2, Document Routing Policy (Rev. 1)
The review was performed using the following industry standard documents:
- INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence PPA AP -907 -001, Procedure Process Description is written at a high level and does not provide specific details, but rather items that should be included in the Procedure Review and Approval process.
5.1. General Review 5.1.1. Recommend changing the procedure title to Procedure Review and Approval or Procedure Process. The facility should have one document that has the information needed to review and approve a procedure.
Page 65 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures 5.1.2. Currently the procedure does not provide any information on the Procedure Owner. This should typically be a Department Head or Manager for the associated procedure type. It is typically not an individual but a title and the individual or their designees would be assigned as the Procedure Owner.
5.1.3. AR 5.2, Document Routing Policy is a total of 11 pages. The first 7 pages are administrative sections and have no actions (Purpose, Scope, Definitions, and Responsibilities). This procedure does not have adequate guidance and rigor to ensure that procedures are adequately verified and validat ed to ensure the safe operation of the facility.
5.1.4. A 10CFR 50.59 screening is not part of the process and should be included as part of the Review and Approval Process.
5.2. Specific Review 5.2.1. Section 4, Responsibilities is over 4 pages long and the whole procedure is only 11 pages long. Many of these responsibilities need to be moved to the Instruction section of the procedures.
5.2.2. Step 4.1.6 is in the Responsibilities section and directs the Author to determine if a Hazard Analysis is required. This is a procedural requirement that should be listed in the Instructions section of the procedure and should not be in the Responsibilities section.
- In PPA AP -907 -005, Step 4.13.6 states this section should be a high -level summary rather than a definitive list and should not repeat procedural requirements.
5.2.3. Step 4.1.6 refers to Section 4.4.7 of AR 5.0. Referring the user to a specific section in another procedure is not usually done. Most facilities will have guidance in the Writers Manual to recommend not referring to a specific section by number, but rather by section name or description of the task. Over a period new sections or steps may be added and the step reference will no longer be accurate.
5.2.4. Step 5.2 directs the author and reviewers to seek outside assistance if they cannot concur on a procedure.
- The term outside assistance is generic and needs to be more specific.
Typically, outside assistance would be the Procedure Process Owner or their designee.
5.2.5. Items that are missing from the AR 5.2, Document Routing Policy:
- Establish a process for Procedure Change Requests (PCRs) which is a process for the procedure user to provide feedback on identified issues with a procedure or a request to develop a new procedure. PPA AP -907 -
001, Section 4.1.
Page 66 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- Plan Procedure Development including the open PCRs, type of procedure revision required (Major, Minor, Editorial Correction) PPA AP -907 -001, Section 4.2.
907 -001, Section 4.3.
- Determine required reviews, perform reviews, and resolve comments.
Validate the procedure, which can be done with a Walkthrough, Comparison or Tabletop. PPA AP -907 -001, Section 4.4.
- Step 4.4.2.1.b, Table - annotates the use of an Independent Qualified Reviewer (IQR) or Independent Technical Reviewer (ITR).
- Approve the procedure. Include the criteria when Safety Evaluation Committee would need to review the procedure. PPA AP -907 -001, Section 4.5.
- Develop/Implement any Change Management Plan. A Change Management Plan is very seldom required but needs to be included when required. PPA AP -907 -001, Section 4.6.
5.3. Recommendation 5.3.1. This procedure does not have adequate guidance and rigor to ensure that procedures are adequately verified and validated to ensure the safe operation of the facility. Administrative Rule (AR) 5.2, Document Routing Policy should be re -written by someone knowledgeable in the Procedure Professional Association (PPA) documents. During the development PPA AP -907 -001, Procedure Process Description should be used as a guide. Interim measures should be put in place to ensure adequate verification, validat ion, and a pproval prior to AR 5.2 being updated to align with the PPA standards.
5.3.2. Adding the IQR/ITR process using a graded approach based on the type of change/revision to the procedure could improve the quality of NCNR -ROE procedures when implemented properly.
- 6. Review of Procedure Template 6.1. General Review 6.1.1. NCNR -ROE currently has two Microsoft Word templates for the procedures program. One template is for Annunciator Procedures (AP). The other template is used for all other documents in the procedure program.
6.1.2. Both templates provide a basic outline of the document structure (headers/footers, styles for paragraph formatting/numbering, etc.).
6.1.3. Neither template has any automation/macros other than the paragraph formatting/numbering.
Page 67 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 6: Assessment of NCNR -ROE Administrative Procedure Program Procedures
- Automation/macros are typically used in templates to simplify use of the template by providing single -click macros that perform a multi -step function (such as inserting an attachment, caution, etc.).
- Without the automation, procedure writers/authors have to manually perform the multi -step functions (such as inserting an attachment) as a multi -click process. This results in writers having to spend significantly more time formatting the document when ins erting content/text. This becomes a writer burden that can lead to errors in documents that can impact performance of the document.
- Automation/macros will also help enforce consistency in format per the writers manual across all the documents. NOTE - during interviews of Operations personnel, operators stated that may procedures have formatting issues that make the procedures difficul t to follow.
- The non -AP template in the current form also requires the writer/author to manually adjust the document section headings to meet AR 5.1, Appendix A (Document Section Requirements) since the section requirements vary based on the type of procedure. This can lead to inconsistencies in major sections used in each type of procedure, which was found during the review of approximately 300 NCNR -ROE procedures.
- The current template is costly, since the procedure writer spends time formatting items that should be auto generated by the procedure template.
6.2. Recommendation 6.2.1. New templates should be created with automation/macros to control content formatting to the maximum extent possible which will reduce writer/author time being spent trying to format the document and enforce consistency in format per the writers manual.
Page 68 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 7: Procedures that Should Not be Implemented in the Field as Written
Procedure Revision Title Issues Number Date
- 1. ICP 1.11 3/27 /2023 Paperless Recorder Calibration There is level of detail missing for disconnecting and connecting recorder wiring for calibration and how to perform channel check.
- 2. ICP 10.6 7/5/2022 Emergency Fan Controller Calibration The procedure only collects as found data and does not have instructions to adjust or collect as left data. This guidance appears to be missing otherwise this is not a calibration procedure.
- 3. ICP 3.1.1 8/24 /2022 Calibration Procedure Note at step 6.1 says initial readings are compared to the restored channel but there is no action step at the end of the procedure to perform this comparison and associated actions if there is an issue.
Step 6.6.8 appears to have a logical error because it states to proceed to Step 6.7.9 resulting in Step 6.6.9 being skipped but no Step 6.7.9 exists.
Step 7.2 uses a hose connected to transmitter testing ports but does not disconnect hose at the end.
- 4. ICP 3.1.2 9/13/2022 Reactor Outlet Plenum Flow FRC -3 Note at Step 6.1 says initial readings are compared to the restored channel but there is no action step at the end of the procedure to perform this comparison and associated actions if there is an issue.
Step 6.6.8 appears to have a logical error because it says to proceed to Step 6.6.10 resulting in Step 6.6.9 being skipped and no path to performing Step 6.6.9 exists.
Step 7.2 uses a hose connected to transmitter testing ports but does not disconnect hose at the end.
Page 69 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 7: Procedures that Should Not be Implemented in the Field as Written
Procedure Revision Title Issues Number Date
- 5. ICP 3.1.3 2/10 /2023 Reactor Outlet Flow FIA -40 There is an error in formatting after Step 6.4.4.
The next paragraph is unnumbered.
Step 7.2 uses a hose connected to transmitter testing ports but does not disconnect hose at the end.
- 6. ICP 3.14 4/23 /2023 Storage Pool Chill Water Cooling Flow Restoration section leaves transmitter isolation Calibration valves closed at the end. Theres no direction to remove voltmeter as part of restoration.
- 7. ICP 3.6 8/4/2022 Thermal Column Flow Indicator Control Section 9 closes isolation valves at end of procedure, meaning system has not been restored to operation.
Step 8.4.3 says to connect voltmeter to transmitter but doesnt specify where to connect.
Step 8.4.7 needs to be split into separate steps for setting zero and span output instead of a single step.
- 8. ICP 5.2 3/2 /2023 Recombine Pressure Indicator It appears Section 10 is supposed to calibrate either PIA -3 or PIA -4 but as written requires calibrating both at once, which isnt possible due to test setup. The procedure cannot be performed as written as a continuous use procedure due to how section 10 is written.
Steps 10.7 and 10.10 say to adjust as necessary but dont explain how to make adjustments.
Step 10.14.3 says to return system to normal with no explanation of what that means.
Page 70 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 7: Procedures that Should Not be Implemented in the Field as Written
Procedure Revision Title Issues Number Date
- 9. ICP 6.5 5/2 /2023 Purification Heat Exchanger Separate sections should be written for each Temperature Calibration channel (i.e., one section for inlet temp channel and another section for outlet temp channel).
Step 8.3.3 removes thermocouple from drywell but there is no restoration step to install thermocouple into drywell.
- 10. ICP 7.3 3/28 /2023 C001 Rabbit Lab Area Radiation Monitor There are no instructions if alarm does not alarm Calibration at 6 mR/Hr. No instructions to remove pulse generator or oscilloscope. Step 8.10.10 seems to be missing level of detail (how details) and has two unrelated actions.
- 11. MP 3.01 9/1 /2022 Testing an Electrical Circuit for Opens Made a Level 4 because the guidance in this document is not sufficient for a worker that is simply qualified in electrical safe work practices.
If the worker was a qualified electrician, it could be a Level 2 or, more likely, a Level 3.
- 12. MP 5.26 3/30/2023 Capping Tool Retraction and Removal No verifications or independent verifications.
Multiple P&Ls should be initial conditions, procedure steps or notes/cautions/warnings.
Multiple notes that include action states (which likely should be procedure steps). P&Ls from other procedures are invoke d in the P&Ls rather than listing the applicable P&Ls in this procedure.
This appears to be a complex and risk significant procedure. At a commercial plant this would be classified as an IPTE, with steps embedded to perform the IPTE brief. Marked a Level 4 due to the large number of issues noted above, and the apparent risk associated with the procedure.
Page 71 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 7: Procedures that Should Not be Implemented in the Field as Written
Procedure Revision Title Issues Number Date
- 13. MP 5.65 1/12/2023 Master Procedure: NBSR Spent Fuel Extremely complex procedure. No verifications or Shipment independent verifications. Many of the notes are action statements, which is problematic because the Notes are placed AFTER the applicable step.
Many of the P&Ls should be procedure steps or notes/cautions. Multiple examples of vague guidance. Assigned Level 4 because of the risk entailed in executing the procedure combined with a large number of defects as noted above.
- 14. TSP 4.3.1(1) 12/22/2021 Testing of the Primary System Relief This procedure should not be utilized until the Valve calculational basis of the acceptance criteria can be determined. If PG -116 is a 100 psig instrument with a +/ - 2.5% accuracy, which is typical, then the acceptance criteria would be exactly 52.5 psig, with n o allowable range. If the instrument is not calibrated, then additional actions would be required.
- 15. TSP 4.3.2(2) 8/19/2022 Operability Test of the Emergency Sump There does not appear to be any type of Pump with Light Water calculational basis for the acceptance criteria, nor was there a requirement to verify that PG -117 was in calibration periodicity. If no calculational basis is required, then this would be a Level 2 or Level 3.
- 16. AR 5.1 11/15/2021 Procedure Writers Guide A detailed review of AR 5.0, 5.1 and 5. 2 was performed against the industry standard documents. These reviews are included in another part of the assessment. It was determined that all the procedures need to be rewritten to align with the industry standard documents.
Page 72 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 7: Procedures that Should Not be Implemented in the Field as Written
Procedure Revision Title Issues Number Date
- 17. AR 5.2 9/2/2021 Document Routing Policy A detailed review of AR 5.0, 5.1 and 5. 2 was performed against the industry standard documents. These reviews are included in another part of the assessment. It was determined that all the procedures need to be rewritten to align with the industry standard documents.
Page 73 of 74
Independent Nuclear Safety Consultant Task 05 - Procedures Program Independent Assessment Report Attachment 8: Reference Documents
NCNR -ROE Documents
- 1. AR 5.0, Procedure Use & Adherence Rev. 2
- 2. AR 5.1, Procedure Writers Guide Rev. 2
- 3. AR 5.2, Document Routing Policy Rev. 1
- 4. Root Cause Investigation of February 2021 Fuel Failure, Revision 2 of September 13, 2021
NRC Documents
- 1. National Institute o f Standards a nd Technology, Center for Neutron Research - Confirmatory Order August 1, 2022
- 2. NRC Technical Evaluation Report (Docket No. 50 -184 March 9, 2023)
Industry Documents
- 1. DOE G 414.1 -1C, Management and Independent Assessments Guide
- 2. INPO 11 -003, Guideline for Excellence in Procedure and Work Instruction Use and Adherence
Page 74 of 74