ML022660459
| ML022660459 | |
| Person / Time | |
|---|---|
| Site: | Cooper  |
| Issue date: | 09/20/2002 |
| From: | NRC Region 4 |
| To: | Nebraska Public Power District (NPPD) |
| References | |
| IR-02-007 | |
| Download: ML022660459 (58) | |
See also: IR 05000298/2002007
Text
Appendix to
Nuclear Regulatory Commission Report on
Results of 95003 Supplemental Inspection
Conducted at Cooper Nuclear Station
NRC Inspection Report 50-298/02-07
APPENDIX A
PARTIAL LIST OF PERSONS CONTACTED
Licensee
D. Anno, EBS
T. Barker, Quality Assurance
R. Beilke, DED Senior Staff Engineer
J. Bess, Security Services Supervisor
K. Billesbach, Quality Assurance
T. Black, Fuels and Reactor Engineering
T. Boring, NIS, Computer Technician
M. Boyce, Senior Manager, Technical Services
D. Buman, Assistant DED Manager
G. Casto, Emergency Preparedness Manager
T. Chard, Chemistry/Radiation Protection Manager
J. Charterina, ECCS Supervisor, Plant Engineering Department
J. Christensen, Training Manager
D. Cook, Senior Manager, Emergency Preparedness
M. Coyle, Site Vice President
J. DeBartolo, ECP Coordinator
L. Dewhirst, Quality Assurance Audit Supervisor
F. Diya, Manager, Plant Engineering
P. Donahue, STE, Operations
A. Dostal, NPPD
R. Drier, Performance Assessment Department
R. Estrada, PAD Manager
B. Fehrman, Vice President - Energy Supply
C. Fidler, Assistant ESD Manager
J. Fix, Outage Manager
J. Flaherty, Site Regulatory Liaison
P. Fleming, Licensing Manager
W. Frewin, DED Supervisor
J. Gadsman, Principle Engineer
R. Gardner, Operations Manager
M. Gillan, Assistant to the Plant Manager
D. Gottula, Quality Inspector, Material Services
R. Grazio, Sequoia Consulting
P. Gritton, Budget Specialist, Business Services
A. Hasanat, Senior Mechanical Engineer
B. Heidzig, Maintenance
J. Hernandez, Mechanic
T. Hottovy, Assistant Manager, Plant Engineering Department
T. Hough, ESD
J. Hutton, Plant Manager
A. Jacobs, NSAG/PAD
D. Johnson, NMC
K. Jones, Manager, Design Engineering
D. Kimball, Assistant Radiation Manager
-2-
K. Kreifels, Project Supervisor, Maintenance
L. Kube, Kube Consulting
D. Kunsemiller, Manager, Risk and Regulatory Affairs
K. Lane, WHSE
J. Lechner, Project Manager, Plant Engineering Department
J. Lewis, Fuel & Reactor Engineering Manager
D. Linnen, TIP
T. McClure, Civil Supervisor, DED
T. McConnell, Polestar Applied Technology
J. McMahan, Work Control Supervisor
W. Macecevic, Work Control Manager
R. Maine, IMC Supervisor, Maintenance
L. Martin, Senior Consultant
D. Montgomery, Human Performance
O. Olson, RRA
D. Pease, Assistant Operations Manager
C. Pelchat, Metrologist, Plant Performance
T. Poindexter, Attorney, Winston & Strawn
J. Ranalli, Senior Manager of Engineering
D. Robinson, Senior Manager of Quality Assurance
J. Rush, Supervisor, Outage
G. Sawtelle, Risk Management
L. Schilling, Administrative Services Manager
D. Skillman, 95003 Team
J. Smith, Supervisor, Maintenance
T. Stevens, Mechanical Engineering Supervisor
S. Stiers, Project Manager, 95003 Team
J. Sumpter, Project Manager, Licensing
K. Sutton, Risk Management Supervisor
K. Thomas, ESD Mechanical Programs Supervisor
B. Toline, 95003 Project Manager
G. Troester, Communications
C. Warren, Chief Operating Officer, Utility Services Alliance
D. Werner, Training
N. Wetherell, Maintenance Manager
D. Wilson, Vice President - Nuclear
L. Young, Consulting Engineer, ESD
APPENDIX B
PARTIAL LIST OF DOCUMENTS REVIEWED
The Strategic Improvement Plan, Revision 1 and Supporting Work Breakdown System (WBS)
Folders.
Listing of procedures in work inventory for both Engineering and Maintenance that are being
revised as of 6/26/2002
All Outstanding Procedure Change Requests (PCRs) as of 6/26/2002
All Currently Existing Operations Desktop Guides as of 5/24/2002
Generic Lessons Learned From Refueling Outage RE18
Copy of CAP items for clearance orders since June 2000, and listing of SCR
Procedures/Guides Description Revision
0-CNS-01 Core Leadership Development Program 6
0-CNS-07 Management Field Observations 4
0-CNS-12 CNS Program Administration 2
0-CNS-23 Contractor Control 0
0-CNS-24 CNS Standards and Expectations 2
0-CNS-25 Self Assessment 6 and 7
0-CNS-31 Clearance Order Accountability 1
0-CNS-59 Event Review Board 6
0-EP-01 Emergency Response Organization Responsibilities 2
0-FFD-01 NPG Fitness for Duty Program 7
0.4 Procedure Change Process 32
0.4A Procedure Change Process Supplement 7
0.5 CAER Corrective Action Effectiveness Review 3
0.5 NAIT Corrective Action Implementation and Nuclear Action 11
Item Tracking
0.5.OPS Operations Review of Notifications/Operability 10C1
Determination
0.5 SCR Preparation of Significant Condition Reports
0.9 Tagout 30
-2-
0.10 Operating Experience Program 9
0.17 Selection and Training of Station Personnel 37
0.26 Surveillance Program 41
0.31 Equipment Status Control 11
0.40 Work Control Program 25
0.40.1 12 Week Work Control Process 8
0.50 Outage Management Program 14C2
2.0.1 Plant Operations Policy 45
2.0.2 Operations Logs and Reports 62C1
2.0.3 Conduct of Operations 32C1
2.0.4 Relief Personnel and Shift Turnover 12
2.0.12 Operator Challenges 2C1
2.1.20 RPV Refueling Preparation, Attachment 2, Inspection 49
of Sand Pocket Drains
2.3.1 General Alarm Procedure 32
3.4 Configuration Change Control 33
3.4.1 Part Evaluation 0
3.4.4 Temporary Configuration Change 0
3.4.5 Engineering Evaluations 4
3.4.7 Design Calculations 19
3.4.8 Design Verification 11
3.7 Drawing Change Notice 25
3.28.3 BWR VIP Program (Draft) 0
6.FP.604 Fire Door Annual Examination 8
6.HPCI.703 HPCI Supervisory Alarm Actuation Timer Channel 1
Functional Test
6.1RPS.307 Reactor vessel Low-High Water Level Channel 10
Calibration (Division 1)
-3-
6.2EE.302 4160V Bus 1G Undervoltage Relay and Relay Timer 7
Functional Test (Div 2)
6.2HPCI.302 HPCI Steam Line Low Pressure Channel Calibration 6
(Div 2)
6.2RHR.101 RHR Test Mode Surveillance Operation (IST) 13C1
(Division 2)
7.0.2 Preventive Maintenance Process 5
7.0.3 Maintenance Rework 1
7.0.4 Conduct of Maintenance 20
7.0.5 Post-Maintenance Testing 19
7.3.50.3 SW 89A\B Minimum Flow Adjustment 6
14.11.3 L & N Recorder SpeedoMax 165 and 250 Calibration 12
and Maintenance
ESDP-13 PSA model Maintenance and Update Procedure 2
ESDP-14 PSA Application 2
OTP806 Conduct of Simulator Training and Evaluation 0
CNS Radiation Radiological Protection Shop Guide -- Condensate and 5/16/02
Protection Shop RWCU Filter Performance Guidance
Guide 48C
Nuclear Training Self-Assessment 8
Procedure 6.3
Training Program Emergency Response Organization 4
Guide 101
Maintenance Work Pre-Job Briefs/Post-Job Critiques 4 and 5
Practice 5.0.7
Maintenance Work Maintenance Ownership 0
Practice 5.0.8
Nuclear Quality Quality Assurance Escalation Process 3
Procedure 2.7
Maintenance Department Principles and Standards 2
-4-
Project Management Desktop Guide 0
CAP Deskguide 5 Corrective Action Program Self Assessment Guideline 2
OER Desktop Industry Operating Experience Review Process 4
Guide 1 Guideline
OER Desktop Guideline for the Communication of Industry Operating 5
Guide 5 Experience to CNS
Self Assessment of the Radiological Programs, Self 3
Assessment Shop Guide
System Engineering Safety System Assessments 1
Desk Top Guide,
Section 15
Corrective Action Program Procedures and Desk Top
Guides
Assessments
Chung Chiu Common Cause Analysis (1993)
Strategic Plan for Performance Improvement (1993)
Enforcement Issue Investigation Team (1993)
Enercon Performance Assessment Report (1993)
Corrective Action Program Self-Assessment Report (1993)
Near Term Integrated Enhancement Program (1994)
Diagnostic Self-Assessment Team (DSAT) (1994)
Special Evaluation Team (SET) (1994)
Phase 1 Performance Improvement Plan (1994)
Phase 2/3 Performance Improvement Plan (1995)
Phase 3 Performance Improvement Plan (1996)
Engineering Self-Assessment (1996)
NRC Common Cause Analysis (1997)
Site Wide Self-Assessment (1998)
-5-
Summary and Evaluation of Cooper Nuclear Station Self-
Assessment of Performance Related to INPO Performance
objects and Previous Findings (1998)
Strategy for Achieving Engineering Excellence (1998)
Engineering Self-Assessment (1999)
Team Exploration and Enhancement Final Report -
Operations and Maintenance Department (1999)
Maintenance Performance Improvement Plan (1999)
Work Management Self-Assessment (2000)
Technical Training Self-Assessment (2000)
RP Self-Assessment (2000)
Site Wide Self-Assessment (2001)
Maintenance Department Plan (2001)
Operations Training Self-Assessment (2001)
CNS Unplanned LCO Action Statement Entry and Equipment
Reliability Review (2001)
Corrective Action Program Self-Assessment (2001)
RE 20 Outage Critique (2002)
Internal Evaluation of Assessments Performed at Cooper
Nuclear Station (2002)
The Strategic Improvement Plan Revision 0 (2002)
Cooper Nuclear Station Assessment Report April 26,
2002
SA-02-012 Design Modifications Self-Assessment
SA-01-077 Program Implementation Review Project (PIRP)
SA-01-085 EQ Program Interface Assessment
BWROG/PSA Cooper PRA Peer Review Report
2001-02
SA-01-082 AOV Program Self-Assessment
SA-01-006 MOV Program Self-Assessment
SA-01-005 IST Program Self-Assessment
-6-
SA-01-004 ISI Program Self-Assessment
122-00- Fire Protection Program Self-Assessment
27.R01
SA-01-007 Circuit Breaker Program Self-Assessment
SA-01-058 CNS Maintenance Rule Program
SA-02-012 Design Modification Self-Assessment
SA-02-029 BWR Vessel and Internals Project Program
Assessment Cooper Nuclear Station Emergency Preparedness Program
082001 Assessment
Quality Assurance Audit Report 01-11 RE20
Outage
Maintenance Rule Periodic Assessment for the Period
12/17/98 Through 12/16/00
Quality Assurance Audit Reports for Emergency
Preparedness written in 2001 and 2002
2001 and 2002 Quarterly Self-Assessments and Quarterly
Roll-Ups
QA Audit Continuous Improvement Program
Report 01-06
QA Audit Continuous Improvement Program
Report 02-06
QA Field Observation Report FO 01-17
QA QA Review of CNS Corrective Actions
Surveillance
Report S408-
0101
QA Continuous Improvement Program
Surveillance
Report S408-
0102
Miscellaneous Documents
SAP code Ce01052 CNS Programs Improvement Project 1
(E/606)
-7-
ESD-DTG-02 Program Health Monitoring and Reporting 0
EQ Improvement Project, Project Plan 1
Top Ten Equipment Issues List May 13,
2002
CED 2001-0020 CNS-2-HPCI-15CV Replacement
CED 6008866 Replace Motor for Diesel Fuel Oil Transfer Pump
CED 6005622 Portable Diesel Generators for Emergency Operations
Facility (EOF) & the Technical Support Center (TSC)
Cooper Nuclear Station Vessel Internals Program 0
BWR Vessel Internals Data Base
CNS Cornerstone Report May 2002
Reactor Oversight Program Index Performance May 2002
Indicators
Regulatory/Licensing Performance Indicators May 2002
Training Performance Indicators May 2002
Operations Performance Indicators May 2002
ESP Generic Topic Continuing Training completed year- June 2002
to-date
ESP Position-Specific Continuing Training completed June 2002
year-to-date
ESP Position-Specific Continuing Training proposed June 2002
through end of 2002
Listing of all hanging Plant Deficiency Tags June 26,
2002
2001 Radiological Effluents Release Report May 2002
List of current Operator Work-Arounds and Operator June 26,
Concerns 2002
List of all Operations Human Performance Clock Resets June 26,
to date 2002
Emergency Response Organization Roster July 26,
2002
-8-
Instructions to Complete the EP Department 10
Performance Indicators
Emergency Preparedness Departmental Performance January
Indicators 2000 -
May 2002
Nuclear Concerns Tracking Program, Emergency February
Preparedness 2002
Update
Post Work-Week Critiques 2001-2002
Preventive Maintenance Deferral List (72 items) July 15,
2002
1998 Maintenance Improvement Plan Action Status Sept. 29,
1998
Surveillance Test Backlog July 12,
2002
Corrective Maintenance Backlog May 2002
Preventive Maintenance Backlog May 2002
Contractor Control Issues
Maintenance, Outage, and Work Control Performance May 2002
Indicators and Goals
Problem Identification and Resolution Documents
Notifications pertaining to outstanding Control Room Deficiencies as of 6/26/2002
Notifications and work orders generated since 7/1/2002, that did not receive an
Operations review as of 7/24/2002
Notifications pertaining to human performance, procedure, communications, and training issues
in Operations and Maintenance from 1/1/2001 to 5/24/2002
Notifications initiated after January 1, 2001, involving issues in the following areas: corrective
action program; performance measures; organizational effectiveness; management oversight;
utilization of industry information; quality assurance activities; and utilization of onsite and offsite
review committees
Problem Identification Reports initiated from November 2001 to July 2002 related to equipment
rework
Problem Identification Reports initiated from 1998 to 2002 related to equipment parts
-9-
Notifications
10121041 10143372 10154921
10178056 101785282 10175283
10175407 10133627 10117697
Condition Reports
RCR98-0152 RCR98-1092 RCR2001-0888
RCR2001-0969 RCR2001-1056 RCR2001-1108
RCR2001-1226 RCR2001-1263 RCR2001-1468
RCR2001-1571 RCR2001-1599 RCR2001-1632
RCR2002-0001 RCR2002-0095 RCR2002-0109
RCR2002-0269 RCR2002-0286 RCR2002-0459
RCR2002-0504 RCR2002-0516 RCR2002-0548
RCR2002-0770 RCR2002-0953 RCR2002-0969
SCR98-1164 SCR99-0465 SCR2001-0849
SCR2001-1161 SCR2002-0001 SCAQ96-0363
SCR 99-0061
Work Orders
4207483 4211070 4210846
4160040 4215116 4215278
4217623 4217858 4326707
4250515 4251417 4188803
4203287 4211305 4215858
4209991 4217764 4217790
4217769 4212793 4203365
4252056 4236967 4216375
4233326
Meetings
CARB Routing Sheet, June 28, 2002
Instant Procedure Changes, SORC Meeting S2002, June 25, 2002
SORC Meeting Minutes S2002-048, June 25, 2002
SRAB Engineering Meeting Minutes, May 24, 2000
SRAB Meetings 218, 219, 220, and 221 Minutes
-10-
Memoranda/Letters
CNS Letter, dated May 12, 1987, NPPD Response to Generic Letter 87-05"
CNS Memorandum, dated January 18, 1989, AEOD Case Study Report - Operational
Experience Involving Losses of Electrical Inverters
CNS Letter, dated February 20, 1989, Response to Generic Letter 88-14
CNS Letter, dated February 7, 1990, Follow-up Response to Generic Letter 88-14"
CNS Letter, dated March 15, 1990, Final Response to Generic Letter 88-14"
CNS Memorandum, dated February 26, 1991, Information Notice 91-06, Lock-Up of
Emergency Diesel Generator and Load Sequencer Control Circuits Preventing Restart of
Tripped Emergency Diesel Generator
CNS Letter, dated September 9, 1994, Generic Letter 94-02"
QA Escalation Memorandum, dated April 23, 2001, Escalation - Failure to Resolve Emergency
Preparedness Audit Findings
QA Escalation Memorandum, dated April 1, 2002, Escalation - Failure to Resolve Long-
Standing Meteorological System Issues
Letter from M.T. Coyle to CNS Managers, Change Management, June 20, 2002
Various Letters from M.T. Coyle to the Cooper Team, May - June 2002
Miscellaneous
Design Calculation NEDC 92-175, Actuator Sizing Verification for RW-AOV-AO82, AO83,
AO94, and AO95
Generic Letter 87-05, Request for Additional Information Assessment of Licensee Measures to
Mitigate and/or Identify Potential Degradation of Mark I Drywells
Information Notice 87-24, Operational Experience Involving Losses of Electrical Inverters
Generic Letter 88-14, Instrument Air Supply Problems Affecting Safety-Related Equipment
Generic Letter 94-02, Long-Term Solutions and Upgrade of Interim Operating
Recommendations for Thermal-Hydraulic Instabilities in Boiling Water Reactors
Information Notice 91-06, Lock-Up of Emergency Diesel Generator and Load Sequencer
Control Circuits Preventing Restart of Tripped Emergency Diesel Generator
-11-
Information Notice 92-33, Increased Instrument Response Time When Pressure Dampening
Devices Are Installed
Information Notice 94-24, Inadequate Maintenance of Uninterruptible Power Supplies and
Inverters
Cooper Nuclear Station, Nebraska Public Power District, 2001-2005 Business Plan,
March 30, 1999
Cooper Nuclear Station, Nebraska Public Power District, 2001-2006 Strategic Plan,
January 16, 2000
Cooper Nuclear Station, Nebraska Public Power District, 2001-2006 Strategic Plan,
November 22, 2000
Current Performance Indicators associated with Action Plans
List of Quality Assurance Auditors and Areas of Expertise/Experience
2002 Regulatory Issue Summary, Information Notices, and Vendor Information related to
BWRs - Requested Notification, OER Screen, and OER Reply
APPENDIX C
LIST OF ACRONYMS USED
CFR Code of Federal Regulations
CNS Cooper Nuclear Station
EQ environmental qualification
ERO emergency response organization
HPCI high pressure coolant injection
LCO limiting condition for operation
NRC U.S. Nuclear Regulatory Commission
PM preventive maintenance
PMIS plant monitoring and information system
QC quality control
RCR resolve condition report
RFO refueling outage
SCR significant condition report
STE shift technical advisor
TIP The Strategic Improvement Plan
WBS work breakdown structure
APPENDIX D
ACTION PLAN REVIEW CRITERIA TABLES
-2-
ACTION PLAN NUMBER - 5.1.1.1
ACTION PLAN TITLE - Organizational Alignment
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The plan referenced WBS 1.1.1 and 1.1.2 inputs.
2. Characterization of issues in WBS are adequate to The WBS extent of condition did not identify all the important individual issues. The team identified three more
address individual issues from database included in the specific problems associated with Organizational Alignment. The specific problems were (in order of the
extent of condition review for that WBS. frequency of appearance):
1. Failure of the station management to establish and enforce standards and priorities.
2. Failure of the station personnel to adhere to written plans, procedures, and guides.
3. Failure of the station management to provide congruent policies, goals and priorities among the various
departments.
3. Action Plan Objective, Problem Statement, and Causal The Action Plan Objective, Problem Statement, and Causal Factors were appropriately aligned. However, the
Factors are appropriately aligned. Action Plan was not complete based on the WBS not identifying all of the important issues (see Review Criteria 2
above).
4. Action Plan steps address the Problem Statement, The Action Plan steps did not address the entire extent of condition, including the three additional problems
Causal Factors, and extent of condition described in missed as part of the WBS extent of condition development. Two of eight examples are included:
WBS package.
1. There was no implementation guide for writing departmental expectations and standards in support of the
station procedure.
2. The plant had not issued a directive to assure that station personnel comply with 0-CNS-24, CNS
Standards and Expectations, even though the WBS extent of condition and Action Plan described this
problem.
-3-
5. Action Plan steps provide sufficient detail to define The Action Plan steps did not contain sufficient detail for the described actions. For example:
action to be taken.
Training for implementing the "new process" had not been incorporated into the Action plan. No technical
assistance staff had been designated to help departments ensure that departmental guidance was consistent
with the station expectations and standards.
Procedure 0-CNS-24 needed to be upgraded. The Action Plan owner indicated it was their intent to do bench
marking visits at other plants and upgrade the procedure based on their findings. This was not in the plan.
The licensee intended to gather information from other plants, but that is not in the Action Plan.
6. Deliverable identified for Action Plan steps are well The deliverables were adequately defined to determine the results of the action taken.
defined and consistent with the action being taken. (Are
words such as effective and reduced to minimal
defined?)
7. Do the performance indicators provide an adequate There was no direct link between the performance indicators and the actions. The behaviors required by
measure of performance as it relates to the objectives for personnel to assure consistency among the standards and expectations were not being monitored by the plan.
improved performance in the Action Plan? Are the The metrics in the plan (outage milestones, management changes, online schedule stability, etc.) measured
thresholds in the performance indicators appropriate? potential outcomes of failures in maintaining congruent standards and expectations.
8. Has a notification been written to enter Action Plan in Notification 10169817
CAP?
-4-
ACTION PLAN NUMBER - 5.1.1.2
ACTION PLAN TITLE - Accountability
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan referenced WBS 1.1.1 and 1.1.3.
2. Characterization of issues in WBS are adequate to address The extent of condition and database information cited the primary problem in this area as weak
individual issues from database included in the extent of condition leadership; however, accountability was not specifically referenced and this was/is a problem at the
review for that WBS. site. The Action Plan owner was not involved in the development of the WBS or in the development of
the Action Plan.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan Objective, Problem Statement, and Causal Factors were appropriately aligned.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Two problem areas identified in the WBS were not included in the Action Plan actions:
and extent of condition described in WBS package.
1. Management provided inconsistent support for station activities.
2. Roles and responsibilities are not consistently established for personnel at the site. Specifically,
there was not a station procedure, policy statement, or directive that established the expected
roles and responsibilities.
Three additional areas identified in the WBS, dealing with management observations were not included
in Action Plan 5.1.1.2, but were covered by Action Plan 5.1.1.5. However, there was nothing in the
WBS that linked Action Plan 5.1.1.5 to the WBS areas of concern.
Nothing in the problem statement or the causal factors identified specific accountability behaviors that
were missing from the organization.
5. Action Plan steps provide sufficient detail to define action to be The steps provided insufficient detail. In particular, Action 3 was to train all station personnel on
taken. accountability behaviors in accordance with OZ training; however, no attempt was made to ensure
the OZ behaviors were consistent with those behaviors outlined during a series of management
meetings (Action Plan Action 1).
6. Deliverable identified for Action Plan steps are well defined and The deliverable statements were adequate for the actions defined in the plan.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
-5-
7. Do the performance indicators provide an adequate measure of The Action plan actions contained no criteria or process for monitoring the effectiveness of the
performance as it relates to the objectives for improved implementation of the accountability behaviors taught in the training program. The specified means for
performance in the Action Plan? Are the thresholds in the monitoring was a self-assessment scheduled for first quarter 2003. No specific measurement criteria
performance indicators appropriate? were included in the self-assessment action. The listed performance indicators may provide an
indirect measurement of changes to accountability behavior.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169824
-6-
ACTION PLAN NUMBER - 5.1.1.3
ACTION PLAN TITLE - Prioritization & Planning
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. WBS folders 2.3.1 and 2.3.2 were not referenced in the action plan. However, the action plan owner
was aware of the issues and the items were appropriately addressed.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are The items within the action plan were appropriately aligned. However, the action plan owner was
appropriately aligned. relying on the satisfactory completion of other action plans in Revision 1 of the TIP. The additional
action plans were not aligned with Action Plan 5.1.1.3.
4. Action Plan steps address the Problem Statement, Causal Factors, The action plan did not include corrective actions to address the development, coordination, and
and extent of condition described in WBS package. prioritization of departmental priorities. The action plan owner was not aware of any other action plan
that would address this issue.
5. Action Plan steps provide sufficient detail to define action to be Steps 2 through 7 did not include sufficient detail to evaluate. Specifically, no documented instructions
taken. had been developed for the use of the site wide schedule by station personnel. A benchmark site had
not been determined nor had the expectations for the benchmark been developed. In addition, the
station resource utilization process and the resource planning process had not been developed.
6. Deliverable identified for Action Plan steps are well defined and Step 3 required the completion of a benchmark trip. The associated deliverable was to complete the
consistent with the action being taken. (Are words such as trip and develop a report. The actual deliverable should involve accomplishment of the benchmark trip
effective and reduced to minimal defined?) objectives. Step 5 required a review of the integrated sitewide schedule. The associated deliverable
was a completed self assessment. The actual deliverable should involve accomplishment of the
review objectives.
7. Do the performance indicators provide an adequate measure of The performance indicators associated with the action plan did not adequately assess the
performance as it relates to the objectives for improved effectiveness of the planned corrective actions or causal factors. Specifically, an assessment of the
performance in the Action Plan? Are the thresholds in the development and integration of priorities would need to be performed.
performance indicators appropriate?
Step 1, which involved the preparation of briefing paper on the station work prioritization process
procedure, was statused as complete. An e-mail was sent to station personnel on May 7, 2002, which
included the briefing paper. There was no required response to indicate that the material had been
read and understood.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169799
-7-
ACTION PLAN NUMBER - 5.1.1.4
ACTION PLAN TITLE - Organizational/Human Behaviors
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan referenced WBS 1.2.1, 1.2.2, 1.2.3, and 1.2.4.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review appropriately identified the areas of concern.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are The problem statement emphasized behaviors that need to be changed at the site including "lack of
appropriately aligned. trust, lack of pride, poor communication, and weak teamwork." The causal factors listed did not include
any reference to those specific behaviors.
4. Action Plan steps address the Problem Statement, Causal Factors, The action plan did not address all of the identified areas of concern. Specifically:
and extent of condition described in WBS package.
1. The action plan objective focused attention on measuring the organizational culture/climate. The
Action Plan actions did not address any changes to be made in the individual behaviors.
2. Current station expectations and standards for human behavior were not included in station
written requirements. Behaviors identified as critical to operation were not identified by the Action
plan and were not to be included, based on the TIP action plan actions. There was no cascaded
set of written procedures, instructions or guides within the functional departments to capture the
implementation of appropriate human behaviors. The TIP action plan did not identify specific
human behaviors that were critical to plant operation and did not include a means for identifying
those specific behaviors in the future.
5. Action Plan steps provide sufficient detail to define action to be The Action Plan steps did not provide sufficient detail to ensure the desired action is completed. For
taken. example:
1. There was no requirement in the Action Plan actions to distribute information to station personnel
about the "new" required behaviors.
2. The Action plan owner stated that training will be needed to implement the "new" behavior
expectations and that training is not included in the TIP actions.
6. Deliverable identified for Action Plan steps are well defined and The deliverable statements were adequate for the actions defined in the plan.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
-8-
7. Do the performance indicators provide an adequate measure of The performance indicators used did not measure directly the use of the "new" behaviors. The
performance as it relates to the objectives for improved indicators used in the plan are NRC allegation, turnover rate, management changes. Per the site
performance in the Action Plan? Are the thresholds in the human resource representative, the turnover rate at the plant was currently about 7percent annually. It
performance indicators appropriate? was driven predominantly by market conditions in the local area and not by plant culture. There was
no behavior change metric included in the performance measures - behavior observations. There was
no accountability measure built into the Action plan for human behaviors.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169821
-9-
ACTION PLAN NUMBER - 5.1.1.5
ACTION PLAN TITLE - Management Observation Program
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. WBS folders 2.4.3 and 3.3.1 were not reviewed prior to the action plan being approved. The action
plan owners did not know the WBS folders existed. However, they were reviewed following approval
of the action plan and the issues had been fortuitously addressed.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are The action plan owners indicated that they were crediting action items in Action Plans 5.1.4.1 and
appropriately aligned. 5.2.6.2. However, no link between the action plans was provided.
Action items were not linked to the specific causal factors.
4. Action Plan steps address the Problem Statement, Causal Factors, The action plan steps appropriately addressed the problem statement, causal factors, and WBS extent
and extent of condition described in WBS package. of condition.
5. Action Plan steps provide sufficient detail to define action to be Steps 5, 6, 7, and 8 did not provide sufficient detail. A decision had not been made on which facility
taken. to benchmark, the benchmark objectives were not developed, the methodology for the self
assessment was not determined, and a self assessment plan was not been developed.
6. Deliverable identified for Action Plan steps are well defined and Step 8 indicated that the deliverable was a notification to change processes and procedures if
consistent with the action being taken. (Are words such as necessary. The deliverable should be the actual procedure change and not an item to perform a
effective and reduced to minimal defined?) procedure change. Step 5 indicated that the deliverable for the benchmark trip was a benchmark
report. The deliverable should be a statement of what the benchmark trip is intended to accomplish.
Step 6 indicated that the deliverable for the self assessment was a self assessment report. The
deliverable should be a statement of what the assessment is intended to accomplish.
-10-
7. Do the performance indicators provide an adequate measure of Performance indicators involved the number and quality of observations. Given the low number of
performance as it relates to the objectives for improved required observations (1 per month for 80 of the 130 supervisors and managers on site), the
performance in the Action Plan? Are the thresholds in the performance indicators may not be useful in improving human performance. In addition, the
performance indicators appropriate? performance indicator monitored the total number of observations performed (mandatory and non-
mandatory participants) but did not assess the completion rate of those required to participate.
The quality index was not formalized in Procedure 0-CNS-07, Management Field Observations.
Therefore a meaningful interpretation of the data was questionable. One individual in the performance
analysis department was responsible for assigning a subjective value of 1 to 5 (not at all useful to
meets all requirements) for each observation. No requirement existed to provide feedback to
individuals completing unsatisfactory observations.
Trending of negative observations was not established. For example, of the 272 negative
observations in 2002 (Jan-May), 31 involved housekeeping issues and 27 involved communication of
roles and responsibilities. However, no review for a potential trend had been completed.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169803
-11-
ACTION PLAN NUMBER - 5.1.1.6
ACTION PLAN TITLE - Performance Monitoring
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. WBS 2.4.3 was not referenced in the action plan. The action plan owner had not reviewed WBS 2.4.3
or 1.1.1 prior to approval of the action plan. As of June 27, 2002, the action plan owner had not
reviewed WBS 2.4.3 and 1.1.1. The action plan owner indicated that he was familiar with performance
monitoring issues and was able to develop the plan without reviewing the WBS folders. In addition,
the action plan owner indicated that the plans would be reviewed before Revision 2 was issued.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Causal factors were not linked to the action items. There was no link to the accountability
appropriately aligned. improvements described in Action Plan 5.1.1.2.
4. Action Plan steps address the Problem Statement, Causal Factors, The action plan did not address weaknesses in the development, use, and accountability of
and extent of condition described in WBS package. departmental performance indicators.
5. Action Plan steps provide sufficient detail to define the action to be Step 6 required a revision to Procedure 0-PI-01, Performance Indicator Program. However, the
taken. details associated with the revision were under development as of June 24, 2002.
6. Deliverable identified for Action Plan steps are well defined and Deliverable statements were appropriate.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Performance measures to monitor improvement do not address the quality of performance indicators.
performance as it relates to the objectives for improved No effectiveness reviews were identified to determine if appropriate performance indicators were
performance in the Action Plan? Are the thresholds in the developed.
performance indicators appropriate?
Step 1 as statused as complete even though performance indicators were under development.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169801
-12-
ACTION PLAN NUMBER - 5.1.1.7
ACTION PLAN TITLE - Succession Planning
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The Action Plan referenced WBS 1.1.4.
2. Characterization of issues in WBS are adequate to address The issues were appropriately characterized in the WBS extent of condition review. The Action Plan
individual issues from database included in the extent of condition owner was not involved in the development of the WBS. As part of the Action Plan development, the
review for that WBS. Action Plan owner stated that no systematic attempt was made to correlate the information. The
Action Plan was developed based on his prior knowledge of the problem and the succession planning
process procedure.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan Objective, Problem Statement, and Causal Factors were properly aligned.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, The Action Plan steps adequately addressed the Problem Statement, Causal Factors, and extent of
and extent of condition described in WBS package. condition.
5. Action Plan steps provide sufficient detail to define action to be All actions contained sufficient detail to define the action(s) to be taken.
taken.
6. Deliverable identified for Action Plan steps are well defined and The deliverables were well defined and indicated a throughly developed plan.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Yes. The performance indicators (employee turnover rate, number of key management positions filled
performance as it relates to the objectives for improved in accordance with succession plan, and successful completion of development plans) were effective
performance in the Action Plan? Are the thresholds in the measures of desired performance.
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169813
-13-
ACTION PLAN NUMBER - 5.1.1.8
ACTION PLAN TITLE - Learning Organization and Industry Participation
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The Action Plan referenced WBS 3.3.1 and 3.3.2
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern. The Action Plan owner
individual issues from database included in the extent of condition did not participate in the develop of the WBS or the Action Plan. In addition, he did not review the
review for that WBS. WBS as part of his review of the Action Plan.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan Objective, Problem Statement, and Causal Factors were appropriately aligned.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Two examples were identified where the Action Plan steps did not address all of the issues:
and extent of condition described in WBS package.
1. The WBS and the Action Plan objective implied accountability was an issue in not getting self-
assessment accomplished and recommended self-assessment findings and recommendations
implemented. However, the Action Plan did not address accountability. The licensee indicated
that accountability was addressed by Action Plan 5.1.1.2, but Action Plan 5.1.1.2 is not linked to
this plan.
2. Causal Factor 1 indicated that self-assessment findings and recommendation were not being
consistently entered into the corrective action program (CAP). There were no steps in the Action
Plan to verify that issues identified during self-assessments were being entered into the CAP and
effectively resolved.
5. Action Plan steps provide sufficient detail to define action to be Steps 2, 3, and 4 did not have sufficient detail to determine the scope of the action. In particular,
taken. step 3 stated that benchmarking will be conducted. There were no details as to how the plants will be
selected; how many plants will be visited; and, how will the self-assessment programs be evaluated.
6. Deliverable identified for Action Plan steps are well defined and Deliverable statements do not always state the desired outcome. For example, Action 5 specified
consistent with the action being taken. (Are words such as conducting a semi-annual assessment of the self-assessment program with the deliverable being the
effective and reduced to minimal defined?) assessment report. The deliverable should also include using the results of the assessment to effect
change.
7. Do the performance indicators provide an adequate measure of The two current performance indicators (new as of February 2002) did not provide an adequate
performance as it relates to the objectives for improved measure of improved performance. The indicators looked at numbers of open self-assessment actions
performance in the Action Plan? Are the thresholds in the and average age of the open items. The indicators did not measure quality of the assessment and
performance indicators appropriate? implementation of findings/recommendations.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169822
-14-
ACTION PLAN NUMBER - 5.1.1.9
ACTION PLAN TITLE - Program Management
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The Action Plan referenced WBS 3.4.4, but did not reference WBS 3.4.2. WBS 3.4.2 cross referenced
Action Plan 5.1.1.9.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern. The Action Plan owner
individual issues from database included in the extent of condition was not involved in the development of the WBS extent of condition. He utilized the WBS as part of
review for that WBS. the Action Plan development.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan Objective, Problem Statement, and Causal Factors were appropriately aligned.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, The Action Plan steps addressed the Problem Statement, Causal Factors, and extent of condition.
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be The Action Plan steps had sufficient detail to define the actions necessary to implement the steps, with
taken. the exception of Step 7. Step 7 simply stated Conduct Effectiveness Review. The step provided no
details as to how the evaluation would be conducted or what to consider.
6. Deliverable identified for Action Plan steps are well defined and Deliverable statement does not always state the desired outcome. Steps 2, 3, and 7 did not discuss
consistent with the action being taken. (Are words such as verifying changes in behavior as a desired outcome. The steps focused only on the procedural or
effective and reduced to minimal defined?) program aspect and not verifying that actual performance was changed. For Step 6, the deliverable
was Completion of Action Plan 5.1.1.4, but that plan did not reference Action Plan 5.1.1.9.
7. Do the performance indicators provide an adequate measure of The two performance indicators did not provide an adequate measure of improved performance. The
performance as it relates to the objectives for improved first indicator, Completion of Schedule Milestones for Each Program Plan, was simply a completion
performance in the Action Plan? Are the thresholds in the date schedule - nothing to do with measuring change in performance. The other indicator, CNS
performance indicators appropriate? Program Health Indicators, was a qualitative report with no defined criteria for measuring program
performance. The report is developed by the program owner, every 2 years, and has no ongoing
performance indicators. The team reviewed several reports and determined that they were snap-shots
of current program performance based on the program owner's opinion (little or no value for
determining trends in program performance). In addition, the deliverables were not defined; therefore,
the team was unable to determine if licensee could measure changes in the effectiveness of program
management and performance.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169820
-15-
ACTION PLAN NUMBER - 5.1.2.1
ACTION PLAN TITLE - Programmatic/Process Changes
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The Action Plan referenced WBS 1.3.2 and 1.4.2.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern. The Action plan owner
individual issues from database included in the extent of condition was not involved in the development of the WBS and did not use the WBS as in input into the Action
review for that WBS. Plan.
3. Action Plan Objective, Problem Statement, and Causal Factors are There was no problem statement dealing with the causal factor of ineffective internal communication.
appropriately aligned. This was a failure to integrate all of the causes identified with the problem statement.
4. Action Plan steps address the Problem Statement, Causal Factors, The planned actions did not address the full range of the problems identified in the causal factors, the
and extent of condition described in WBS package. extent of condition reviews, and the WBS. There was no accountability in the Action Plan actions for
implementing the change management process. The licensee has change management guidance and
it was not being used by site personnel. The Action Plan did not require the use of the guidance.
5. Action Plan steps provide sufficient detail to define action to be The Action Plan steps lacked detail. For example:
taken.
1. Technical assistance for implementing the change management process was not established in
the Action Plan actions.
2. Station management has not specified in the Action Plan the breadth and depth of the information
searches required for changes.
3. Action Plan Step 3 specified that the management guidance be proceduralized; however, the
current guidance had significant short comings and a revision to the guidance was not included in
the plan.
6. Deliverable identified for Action Plan steps are well defined and Steps 5, 6, and 7 did not provide sufficient detail. A decision had not been made on which facility to
consistent with the action being taken. (Are words such as benchmark; the benchmark objectives were not developed; the methodology for the self assessment
effective and reduced to minimal defined?) was not determined; and a self-assessment plan was not developed. For instance: Step 5 indicated
that the deliverable for the benchmark trip was a benchmark report. The deliverable should be a
statement of what the benchmark trip is intended to accomplish.
7. Do the performance indicators provide an adequate measure of This plan had no current performance indicators and none were planned.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169825
-16-
ACTION PLAN NUMBER - 5.1.3.1
ACTION PLAN TITLE -External Communications
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address Yes.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are WBS 1.3.1 suggests that there is a general lack of understanding of the regulatory process with
appropriately aligned. respect to two-way communications. The existing causal factors do not seem to cover this.
Additionally, the action plan objective does not discuss external communications issues with outside
organizations other than the NRC, yet this was an issue discussed in the WBS.
4. Action Plan steps address the Problem Statement, Causal Factors, The action plan steps are largely a series of procedural enhancements followed by some form of
and extent of condition described in WBS package. assessment. From the WBS, however, much of the problem seems to revolve around the siloing
effect amongst the workgroups involved. This is not addressed in the action plan. However, the
licensee did self-identify this issue as a generic problem prior to the teams arrival on site, and intends
to create an additional action plan (5.1.4.3, Teamwork) to address the issue.
5. Action Plan steps provide sufficient detail to define action to be Yes.
taken.
6. Deliverable identified for Action Plan steps are well defined and The deliverables for Steps 11, 12, and 13 are vague. The actual scope of the documents to be revised
consistent with the action being taken. (Are words such as and implemented is not specified. The target date for completion of these steps is listed as TBD.
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Performance Indicators for this action plan are still in the very early stages of development.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10165879
-17-
ACTION PLAN NUMBER - 5.1.4.1
ACTION PLAN TITLE - Pride/Excellence
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Not in all cases. The action plan provided a good outline but some steps lack detail. For example,
taken. step 1 included benchmarking to define a standard of excellence. Subsequent steps develop those
standards into procedures, present it to personnel, and monitor its effectiveness. Without knowing the
results of step 1, it is difficult to include detail in the subsequent steps. Another example is step 12
which develops a peer observation program. It does not describe what steps were necessary to
complete this action, such as, to whom the program would apply, what observations would be
included, what training would be necessary, what actions would be taken based on the observations,
or how the observations would be tracked and trended procedure development.
6. Deliverable identified for Action Plan steps are well defined and Yes
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of The PIs and their thresholds have not been fully developed. Additionally, action plan step 10
performance as it relates to the objectives for improved references analyzing plant data such as management observations, self-assessments, CAP data, etc.,
performance in the Action Plan? Are the thresholds in the while the Pis selected to monitor overall plan performance measure something else.
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169819
-18-
ACTION PLAN NUMBER - 5.1.4.2
ACTION PLAN TITLE - Trust/Culture
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. WBS 1.2.4 was not referenced.
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Not in all cases. Causal Factor 1 is addressed in Action Plan 5.1.1.5; however, that action plan lacks
appropriately aligned. sufficient detail to verify that it directly address the problem stated in the causal factor. The problem
stated is that managers lack visibility with the workforce. Action Plan 5.1.1.5 develops the observation
program but does not contain detail regarding the number of required observations, ie., the number of
times management will be required to be in the plant. The licensees intent is to significantly increase
the number of required observations, which will address this problem.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Not completely. 1) Steps 1 & 2 collect data and Steps 3 & 4 revise policies and processes, however,
taken. neither Step 3 or 4 indicate that the info collected in Steps 1 & 2 would be used. 2) There is no step to
present the changes made in step 3 to the employees, even though this is already underway to some
degree. 3) Step 3 does not include actions to include the findings from this step into a policy.
Discussions with action plan owner indicated that he intended to revise the policy that is being
developed in step 2 once the info is gathered.
6. Deliverable identified for Action Plan steps are well defined and Yes.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of No, the PI section indicated that an Employee Concerns PI would be used. In fact, there is no PI,
performance as it relates to the objectives for improved however, there are survey results. The action plan owner identified this during discussions with
performance in the Action Plan? Are the thresholds in the inspector. Also, the action plan owner indicated that they might use self-identified notifications as an
performance indicators appropriate? indicator for this action plan. The PIs are still being developed.
8. Has a notification been written to enter Action Plan in CAP? Notifications 10169910 and 10169918
-19-
ACTION PLAN NUMBER - 5.1.5.1
ACTION PLAN TITLE - Oversight and Assessment
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The Action Plan referenced WBS 3.3.1 and 3.3.2.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition identified the significant areas of concern. The Action Plan owner was
individual issues from database included in the extent of condition not involved in the development of the WBS or the Action Plan and did not review the WBS as part of
review for that WBS. his review of the Action Plan.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan objective discussed defining individual roles and responsibilities and holding those
appropriately aligned. individuals accountable for self-assessments; however, nothing in the Problem Statement or Causal
Factors discussed accountability.
4. Action Plan steps address the Problem Statement, Causal Factors, The Action Plan steps did not address two of the issues identified in the Problem Statement, Causal
and extent of condition described in WBS package. Factors, and extent of condition. Specifically:
1. Causal Factor 1 dealt with not consistently entering self-assessment findings into the corrective
action program (CAP). The program was modified to require entering items into CAP, but did not
have the licensee actually verify that the self-assessment findings are being entered into CAP.
2. The WBS did not specifically state that the self-assessment process lacked accountability, but it
implied that it lacked accountability. The lack of accountability is not addressed in the Action
Plan.
5. Action Plan steps provide sufficient detail to define action to be Steps 2, 3, and 4 did not contain sufficient detail to define the action to be taken. As an example,
taken. step 3, Initiate engagement with departmental Self-assessment coordinators and establish roles and
responsibilities, did not contain any details as to how this is to be accomplished and what is to be
discussed.
6. Deliverable identified for Action Plan steps are well defined and Deliverable did not always state the desired outcome. Steps 3, 4, 5, and 7-11 did not discuss verifying
consistent with the action being taken. (Are words such as changes in behavior as a desired outcome. The steps focused only on the procedural or program
effective and reduced to minimal defined?) aspect and not verifying that actual performance was changed.
7. Do the performance indicators provide an adequate measure of The Action Plan listed two current performance indicators and three additional performance indicators
performance as it relates to the objectives for improved under consideration for development. Neither of the current performance indicators measured
performance in the Action Plan? Are the thresholds in the changed performance. Additionally, the current performance indicators did not specify the actions to
performance indicators appropriate? be taken when thresholds were crossed.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169827
-20-
ACTION PLAN NUMBER - 5.2.1.1
ACTION PLAN TITLE - Create an Operationally Focused and Aligned Organizational Culture
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. No, It should have referenced 3.4.2 and 3.4.3
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are No. The causal factor states that Operations leadership is lacking, however, the Focus Area and
appropriately aligned. Problem Statement considers the problem to be site-wide.
4. Action Plan steps address the Problem Statement, Causal Factors, Not completely. The lack of operational focus was identified as being exhibited site-wide; however, the
and extent of condition described in WBS package. majority of the action plan steps were focused primarily in the Operations Department.
5. Action Plan steps provide sufficient detail to define action to be No, detail is lacking in most steps. For example; Step 1.1 does not discuss how this step will be
taken. completed, if it involves procedure changes, training, benchmarking, etc.; Step 1.3 doesnt discuss
where the problem lies, how it will be fixed, any training needed or procedure changes needed, etc.
Nor does it address any other organizations outside of Operations. Step 1.4 gives no detail and
provides no plan as to how the step is to be accomplished. Step 3.2 similarly provides no plan for
completion.
6. Deliverable identified for Action Plan steps are well defined and No. Step 1.2 is vague in that it simply states that an enhanced process will be provided. Step 1.4
consistent with the action being taken. (Are words such as requires a review to be conducted but doesnt actually reduce any of the workarounds. Step 3.2 only
effective and reduced to minimal defined?) requires that a charter and schedule be developed. It provides no details as to who the team will
consist of, what the charter will be, what their purpose is, etc.
7. Do the performance indicators provide an adequate measure of No. The licensee identified 14 PIs which collectively could provide some indication that the action plan
performance as it relates to the objectives for improved was not successful. However, there is no direct tie between the indicators and the action plan
performance in the Action Plan? Are the thresholds in the performance. Most of the PIs are in the process of being revised. Definitions for the indicators and
performance indicators appropriate? thresholds do not exist in many cases.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169831
-21-
ACTION PLAN NUMBER - 5.2.2.1
ACTION PLAN TITLE - Emergency Response
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan references WBS 3.5.1, Emergency Preparedness.
2. Characterization of issues in WBS are adequate to address The specific technical database issues associated with WBS 3.5.1 are not directly addressed in the
individual issues from database included in the extent of condition WBS characterization of problems, and the characterizations are themselves high-level, general, and
review for that WBS. not well focused. Each of the database issues can be indirectly correlated or associated with one or
more of the of problems characterized in WBS 3.5.1.
The licensee identified only 8 technical issues with applicability to WBS 3.5.1. This small sample size
does not provide sufficient information to support a useful extent of condition determination. Because
the licensee did not believe they had an adequate root or common cause analysis of emergency
preparedness problems, the characterization of issues in WBS 3.5.1 was largely based on WBS 3.4.4.
WBS 3.5.1 concludes that: The CNS EP program has exhibited multiple symptoms of declining
performance over an extended period of time. The Assessment Results section of WBS 3.5.1 fails
to clearly identify or describe these symptoms, and fails to relate them to the identified problems.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan 5.2.2.1 Objective is not consistent with the Problem Statement in that an emergency
appropriately aligned. response organization whose performance is inconsistent, not at highest standards, and is
occasionally ineffective, is not the stated problem intended to be addressed by the Action Plan.
The problem identified in the problem statement is that management failed to recognize and correct
declining performance. Causal Factors 2 and 4 have strong linkages to this problem, Causal Factors 1
and 3 have weak linkages, and Casual Factor 5 does not appear to be associated with the problem
statement.
A strong linkage exists between the Objective and Causal Factor 1. A clear but weaker linkage exists
between the Objective and Causal Factor 4. There is no apparent linkage between the Objective and
the remaining Causal Factors.
-22-
4 Action Plan steps address the Problem Statement, Causal Factors, The steps in Action Plan 5.2.2.1 have only an indirect linkage to the technical issues identified in the
and extent of condition described in WBS package. issues database.
9 of 40 steps in Action Plan 5.2.2.1 have strong and direct linkage to the action plan Problem
Statement. Although some of the remaining steps have weak or indirect linkage to the Problem
Statement, most have no apparent linkage. 13 of 40 steps have strong and direct linkage to the stated
action plan Objective. 18 of 40 steps do not appear to be associated with any of the listed Causal
Factors.
5. Action Plan steps provide sufficient detail to define action to be The steps in Action Plan 5.2.2.1 generally provide good levels of detail to define the actions to be
taken. taken.
6. Deliverable identified for Action Plan steps are well defined and The identified deliverables for the steps in Action Plan 5.2.2.1 are generally well defined and consistent
consistent with the action being taken. (Are words such as with the action being taken. However, the deliverables for many steps contain words such as fully
effective and reduced to minimal defined.? functional, and acceptable, which are not defined in the Action Plan.
7. Do the performance indicators provide an adequate measure of The performance measures did not provide a direct measure of the effectiveness of the action plan
performance as it relates to the objectives for improved steps. The three performance indicators measured the ERO performance and aligned closely with the
performance in the Action Plan? Are the thresholds in the action plan objective; however, no specific performance measures were developed for the steps
performance indicators appropriate? related to maintaining and upgrading emergency preparedness equipment. The performance indicator
ERO Staffing (Vacancies), intended to measure whether the overall ERO was fully staffed, was not
an effective measure because the input parameters were poorly defined and the red threshold could
not be exceeded regardless of the number of vacancies in some emergency response positions. Also,
the licensee had not proceduralized definitions and calculational methods for any of the performance
indicators intended to measure the effectiveness of the action plan.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169823
-23-
ACTION PLAN NUMBER - 5.2.3.1
ACTION PLAN TITLE - Outage Management
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. WBS 2.4.2 is not listed in the action plan.
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Yes
taken.
6. Deliverable identified for Action Plan steps are well defined and Yes
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Yes
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169838
-24-
ACTION PLAN NUMBER - 5.2.3.2
ACTION PLAN TITLE - Planning/Timeliness
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan references WBS 1.1.1 and 1.1.3.
2. Characterization of issues in WBS are adequate to address No. Issues involving rework and human performance errors impacting the outage schedule that were
individual issues from database included in the extent of condition identified in the RFO 20 Outage Critique, RCR 2002-0051, were not characterized in the WBS. After
review for that WBS. additional review and interviews with the Action Plan Owner a determination was made that the Work
Package Development focus area, specifically Action Plan 5.2.5.2, and TIP Human Performance
Action Plan, 5.1.4.1, had actions that would address the human performance issues.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Yes
taken.
6. Deliverable identified for Action Plan steps are well defined and Some of the deliverables lack definition. For example, several are just a re-statement of the action
consistent with the action being taken. (Are words such as plan step and do not discuss the specifics or quality of the completed product. Steps lacking definition
effective and reduced to minimal defined?) included Steps 4, 5, and 7.
7. Do the performance indicators provide an adequate measure of One of the performance indicators, Pre-outage Milestone Schedule Adherence, was reviewed and
performance as it relates to the objectives for improved determined to be adequate. However, the performance indicator to track outage scope growth had not
performance in the Action Plan? Are the thresholds in the yet been developed.
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169837
-25-
ACTION PLAN NUMBER - 5.2.3.3
ACTION PLAN TITLE - Scheduling/Monitoring
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan references WBS 1.1.1 and 1.1.3
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Yes
taken.
6. Deliverable identified for Action Plan steps are well defined and Yes
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Yes
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169846
-26-
ACTION PLAN NUMBER - 5.2.4.3
ACTION PLAN TITLE - Monitoring
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes, with the exception of 2.4.3
2. Characterization of issues in WBS are adequate to address The extent of condition review for outage monitoring did not reflect the issues that were identified in the
individual issues from database included in the extent of condition WBS 2.4.3 issues database. The issues that were identified, eight total, were problems relating to the
review for that WBS. engineering department and had nothing to do with management monitoring of outages. Following
interviews, it was found that the licensee had used corrective action program documentation to
conceptually validate the need for the action plan. The WBS extent of condition review was developed
based on the need for the action plan rather than having been developed based on issues identified in
the issues database. It was determined that this was an example of the licensee not following an
established process for TIP development in that site knowledge was used to identify the problem and
then develop the extent of condition.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, The issues in the WBS database are specific to Engineering. There were a total of 8 issues, one of
and extent of condition described in WBS package. which had to do with quality and timeliness of engineering support for plant activities. There are no
specific causal factors or actions in this action plan to fix the Engineering problems identified in the
WBS issues database. All of the issues that were listed in the WBS issues database were covered by
other action plans, particularly plan 5.2.6.3.
5. Action Plan steps provide sufficient detail to define action to be No.
taken.
6. Deliverable identified for Action Plan steps are well defined and No, terms such as identify and establish are used for the action items.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of There were no performance indicators identified.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169851
-27-
ACTION PLAN NUMBER - 5.2.4.4
ACTION PLAN TITLE - Contract Administration
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes
2. Characterization of issues in WBS are adequate to address WBS 2.4.4 extent of condition review did not identify the performance issues related to human
individual issues from database included in the extent of condition performance errors that were identified in the WBS issues database. However, Action Plan 5.2.5.2,
review for that WBS. Action Plan, 5.1.4.1, had actions that addressed human performance errors. However, Action Plan
5.2.4.4 did not reference these other action plans.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Yes
taken.
6. Deliverable identified for Action Plan steps are well defined and No, deliverables for steps 1, 3, and 4 were not well defined.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of No, performance indicators had not yet been developed.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification10169876
-28-
ACTION PLAN NUMBER - 5.2.5.1
ACTION PLAN TITLE - Purpose/Accountability
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes
2. Characterization of issues in WBS are adequate to address WBS 2.1.1, Accountability/Purpose, was only assigned to this action plan. A review of the database
individual issues from database included in the extent of condition generated for this WBS indicated that there were several (2 Pages) of issues that were failures to
review for that WBS. follow/ failure to implement or adhere to procedures. The failure of the worker to follow procedures
does not show up in the characterization of the issues. These performance issues were addressed by
Action Plan 5.1.4.1.
During the review of the applicable WBS folders, it was identified that two folders, WBS 2.1.3,
Teamwork and WBS 2.2.4, Technical Support, were not referenced appropriately by any of the action
plans for this focus area. Consequently, issues identified during the extent of condition reviews that
were applicable to this focus area were not characterized in the causal factors for the action plans
reviewed. Specifically, WBS 2.1.3 identified weaknesses associated with Teamwork in that scheduling
and coordination problems frequently existed but were not effectively identified and communicated,
management/supervision did not establish and reinforce expectations, and there was a lack of
ownership and accountability in unit organizations. The licensee had identified this prior to the teams
on-site inspection and they stated that the issues identified in WBS 2.1.3 would be addressed by the
development of Action Plan 5.1.4.3, Teamwork, for Revision 2 of TIP. Also, WBS 2.2.4 identified
numerous issues associated with the lack of engineering support during the work package planning
process. Upon further review of the action plans in this focus area and interviews with the Action Plan
Owner, the team determined that this issue was addressed by step 4 in Action Plan 5.2.5.1. Action 4
addresses the definition of the roles and responsibilities of various individuals/groups involved in the
development of work packages.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
-29-
5. Action Plan steps provide sufficient detail to define action to be Sufficient detail was not provided. The ability to determine whether the actions will correct the
taken. problems identified in the causal factors cannot be concluded since the actions are not specific. For
example:
Action 1. This action was to develop controls to implement the 12 week Rolling system window
scheduling process which was already in place. The deliverable was to revise procedure 0.40.1. It
was not known what the controls would be. Note: 0.40.1 procedure appeared to have well defined
controls when reviewed.
Action 2. Appropriate procedures will have to be revised to define single point accountability of work
packages - appropriate is not defined.
Action 3. Ensure roles and responsibilities of various individuals/group are defined was too general.
The deliverable was to generate a weekly report.
Action 4. Review work week director roles and responsibilities and revise appropriate procedures was
too general.
Action 5. Develop prioritization and decision making tool. Revise appropriate procedures was too
general.
Action 6. Develop modify performance indicators thresholds, data gathering, frequency, actions was
too general.
6. Deliverable identified for Action Plan steps are well defined and The deliverable statements for Steps 1, 2, 4, 5, 6, 7, and 10 used the same terminology as the action
consistent with the action being taken. (Are words such as steps, which was vague. Step 3 was to ensure roles and responsibilities of various individuals/group
effective and reduced to minimal defined?) were defined, however the deliverable was to generate a weekly report.
7. Do the performance indicators provide an adequate measure of The performance indicators were not completely developed.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169913.
-30-
ACTION PLAN NUMBER - 5.2.5.2
ACTION PLAN TITLE - Completeness/Accuracy/Timeliness
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes. However WBS 2.2.4 was not referenced anywhere, but has issues associated with this action
plan.
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, No. Issues identified in the extent of condition review were not tied to the action plan. First,
and extent of condition described in WBS package. WBS 2.1.2 states that work packages need to include assessments of potential unknowns, alternative
plans, options, lessons learned, and pre-walkdown requirements. The action plan only has actions to
address pre-walkdown requirements by establishing clear standards for when the planner needs to
perform a walkdown jointly with the craft and/or engineer. Upon further review and discussion with the
Action Plan Owner, it was determined that Action Plan 5.2.5.1 had steps that addressed these
remaining issues. However, there was no clear tie between the action plan causal factors and the
steps that addressed the issues. Following discussion with the licensee, they stated that this problem
would be evaluated for TIP Revision 2.
And second, WBS 2.2.4 has numerous issues associated with the lack of engineering support during
the work package planning process. However, there are no actions in the plan to correct this
weakness. Upon further review of the action plans in this focus area and interviews with the Action
Plan Owner, the team determined that this issue was addressed by step 4 in Action Plan 5.2.5.1.
Action 4 addresses the definition of the roles and responsibilities of various individuals/groups involved
in the development of work packages.
The plan indicated that Steps 4 and 6 addressed Causal Factor 2. However, these steps did not
address Causal Factor 2. Upon further review and discussion with the Action Plan Owner, it was
determined that Action Plan 5.2.5.1 had steps that addressed this issue. However, there was no clear
tie between Causal Factor 2 and the steps in Action Plan 5.2.5.1 that address this problem.
5. Action Plan steps provide sufficient detail to define action to be No. Seven out of the nine actions specified are very general. It is difficult to assess whether the
taken. actions correct the problems identified in the causal factors. Steps 1-7 lacked detail.
6. Deliverable identified for Action Plan steps are well defined and Yes
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
-31-
7. Do the performance indicators provide an adequate measure of Performance indicators had not yet been developed.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169916.
-32-
ACTION PLAN NUMBER - 5.2.6.1
ACTION PLAN TITLE - Work Practices
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes
2. Characterization of issues in WBS are adequate to address The causal factors do not clearly address the WBS issues. The causal factors states that
individual issues from database included in the extent of condition management oversight has not been effective in addressing improper work practices. The WBS
review for that WBS. discussed the 1994 CNS Diagnostic Self-Assessment identification of what improper (poor) work
practices were defined to be. They were identified as industrial safety issues, inappropriate
implementation of procedures, improper or unsuccessful repairs to equipment, low housekeeping
standards, and an unacceptable level of human performance errors. Because the causal factors did
not break down what improper work practices were defined to be, no actions are identified in the action
plan to correct the specific problem areas of improper work practices (defined in the 1994 CNS
Diagnostic Self-Assessment). Two specific problems that do not have actions are improper or
unsuccessful repairs to equipment and human performance errors. The other problems that are listed
will be addressed either directly or indirectly by an action to review the maintenance principles and
standards. Upon further review of TIP it was determined by the team that Action Plan 5.2.5.2,
Completeness/Accuracy/Timeliness, and Action Plan 5.1.4.1, Human Performance, had actions that
addressed these performance issues. However, the Work Practice action plan did not provide any
cross reference to the other action plans.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, The steps do not address two of the problems that were defined to be examples of poor work
and extent of condition described in WBS package. practices. The two specific problems that do not have actions are improper or unsuccessful repairs to
equipment and human performance errors.
5. Action Plan steps provide sufficient detail to define action to be Steps 1 and 8 lack detail.
taken.
6. Deliverable identified for Action Plan steps are well defined and No, the deliverables restate what the actions are going to be, for example revise procedures as
consistent with the action being taken. (Are words such as necessary. Steps 1, 2, and 7 are not well defined.
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Yes.
performance as it relates to the objectives for improved
performance in the Action Plan?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169918
-33-
ACTION PLAN NUMBER - 5.2.6.2
ACTION PLAN TITLE - First Line Supervision
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Steps 4, 6, and 7 lacked sufficient detail.
taken.
6. Deliverable identified for Action Plan steps are well defined and Yes
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of No. Performance indicators had not yet been developed.
performance as it relates to the objectives for improved
performance in the Action Plan?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169867
-34-
ACTION PLAN NUMBER - 5.2.6.3
ACTION PLAN TITLE - Technical Support/Lessons Learned
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are The problem statement only discusses technical support issues which is inconsistent with the action
appropriately aligned. plan title (Technical Support/Lessons Learned).
4. Action Plan steps address the Problem Statement, Causal Factors, No. Causal Factor 1 discusses the problem of consistently capturing Lessons Learned to improve
and extent of condition described in WBS package. future performance. There are numerous methods that the licensee currently has and can implement
to capture lessons learned such as the work week critique, CAP, NRC inspection reports, post job
critiques, etc. The action plan only considers one aspect of capturing lessons learned by improving
the feedback form used in post job critiques. Other sources of lessons learned were not addressed.
Causal Factor 2 lists actions to establish engineering expectations. No actions address
communicating the standards and expectations, which is a historical weakness as described in this
causal factor, extent of conditions, and assessment issues database.
5. Action Plan steps provide sufficient detail to define action to be No. Four out of eight actions lacked the details necessary to assess the ability of the actions to correct
taken. the problems specified in the causal factors.
6. Deliverable identified for Action Plan steps are well defined and No. Deliverables are too general to determine end product.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of No. PIs were not fully developed.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169870
-35-
ACTION PLAN NUMBER - 5.2.7.1
ACTION PLAN TITLE - Improve Reinforcement of CAP Standards and Expectations
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan referenced WBS 3.1.1, 3.1.3, and 3.1.4.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern. The action plan owner
individual issues from database included in the extent of condition was actively involved during the development of the WBS extent of condition and the action plan.
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan Objective, Problem Statement, and Causal Factors were appropriately aligned.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Action items did not address a causal factor involving ineffective trending of performance issues. The
and extent of condition described in WBS package. action plan addressed improvement of corrective action program database trending but did not
address improved trending of equipment reliability issues identified in the work item database.
5. Action Plan steps provide sufficient detail to define action to be The action plan owner had already developed and implemented some of the action items. Therefore
taken. sufficient detail to assess the item was provided in the supporting documentation. For example, step 2
involved a revision to the corrective action review board (CARB) charter. The details of the revision
were not included in the action plan; however, the procedure had been revised and the charter had
been modified.
6. Deliverable identified for Action Plan steps are well defined and Deliverables do not state the desired outcome. For example, step 2 specified that the deliverable was
consistent with the action being taken. (Are words such as a revised CARB charter. In actuality, the deliverable is improved CARB oversight of the corrective
effective and reduced to minimal defined?) action program processes.
7. Do the performance indicators provide an adequate measure of The root cause quality index performance indicator monitored the as-left, performance analysis
performance as it relates to the objectives for improved department corrected version of the root cause analysis. The team determined that unsatisfactory root
performance in the Action Plan? Are the thresholds in the cause analyses continued to be submitted. However, the performance indicator for root cause
performance indicators appropriate? analysis quality specified positive results.
The significant condition report recurrence indicator did not include items which repeated on multiple
occasions. For example, there were three recurrences of a significant condition report involving the
corrective action program in the previous 36 months. However, the indicator only reflected one of the
recurrences because the same issue was repeated.
The corrective action program indicators reported the 12-month rolling average value. This method of
data reporting normalizes the data and may disguise adverse/positive results for a given month.
Consequently, corrective actions may not be taken to investigate a negative result.
8. Has a notification been written to enter Action Plan in CAP? Notifications 10169881 and 10169811.
-36-
-37-
ACTION PLAN NUMBER - 5.2.7.2
ACTION PLAN TITLE - Root Cause
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The action plan did not reference WBS 3.1.3 and 3.1.4. The action plan owner had reviewed the
packages as part of the action plan development.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern. The action plan owner
individual issues from database included in the extent of condition was actively involved during the development of the WBS extent of condition and the action plan.
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Action Plan Objective, Problem Statement, and Causal Factors were appropriately aligned.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, The Action Plan steps address the Problem Statement, Causal Factors, and extent of condition
and extent of condition described in WBS package. described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be The action plan owner had already developed and implemented some of the action items. Therefore
taken. sufficient detail to assess the item was provided in the supporting documentation. For example, there
were no specific items to address improvements in apparent cause analysis. However, the revised
corrective action program training included a discussion on apparent cause analysis.
6. Deliverable identified for Action Plan steps are well defined and Deliverables do not state the desired outcome. For example, the deliverable for reducing root cause
consistent with the action being taken. (Are words such as investigators from 200 to 50 was to establish and maintain a list of 50 root cause evaluators. The
effective and reduced to minimal defined?) actual desired outcome was to improve the quality of root cause analysts by limiting the number of
certified individuals and increasing the frequency of the task.
-38-
7. Do the performance indicators provide an adequate measure of The root cause quality index performance indicator monitored the as-left, performance analysis
performance as it relates to the objectives for improved department corrected version of the root cause analysis. The team determined that unsatisfactory root
performance in the Action Plan? Are the thresholds in the cause analyses continued to be submitted. However, the performance indicator for root cause
performance indicators appropriate? analysis quality specified positive results.
The significant condition report recurrence indicator did not include items which repeated on multiple
occasions. For example, there were three recurrences of a significant condition report involving the
corrective action program in the previous 36 months. However, the indicator only reflected one of the
recurrences because the same issue was repeated.
The corrective action program indicators reported the 12 month rolling average value. This method of
data reporting normalizes the data and may disguise adverse/positive results for a given month.
Consequently, corrective actions may not taken to investigate a negative result.
Item 1, which reduced root cause investigators from 200 to 50, was statused as complete in January
2002. However, the quality of root cause analyses was not improved. In addition, the licensee had
not implemented interim corrective actions to monitor the quality of the newly selected analysts.
Interim actions were not implemented even though the licensee indicated that they expected the
quality of root cause analyses to decrease following the selection of the 50 analysts.
8. Has a notification been written to enter Action Plan in CAP? Notifications 10169881 and 10169811.
-39-
ACTION PLAN NUMBER - 5.2.7.3
ACTION PLAN TITLE - Improved Utilization of OER
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. The Action Plan referenced WBS 3.5.2.
2. Characterization of issues in WBS are adequate to address The WBS extent of condition review identified the significant areas of concern. The Action Plan owner
individual issues from database included in the extent of condition was not involved in the development of the WBS, but did review the WBS as part of his review and
review for that WBS. development of the Action Plan.
3. Action Plan Objective, Problem Statement, and Causal Factors are The Problem Statement discussed line supervision and workers not using operating experience, but
appropriately aligned. the causal factors focused only line management. The workers not using operating experience was
not addressed in the Causal Factors.
4. Action Plan steps address the Problem Statement, Causal Factors, Two of the conclusions in the extent of condition were not covered in the Action Plan. Specifically:
and extent of condition described in WBS package.
1. Not sharing in-house lessons learned with the industry
2. Weaknesses in the internal dissemination and feedback loop for industry OE, caused inconsistent
implementation.
The Action Plan owner stated that both of these items were appropriately addressed previously, but no
reviews were done to verify the effectiveness of the corrective actions. In addition, for Example 2, the
team did an independent extent of condition and determined that three NRC issued generic letters
and/or information notices were not properly addressed. The licensee did not conduct an independent
assessment of previously addressed operating experience information to ensure actions taken
addressed those concerns.
5. Action Plan steps provide sufficient detail to define action to be The Action Plan Steps were not sufficiently detailed. Steps 2 and 3 discussed doing observations of
taken. prejob briefs, work planning, and job performance, but the steps did not indicate how many
observations per department would meet the necessary observations. Step 5 discussed
benchmarking. Again, no details as to which plants meet the criteria for good OER programs; the
criteria to be used to evaluate the plants selected; and, the step did not indicate how many plants need
to be benchmarked.
6. Deliverable identified for Action Plan steps are well defined and The deliverables do not measure changes in performance as based on the actions taken. For
consistent with the action being taken. (Are words such as instance, Step 5, benchmarking, specified the deliverable as a completed trip report. This will not do
effective and reduced to minimal defined?) anything to change performance. Changes to the operating experience program/process as a result of
the evaluation of the benchmarking activity should be the deliverable.
-40-
7. Do the performance indicators provide an adequate measure of There was no current performance indicator for this program. The licensee was working on developing
performance as it relates to the objectives for improved an indicator, Number of observations of OER use in the field performed per quarter. The team was
performance in the Action Plan? Are the thresholds in the not able to determine how this indicator might indicate improved use of operating experience.
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169892
-41-
ACTION PLAN NUMBER - 5.2.8.1
ACTION PLAN TITLE - Vendor Manual Upgrade Program
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. No
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be No. Steps were general in nature and lacked sufficient detail.
taken.
6. Deliverable identified for Action Plan steps are well defined and No. All of the deliverables are re-statements of the action plan steps and do not discuss the specifics
consistent with the action being taken. (Are words such as or quality of the completed product.
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Performance indicators had not yet been developed
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169911
-42-
ACTION PLAN NUMBER - 5.2.8.3
ACTION PLAN TITLE - Procedure Change Process
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. No
2. Characterization of issues in WBS are adequate to address Yes
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, No. The WBS identified the procedure change process to be cumbersome and non-responsive in
and extent of condition described in WBS package. addressing needed changes or enhancements. The action plan only addresses the implementation of
an Independent Qualified Reviewer/Approver process, which will reduce the procedure review approval
burden of SORC. The inspectors determined that there was not a single process for revising
procedures, but that each department used their own informal method, The inspectors identified that
pending procedure revisions were not tracked as to when the request was received, whether the
change warranted a high priority, what procedure was involved, or when the change would be
incorporated. Additionally, significance determinations were not found for many of the pending
changes.
5. Action Plan steps provide sufficient detail to define action to be Step 9 requires follow-on assessments. However, there is no detail as to what the assessment
taken. objectives will be, who will perform the assessment, which departments would be involved in the
assessment, etc.
6. Deliverable identified for Action Plan steps are well defined and Yes
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of No, the PIs only address the performance of the procedure change process after a change is
performance as it relates to the objectives for improved delivered to the site procedures group. It does not address the period of time from submittal of the
performance in the Action Plan? Are the thresholds in the change request until final approval.
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification10169904
-43-
ACTION PLAN NUMBER - 5.3.1.1
ACTION PLAN TITLE - System/Equipment Performance
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address No. The extent of condition discusses some of the current long-term equipment reliability issues. A
individual issues from database included in the extent of condition more complete list is available in the 2002 CNS Unplanned LCO Action Statement Entry and
review for that WBS. Equipment Reliability Review. The extent of condition does not develop the specific longstanding
equipment reliability issues facing the station, but rather focuses on the cultural aspects that caused
the station to fail to correct them over prolonged periods. The TIP does not include actions to actually
correct the long list of specific equipment reliability issues that have caused frequent diversion of
significant attention and resources away from planned work to correct emergent work when problems
recur.
3. Action Plan Objective, Problem Statement, and Causal Factors are No. The objective to complete ongoing activities in support of achieving an improved level of
appropriately aligned. equipment reliability does not specify which activities are encompassed. The problem statement
focused on failure to proactively address equipment issues, and did not address correcting existing
issues. The causal factor statements were more complete, but still did not address fixing existing
equipment reliability issues or obsolescence issues.
4. Action Plan steps address the Problem Statement, Causal Factors, The extent of condition mentions, but does not develop, a list of longstanding equipment reliability
and extent of condition described in WBS package. issues. The action plan addresses improving the organizational infrastructure to improve how the
station handles future equipment reliability issues and to identify them before they become larger
problems. However, the TIP does not address actually correcting the long list of known equipment
reliability issues. These equipment issues must be corrected if the station is to avoid challenging itself
and diverting resources when problems recur.
5. Action Plan steps provide sufficient detail to define action to be Yes.
taken.
-44-
6. Deliverable identified for Action Plan steps are well defined and Action 1.1 to review the PM change process for improvement did not specify the form of the product.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?) Action 1.3 to minimize the preventive maintenance closeout backlog was unclear on what level of
backlog was acceptable.
Action 1.4 to develop a PM feedback process was not specific as to what form the process would be
(i.e. a station procedure?).
Action 3.2 specified performing a gap analysis between station practices and industry standards. The
deliverable statement did not specify how the identified gaps would be captured (i.e. in the corrective
actions process, a self-assessment document, etc.).
The same was true for Action 4.1, which specified identifying obsolescence issues.
7. Do the performance indicators provide an adequate measure of The PIs specified provide a good measure of the impacts to the site due to equipment reliability
performance as it relates to the objectives for improved problems. They do not, however, provide clear information about whether actions to address issues
performance in the Action Plan? Are the thresholds in the were effective.
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169914.
-45-
ACTION PLAN NUMBER - 5.3.2.1
ACTION PLAN TITLE - Programs
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address No. The WBS did not discuss each program in any level of detail. The limited extent of condition
individual issues from database included in the extent of condition discussed issues thought to be common to all programs. The action plan caused self-assessments
review for that WBS. and interface assessments to be performed in order to actually develop the full extent of condition.
3. Action Plan Objective, Problem Statement, and Causal Factors are No. The objectives did not fully address all the issues identified in the problem statement nor all of the
appropriately aligned. casual factors listed. The objective did not actually list any performance improvements or define a
level of improvement to be achieved. With the exception of upgrading Procedure 0-CNS-12, the
objectives only implemented program improvement activities which were in progress at the time the
action plan was developed.
4. Action Plan steps address the Problem Statement, Causal Factors, No. The action plan steps were written to align with the plan objectives. Twelve of the fourteen steps
and extent of condition described in WBS package. were intended to implement program improvement activities which were already in progress at the time
the action plan was developed. The action plan steps did not address the problem statement issues of
lack of commitment to program implementation and lack of organizational depth.
Steps 3 through 6 were intended to implement the previously existing Program Improvement
Project (PIP) which addressed programs classified Category A and B per procedure 0-CNS-12.
However, the scope of the PIP and the action plan did not include the Probabilistic Risk Assessment
Program which was a Category B program per 0-CNS-12.
The action plan steps did not fully address Casual Factor 1 concerning ownership of programs. The
roles and responsibilities for program implementation are documented in program notebooks. The
program notebooks are informal documents developed by the program owners. The notebooks are
not available to the whole station, do not require concurrence of the other organizations involved and
do not require compliance as would formal procedures.
5. Action Plan steps provide sufficient detail to define action to be No. Steps 3 through 6 required a series of self-assessments and interface assessments to be
taken. performed for the individual Category A and B programs. The action plan was not sufficiently detailed
to define the scope of the assessments. Many of the program owners did not have significant industry
involvement outside of CNS, which would assist in establishing a performance level reference. None
of the program owners planned external benchmarking to identify the industry standards and practices.
6. Deliverable identified for Action Plan steps are well defined and No. Steps 3 through 6 required a series of self-assessments and interface assessments to be
consistent with the action being taken. (Are words such as performed. The deliverables did not develop formal action plans, define goals for improved
effective and reduced to minimal defined?) performance or establish effectiveness measurements to improve individual programs. The PIP
Project Charter included the concept of preparing Programs Improvement Action Plans which were
Required to provide clear documentation and prioritization of actions to be taken to resolve
assessments deficiencies. This concept was not implemented by the PIP or incorporated into the TIP
action plan.
-46-
7. Do the performance indicators provide an adequate measure of No. The performance indicators did not provide an effective or timely means of monitoring program
performance as it relates to the objectives for improved health or changes in performance for individual programs. The CNS Program Cumulative Health
performance in the Action Plan? Are the thresholds in the Indicator listed was not adequate to trend the performance of the individual programs being addressed
performance indicators appropriate? because the health of each program was only required to be assessed once every two years. This
indicator was not sufficiently sensitive because it numerically averaged 7 attributes in each of the 16
programs into one overall rating. The other indicators listed only monitor the schedule and costs for
implementing breaker refurbishment programs and performing MOV calculations. The indicators listed
would not allow assessment of the effectiveness of the action plan steps or measure actual
performance improvements in individual programs.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169884.
-47-
ACTION PLAN NUMBER - 5.3.3.1, 5.3.3.2, and 5.3.3.3
ACTION PLAN TITLE - Design Basis Information/Licensing Basis Information Translation Project, Offsite Power
Reliability Improvement - Phase 1, and Unauthorized Modification Followup Project
FOCUS AREA - Key Modifications and projects; Configuration
REVIEW CRITERIA OBSERVATION
1. Action Plans reference WBS inputs. No. Action plans 5.3.3.2 and 5.3.3.3 contained no reference to a WBS, although both were intended to
reference WBS 3.4.1.
2. Characterization of issues in WBS are adequate to address No. The extent of condition for issues associated with design basis/license basis information and
individual issues from database included in the extent of condition offsite power reliability were not developed in the WBS. The causal factors developed were also
review for that WBS. incomplete. The team could not assess the completeness of the action plan steps without an accurate
characterization of the extent of condition. In the case of offsite power reliability, the action plan was
for Phase 1 of the project, with additional phases undeveloped. It was unclear if any additional phases
would be appropriate for inclusion in the TIP. In the case of design/license basis problems, the lack of
development of the issue in the extent of condition review caused development of incomplete actions,
discussed below.
The WBS also did not document the extent of condition or causal factors associated with unauthorized
modifications issues. However, this project has been ongoing since 1996 and is well-documented on
the docket.
3. Action Plan Objective, Problem Statement, and Causal Factors are No. The combination of the three action plan problem statements and causal factors did not address
appropriately aligned. the full scope of the problems identified in the extent of condition review. Specifically, problems
relating to the quality of calculations, evaluations, analyses, and plant changes were not addressed by
action plans.
The problem statement and causal factor listed in the action plan for Offsite Power Reliability
Improvement did not have clear ties to issues, findings, or the discussion of extent of condition in the
WBS. The team noted numerous examples in NRC inspection findings which should have been
identified during the extent of condition review. This issue was appropriately included in the TIP, but
the team was unable to determine the completeness of actions because the licensee did not develop
the extent of condition - it was listed as Phase 1 without future phases being scoped. Also, this action
plan appropriately addressed industry experience and recommendations, but this aspect was not
addressed in the problem statement or causal factors.
Action plan 5.3.3.3 did not develop the problems, causes, or objectives associated with unauthorized
modifications. The WBS also did not document the extent of condition or causal factors for this issue.
The stated objective of the action plan was to track completion of actions in the existing project.
-48-
4. Action Plan steps address the Problem Statement, Causal Factors, No. The characterization section of WBS 3.4.1, Key Modifications and Projects Configuration,
and extent of condition described in WBS package. identified that process, programmatic or organizational deficiencies contributing to discrepancies
between design/license basis information and actual configuration of the plant needed to be
addressed. However, this issue was not addressed by an action plan. The licensee inappropriately
decided not to include this issue in the TIP because these issues had recently been addressed by
implementing a significant change to the configuration control process procedures by revising the
Engineering Procedure 3.4 series documents. WBS 3.4.1, Key Modifications and Projects
Configuration, stated that an effectiveness review needed to be completed following RE21 to identify
and correct any problems experienced. However, no action plan addressed these actions.
Action plan steps to address improving engineering training was not fully developed, so the team was
unable to assess it for completeness.
5. Action Plan steps provide sufficient detail to define action to be Yes. However, the Project Plan for the Design/License Basis Translation Project stated that validation
taken. efforts were intended to provide reasonable assurance that design basis information was consistently
reflected in the physical plant and the controlled documents used to support plant operations. The
licensees plan did not include actions to validate the physical plant.
6. Deliverable identified for Action Plan steps are well defined and Yes.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of No. Action Plans 5.3.3.1 and 5.3.3.3 listed only TIP schedule adherence as performance indicators.
performance as it relates to the objectives for improved These were not expected to provide a valid indication of the effectiveness to improvement efforts in
performance in the Action Plan? Are the thresholds in the this focus area. As noted above, the focus area was much more broad than the three action plans
performance indicators appropriate? addressed. The performance indicators for the offsite power reliability action plan appeared to
adequately measure performance in the area of concern. The combined PIs of the three action plans
in this focus area did not address the full scope of the focus area, as described above.
8. Has a notification been written to enter Action Plan in CAP? Notifications 10169889, 10169908, and 10169887.
-49-
ACTION PLAN NUMBER - 5.4.1.1
ACTION PLAN TITLE - Management Ownership of Training
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address Yes.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are One objective refers to a decline in station human performance errors. This is not consistent with the
appropriately aligned. problem statement for this action plan, in that, the problem statement does not indicate that human
performance errors have resulted from inadequate training.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes.
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Yes.
taken.
6. Deliverable identified for Action Plan steps are well defined and For Step 2, the process to be used lacks detail. For Step 4, effective corrective action is not defined.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Not in all cases. The team identified an issue with one indicator which tracked the number of
performance as it relates to the objectives for improved instances where individuals were utilized to perform work tasks for which they were not qualified. The
performance in the Action Plan? Are the thresholds in the thresholds associated with this indicator were set such that one instance of an unqualified person
performance indicators appropriate? performing a work activity per quarter, or as many as four such instances per year, would indicate
acceptable performance. In theory, this rate of unqualified personnel performing work could continue
on indefinitely without the performance indicator triggering any corrective action. When the team
brought this to the attention of the licensee, they entered the issue into their corrective action program
and immediately revised the indicators acceptability thresholds. (This is the same comment as for
Action Plan 5.4.1.2)
8. Has a notification been written to enter Action Plan in CAP? Notification 10169893
-50-
ACTION PLAN NUMBER - 5.4.1.2
ACTION PLAN TITLE -Training Program -- Evaluation and Qualification of Personnel
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address Yes.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes, however, several of the causal factors are addressed in other action plans. For example, Causal
and extent of condition described in WBS package. Factor 3 is addressed in Action Plan 5.1.1.2. However, the steps in Action Plan 5.1.1.2 lack detail
regarding specific applicability to Action Plan 5.4.1.2. This is because Action Plan 5.1.1.2 addresses
the issue of Accountability site-wide and, therefore, should be applicable to the Training department.
5. Action Plan steps provide sufficient detail to define action to be Action Plan step 3 lacks detail regarding implementation.
taken.
6. Deliverable identified for Action Plan steps are well defined and Yes.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Not in all cases. The team identified an issue with one indicator which tracked the number of
performance as it relates to the objectives for improved instances where individuals were utilized to perform work tasks for which they were not qualified. The
performance in the Action Plan? Are the thresholds in the thresholds associated with this indicator were set such that one instance of an unqualified person
performance indicators appropriate? performing a work activity per quarter, or as many as four such instances per year, would indicate
acceptable performance. In theory, this rate of unqualified personnel performing work could continue
on indefinitely without the performance indicator triggering any corrective action. When the team
brought this to the attention of the licensee, they entered the issue into their corrective action program
and immediately revised the indicators acceptability thresholds.
8. Has a notification been written to enter Action Plan in CAP? Notification 10169912
-51-
ACTION PLAN NUMBER - 5.4.1.3
ACTION PLAN TITLE -Training Program -- Organizational Effectiveness
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address Yes.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes.
appropriately aligned.
4. Action Plan steps address the Problem Statement, Causal Factors, Yes.
and extent of condition described in WBS package.
5. Action Plan steps provide sufficient detail to define action to be Action Plan step 3 regarding the Training Program Effectiveness Scorecard lacks detail.
taken.
6. Deliverable identified for Action Plan steps are well defined and The Deliverable for step 4 lacked detail in that it only requires that training be attended, which in and of
consistent with the action being taken. (Are words such as itself is not a measure of the success of a training issue. The action plan step will provide training to
effective and reduced to minimal defined?) instructors for the purpose of improving exam development and quality of task analysis. However,
these details are not captured completely in the deliverable.
7. Do the performance indicators provide an adequate measure of Yes.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169906
-52-
ACTION PLAN NUMBER - 5.4.1.4
ACTION PLAN TITLE -Training Program -- Process Enhancements
REVIEW CRITERIA OBSERVATION
1. Action Plan references WBS inputs. Yes.
2. Characterization of issues in WBS are adequate to address Yes.
individual issues from database included in the extent of condition
review for that WBS.
3. Action Plan Objective, Problem Statement, and Causal Factors are Yes, however, all of the causal factors are addressed entirely by other action plans. This is because
appropriately aligned. the training programs have declined due to ownership, lack of self-assessments, etc. The objective of
this action plan addresses restoring the programs to a satisfactory condition.
4. Action Plan steps address the Problem Statement, Causal Factors, The Action Plan steps are programmatic upgrades only. The causes of the problems are addressed
and extent of condition described in WBS package. entirely in other action plans.
5. Action Plan steps provide sufficient detail to define action to be The steps lack detail regarding material development and revision for program upgrades.
taken.
6. Deliverable identified for Action Plan steps are well defined and Industry Guidelines is stated in almost every Deliverable. As it is used, the term is undefined.
consistent with the action being taken. (Are words such as
effective and reduced to minimal defined?)
7. Do the performance indicators provide an adequate measure of Yes.
performance as it relates to the objectives for improved
performance in the Action Plan? Are the thresholds in the
performance indicators appropriate?
8. Has a notification been written to enter Action Plan in CAP? Notification 10169906