ML052790421

USA Inc, Comments in Addition to Our Letter of August 31, 2005 Concerning NRCs Safety Culture Initiative

ML052790421
Person / Time
Site:	Fermi, Wolf Creek, Cooper, Susquehanna, Columbia, San Onofre, Cook, Fort Calhoun
Issue date:	09/29/2005
From:	Parry C Utilities Service Alliance
To:	Mary Johnson NRC/OE
References
Download: ML052790421 (26)

Text

Utilities Service Alliance, Inc.

A Not-for-Profit, Membership Corporation WX'orking Together For Mutual Success 9200 Indian Creek Pkuy.

Suite 201 Overland Park, AS 66210 913-451-5641 913-451-3893 (Fax)

September 29, 2005 wwum uainc.org Mr. Michael R. Johnson Director, Office of Enforcement Chair, Safety Culture Steering Committee U.S. Nuclear Regulatory Commission One White Flint North 11545 Rockville Pike Rockville, Maryland 20852

Dear Mr. Johnson:

The purpose of this correspondence is to provide comments, in addition to our letter of August 31, 2005, concerning the Nuclear Regulatory Commissions's (NRC's) "Safety Culture Initiative."

Utilities Service Alliance, Inc. (USA) is a non-stock, not-for-profit cooperative of nuclear utilities and is incorporated in the state of Kansas.

Its full members include American Electric Power (DC Cook), DTE Energy (Fermi 2), Energy Northwest (Columbia), Nebraska Public Power District (Cooper), Omaha Public Power District (Fort Calhoun), Ontario Power Generation (Darlington & Pickering), PPL-Susquehanna (Susquehanna),

Southern California Edison (San Onofre) and Wolf Creek Nuclear Operating Corp (Wolf Creek).

The importance of nuclear safety has been recognized since the beginning of nuclear power in this country. This concept has been factored into the initial design, construction and in operation of all our plants. Each USA member views establishing and maintaining a strong safety culture at their plant as their full responsibility. Consequently, our members have demonstrated a strong and continuing commitment to the safe operation of their plants and in the development, assessment, and continuous improvement of their safety culture.

The USA also concurs with the NRC that a strong safety culture is an important part of the NRC's obligation to ensure safe operation of nuclear power plants and to protect the Public Health and Safety.

G Mr. Michael R. Johnson September 29, 2005 Director, Office of Enforcement Chair, Safety Culture Steering Committee The USA members' experience in assessing safety culture.

Recent events in our industry has prompted us to develop and perform an independent safety cultural assessment of our facilities. To perform this assessment, the USA has developed, and used, the "USA Nuclear Safety Culture Assessment" process. The USA assessment process is based on elements from Institute of Nuclear Power Operations (INPO) documents regarding conservative and operational decision-making, lessons learned from events at Davis-Besse and NEI 97-05's basic Safety Conscious Work Environment survey tool.' The assessments were designed to be a very critical comparison of each plant to approximately 90 behavioral characteristics associated with a conservative safety culture model. The purpose of the assessments was to determine to what degree each member's plant has a healthy respect for nuclear safety and that nuclear safety is not compromised by production priorities.

The assessments at each USA plant identified "Strengths" and "Areas For Improvement" and a formal assessment report was prepared in accordance with the self-assessment process for that plant. As applicable, "Areas For Improvement" were entered into each plant's corrective action process. The USA's general approach was to perform an assessment at each member plant utilizing a team of experienced leaders from outside the host plant. Most of the team members participated in 3 or 4 of the assessments to ensure continuity of the process. USA members regard their process as an effective way to assess certain aspects of a plant's safety culture. See attachment 1 for an outline of the USA process.

The USA found their process to be an effective, structured, and systematic way to assess each plant's unique safety culture. At the same time, it has highlighted the difficulty of objectively assessing a matter as inherently subjective and subtle as a safety culture. The USA process includes a quantitative measure (score) of a safety culture and many of its important aspects and elements. However, in our experience, the quantitative measures, while useful, have not provided particularly insightful, or revealing information, since the scores were in a relatively narrow range. In contrast, the consistently and systematically identified and analyzed, but still subjective, insights and observations, such as each plant's "Strengths" and "Areas for Improvement," were of more value.

2

I Mr. Michael R. Johnson September 29, 2005 Director, Office of Enforcement Chair, Safety Culture Steering Committee The USA's Comments on the NRC's Safety Culture Attributes Table.

In attachment 2 the USA provides specific comments on many of the elements of the NRC's Safety Culture Attributes Table. In general, the USA is concerned that use of many of the sources of "Potential Safety Culture Inspection Information" and the "Potential Safety Culture Measures (SCMs)" to assess a safety culture would be inherently flawed.2 The USA believes these inherent flaws to be:

Intrinsic ambiguity in the interpretation of the measures. An increase or a decrease in the value of or the trend of the value of several of the SCMs can be interpreted as an improvement or decline in certain aspects of a safety culture. While the underlying parameter may be numeric and objective, interpretation of the results must be substantially subjective and hence inconsistent with the ROP principles.

Use of measurements relying on small numbers or ratios of small numbers. For example: The median number of NRC allegations from on-site sources has varied from 3 to 4, per site, per year for the last few years. Measurements based on such small numbers and sub-categories within these small numbers are likely to vary significantly. Use of the ratios of these numbers may amplify even small changes. Again, while the underlying parameter may be numeric and objective, interpretation of the results must be substantially subjective.

Variation between plants. The same measures or trends in measures might well indicate different things at difference plants.

The variation in plants' culture might assign strikingly different meanings to the same data.

Use of measures with differing time-frames. Some of the proposed measures are available monthly (NRC Allegations) while others vary in frequency as a result of factors outside the safety culture such as the timing of license examinations and thus the pass/fail fraction. Further, the timing, the planned, and the actual durations of refueling outages regularly affect many aspects of the Corrective Action Program. Bench-marking and self-assessments, and thus the availability of results for analysis, are also affected by outage schedules.

3

Mr. Michael R. Johnson September 29. 2005 Director, Office of Enforcement Chair, Safety Culture Steering Committee Thus, while some of the measures in the Safety Culture Attributes Table may contain information on aspects of the safety culture, the time reference of the different measures varies considerably and is strongly influenced by factors unrelated to the safety culture. This increases the difficulty of interpreting the results. The USA notes that most other methods of assessment, including its own, are typically performed over a short period and provide a body of information collected and evaluated at one point in time.

Inadequate scope. Many of the SCEs and SCIs describe a scope of activity and influence far beyond the SCMs identified.

Reliance on non-regulated activities. Several of the SCMs involve activities beyond the scope of any specific NRC regulation.

What should the NRC do?

Recognizing both the value and limitations of its process, the USA has serious reservations that the use of measures of safety culture, such as those in the table, would comply within the predictable, objective, understandable and risk-based regulatory principles of the Reactor Oversight Process.

Instead, the USA recommends:

1. The NRC establish a policy statement, similar to its 1996 policy on an SCWE, requiring each plant to establish and maintain a strong and effective safety culture.

2. The NRC's policy statement should encourage plants to assess their safety culture on a periodic basis. The assessment should include monitoring and trending of symptoms of a declining safety culture. The USA views its assessment process, with additional development, as one of several ways that might be used to meet the expectations of such a policy. Each plant should be free to adopt a flexible assessment process consistent with its own unique safety culture and work environment.

3. The NRC should have the responsibility to review a plant's safety culture assessment process and results, when the NRC identifies concerns with a plant's safety culture. If necessary, the NRC could perform independent oversight of the assessment process.

4

Mr. Michael R. Johnson September 29, 2005 Director, Office of Enforcement Chair, Safety Culture Steering Committee

4. That a plant's safety culture assessment process examine the culture by both subjective and objective means.

5. That the characteristics of the safety culture policy and the assessment processes be based on existing international regulatory and industry guidance.3 Developing guidance for NRC oversight of safety culture is difficult and should not be rushed.

The USA agrees with IAEA-TECDOC-1 329, "Safety culture is a complex concept and there is no simple indicator that measures its state. The multilevel nature of culture, and the tacit nature of some of the levels (basic assumptions), increases the difficulty of measurement."

This document further points out, "To assess the safety culture of an organization is not easy. We must remember that the safety culture will be influenced by the organizational culture, and possibly other cultures.

Culture is a complex concept involving multiple levels with some levels such as basic cultural assumptions being tacit and out of awareness. There are no commercially available safety culture questionnaires that will satisfactorily assess the safety culture of an organization. This should not be surprising as each organization is unique in terms of its history, its organizational culture and its employees.... "

Despite its experience, the USA does not believe that it has fully developed a process to quantitatively assess all aspects of safety culture. The USA is currently expanding its assessment process beyond its Davis-Besse focus to include INPO's, Principles for a Strong Nuclear Safety Culture. When the USA assessment process has been expanded, the USA anticipates using the process at many of its member plants in 2006 and 2007.

However, the NRC's proposed time-line for its Safety Culture Initiative is compressed to the point that may be counter productive in developing a comprehensive, effective, and sustainable process. A flawed process developed in haste might allow a declining safety culture, as was found at Davis-Besse, to cause an event and still not be detected.

The USA would be pleased to participate as a stakeholder in a careful, deliberate, thoughtful, and comprehensive process to develop an NRC policy on safety culture.

5

Mr. Michael R. Johnson September 29, 2005 Director, Office of Enforcement Chair, Safety Culture Steering Committee The USA member utilities appreciate this opportunity to comment and offer assistance on this very important issue. If you have questions or would like to discuss this further, please contact the Chairman of our Board of Directors, Mr. William T. O'Connor, Jr., Vice President Nuclear Generation for DTE Energy at 734 586-4445, (oconnor@dteenergy.com) or the undersigned.

Respectfully, Carl E. Parry President & Chief Executive Officer Utilities Service Alliance, Inc.

Attachments:

1. Outline of the USA Safety Culture Assessment Process

2. Specific comments on the NRC's Safety Culture Attributes Table cc: 1. Schoenfeld, Sr. Assistant to the Director, Office of Enforcement J. Perseasky, Office of Nuclear Regulatory Research; Member, Safety Culture Working Group J. Jacobson, Safety Culture Attributes Table Office of Nuclear Reactor Regulation; Member, Safety Culture Working Group NRC Document Control Desk Marvin S. Fertel Nuclear Energy Institute

'NEI 97-05 'Nuclear Power Plant Personnel-Employee Concerns Program-Process Tools in a Safety Conscious Work Environment."

2For ease of discussion each 'Safety Culture Attribute," 'Safety Culture Element."

'Potential Safety Culture Inspection Information," and 'Potential Safety Culture Measure' had been identified as a numbered 'SCA," "SCE," SCi," and 'SCM," respectively.

3Such as, IAEA-TECDOC-1 329 'Safety Culture in Nuclear Installations," December 2002, INPO 'Principles for a Strong Nuclear Safety Culture," November 2004, and NEI 97-05

'Nuclear Power Plant Personnel-Employee Concems Program-Process Tools in a Safety Conscious Work Environment."

6

Plan and General Description of the Page 1 USAs Safety Culture Assessment Process Plan and General Description of the USA's Safety Culture Assessment Process The Utilities Service Alliance (USA) will sponsor the SOER 02-4 required Self-Assessment (SA) for member utilities. This self-assessment would be completed as a round robin assessment, using a common assessment plan.

Purpose:

The purpose of the assessment is to determine to what degree station personnel have a healthy respect for nuclear safety and that nuclear safety is not compromised by production priorities. The assessment will emphasize leadership skills and approaches necessary to achieve and maintain the proper focus on nuclear safety.

Assessment Objectives:

The objectives of the assessment are to evaluate the health of the station's safety culture and provide recommendations to station management to improve or sustain this health.

The following elements will be specifically assessed:

6 Employees are encouraged to identify degraded conditions and demonstrate a willingness to escalate their concerns when the conditions are not corrected.

6 Station personnel exhibit a questioning attitude, pursue resolution of important and long-standing equipment and material problems, and execute plant shutdowns, if appropriate, to effect repairs.

6 Management is involved in important plant activities, especially those having the potential to affect nuclear safety, and exercises accountability and follow-up as appropriate.

6 Events determined to be significant by the station are recognized and aggressively addressed to determine their root causes and the corrective actions necessary to prevent recurrence.

6 Management emphasizes safety as the highest priority and exhibits conservative, safety-conscious, and defense-in-depth decision making.

Plan and General Description of the Page 2 USA's Safety Culture Assessment Process 4

Managers seek critical feedback from both internal and external sources, and first-hand information is actively sought from those personnel intimately involved with the issues.

4 Management fosters a safety-conscious work environment - one in which plant staff feel they can (and do) raise concerns without fear of reprisal.

4 Training is provided for all staff, at all levels, to ensure that each employee understands his or her responsibilities for ensuring safe operations.

Assessment Team Composition:

The "self-assessment" will be completed as a Utilities Service Alliance (USA) round robin assessment. A core team will be established through this USA initiative with team members from each USA member site. The assessment team will consist of a team host from the assessed site and several off-site independent team members from USA member sites. At least one off-site USA member site senior executive will participate on the assessment team. Core team members selected will have sufficient diversity to ensure adequate coverage of all assessment objectives. INPO will be notified of the assessment schedule and invited to participate in or observe the assessment.

Assessment Schedule:

USA member plants (DC Cook, Wolf Creek, Fort Calhoun, Cooper, Fermi, Susquehanna, and Columbia Generating Station) will complete SOER 02-4, Recommendation 2, Self-Assessment by July 2003.

Assessment Implementation:

The Assessment will combine data review, interviews, and behavioral observations to assess station management behaviors against key principles outlined in INPO Document, "Principles for Effective Operational Decision-Making" and related documents. The assessment will be conducted in the following three phases.

4 Phase 1 Pre-screening Data Collection and Review 4

Phase 2 Assessment Implementation 4

Phase 3 Final Report Preparations with Recommendations Each phase is explained in further detail below.

Phase 1-Pre-screening Data Collection and Review Pre-screening material will be requested by the Assessment Team Lead a few weeks prior to the assessment week. The type of pre-screening material requested will provide insight into the station's decision-making process and behaviors. The material potentially correlates to material involved in or related to missed opportunities that occurred at Davis-Besse. The pre-screening material should include the past year except where specifically noted. The

Plan and General Description of the Page 3 USA's Safety Culture Assessment Process Attachment I Team Lead will review the pre-screening material and provide a summary report to the assessment team prior to the teams' arrival on-site.

Team members will review the summary report prior to the assessment week and as required may request further details on the pre-screening material during the assessment week document reviews and interviews.

Pre-screening material may include but is not limited to the following:

6 Justifications for Continued Operation 6

Most recent Safety Culture Survey Results and actions (assigned And taken) 6 Corporate Safety Review Board Minutes 6

Root Cause Analyses for Significant Conditions Adverse to Quality (SCAQ) 6 SCAQ CAP extensions 6

A random sampling of lower level CAP reports 6

Adverse trends as identified in the CAP 6

O&M and Capital Cost Reductions 6

Resource Reductions 6

NRC Reports (LERs, Enforcement Discretion, etc.)

6 NRC Violations 6

INPO Evaluation and Site Assist Visit Reports 6

CAP backlog 6

Deferred outage work 6

Capital Projects Funding list (status, priorities, issues) 6 QA Audits and Assessments 6

Management Policy on Safety 6

HP Clock Reset Summary 6

CAP Procedure 6

SOER 02-04 Responses. Other SOER responses or GL responses 6

GET Training Modules related to SCWE 6

JCO Administrative Procedure

Plan and General Description of the Page 4 USAs Safety Culture Assessment Process Attachment I Phase 2-Assessment Implementation After site orientation the assessment will begin and include behavioral observations and interviews. The on-site assessment duration will be approximately 4 days, excluding site orientation and entrance and exit meetings with management. The assessment week activities will include behavioral observations, interviews, and, as required, document reviews.

Behavioral observations will be performed during meetings (e.g., daily plant meetings, safety review committee meetings, pre-job briefs, corrective action report screening meetings etc.). A behavioral observation guide has been developed for use by team members. Interviews will occur with a cross section of employees at all levels throughout the organization. Key executives will also be interviewed. A set of interview questions has been developed from the documents listed in the Reference Section of this White Paper.

Through the combination of interviews, observations, and data review, the team will score a matrix based on INPO's "Principles for Effective Operational Decision-Making". The matrix line item scores by themselves will provide limited usefulness; however, a ranking of the matrix scores will highlight areas in need of improvement for the assessed station leadership team. Just prior to the exit meeting the team will also collectively score a "Leadership Matrix" developed from INPO's "Warning Flags from Plants in Extended Shutdowns."

Team leads will debrief senior management on the results of the assessment periodically during the assessment week. The team will perform an overall debrief with site leaders at the exit meeting. The site leadership team will be responsible for carefully evaluating any problems identified in the assessment that could adversely affect nuclear safety. Opportunities for improvement will be identified by the site leadership team and as required promptly documented in the site's Corrective Action Program.

Phase 3-Final Report Preparation with Recommendations A preliminary assessment report will be provided to station management at the assessment exit meeting. The team lead and host peer will provide a final assessment report within approximately two weeks of the exit meeting. The final assessment report will be documented in accordance with the site's self-assessment process.

Plan and General Description of the Page 5 USAs Safety Culture Assessment Process The final assessment report will include as a minimum:

Final Assessment Report formatted to the site's SA process requirements.

4 Team recommendations for improvement.

4 "Effective Operational Decision-Making Matrix Tally Sheet" scores 4

"Effective Operational Decision-Making Matrix" scores "Leadership Matrix" scores (based on "Warning Flags from Plants in Extended Shutdowns Matrix").

4 Supporting documentation.

Safety Culture Survey A Safety Culture Survey extracted from NEI 97-05, "Nuclear Power Plant Personnel-Employee Concerns Program-Process Tools in a Safety Conscious Work Environment" will be performed independent from the assessment activities themselves. The results of the survey combined with the results of the assessment will then be available to the site's leadership team for the overall evaluation of site safety culture. The survey will be administered to the site by July 2003. The survey covers four general topic areas: Safety Conscious Work Environment, Employee Concerns Program, the Corrective Action Process, and Management Conduct.

Plan and General Description of the Page 6 USA's Safety Culture Assessment Process

References:

4 INPO SER 02-02, Undetected Leak in Control Rod Drive Mechanism Nozzle and Degradation of Reactor Pressure Vessel Head 4

INPO, Principles for Effective Operational Decision-Making INPO SOER 02-04, Reactor Pressure Vessel Head Degradation at Davis-Besse 4

INPO 02-005, Analysis of Significant Events 4

INPO, Principles for Effective Self-Assessment and Corrective Action Programs INPO 98-003, Managing By Experience 4

INPO, Principles for Enhancing Professionalism of Nuclear Personnel 4

INPO 97-003, Safety Focus During Changing Times 4

NRC 9130/02, Degradation of the Davis-Besse Nuclear Power Station Reactor Pressure Vessel Head Lessons Learned Report 4

NRC SECY-97-260, Resolution of public comments in response to request for public comments in the federal register notice, 'Safety Conscious Work Environment" 4

NRC, Policy statement for nuclear employees raising safety concerns without fear of retaliation 4

Dr. Richard A. Meserve, NRC, Meeting On Safety Goals And Safety Culture, Milwaukee, Wisconsin, June 18, 2001 INPO 97-002, Performance Objectives and Criteria for Operating Nuclear Electric Generating Stations 4

INPO, Warning Flags from Plants in Extended Shutdowns 4

FENOC, Technical Root Cause 4/17/02 4

INPO, Warning Flags C:\\DOC\\NSC\\PROGRAMtusa'lo.nrc.0905\\attachment 1.usa.process.wpd

ATTACHMENT 2 TO SEPTEMBER 29, 2005 LETTER TO M.R. JOHNSON "UTILITIES SERVICE ALLIANCE, INC.'S SPECIFIC COMMENTS ON ELEMENTS OF THE NRC'S SAFETY CULTURE ATTRIBUTES TABLE"

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 1 Safety Culture Attribute i-Safety Culture Element (SCE),

Potential Safety Culture Inspection Pot ntial Safe Culture (SCA)

-Information (SC) measurei (SCM)

An inherent A specific factor, process, or Qualitative Information that is acquired from an Quantifiable information that Is characteristic, quality, or process outcome that can either inspection to assess change or performance of acquired through an inspection (e.g.,

property that is critical to a be Inspected or measured and a Safety Culture Element.

that can be counted, trended or licensee's safety culture.

that can be used to assess a noted) which can be used to assess licensee's performance with change or performance of a Safety respect to the Safety Culture Culture Element Attribute(s).

SCA-1 Safety Con-SCE-1 Organizational re-SdCI A Status of SCWE is monitored SCM-1 Trend in the number or scious Work Environ-sponsibility for SCWE using appropriate performance measures type of NRC allegations/number ment (SCWE) and actions are taken in response to neg-or type of ECP concerns ative findings SCI-2 Employee Concerns Program SCM-2 Percentage of anony-(ECP) is effectively implemented (e.g.

mous ECP submittals confidential, accessible, appeal process, timely, appropriate resolution)

Comments SCWE is one important aspect of Safety Culture. Licensees have guidance from both the industry (NEI 97-05 [Rev. 2], NuclearPower Plant Personnel-Employee Concerns Program, Process Tools In A Safety Conscious Work Environment, December 2003) and the NRC (Policy Statement, "Freedom of Employees in the Nuclear Industry to Raise Safety and Compliance Concems Without Fear of Retaliation,"

published in 1 996,and 'Guidance for Establishing and Maintaining a Safety Conscious Work Environment," 2005) to allow them to establish, maintain, and assess a SCWE.

SCWE is already evaluated in an existing NRC Inspection Procedure (NRC Inspection Manual, Inspection Procedure 71152, Identification and Resolution of Problems). The Annual Status of Allegation Program already uses SCM-1, SCM-4 and SCM-5 (For example in Status of Allegation Program Calendar Year 2004 Annual Report, June 6, 2005) as well as other characteristics of NRC Allegations and ECP submittals.

For ease of discussion each Safety Culture Attribute,' 'Safety Culture Element,' 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 2 Safety CultureAttribute Safety Culture Element SCE)

Potetlafety Culture Insetio n otentalSafety C lt ure-:-

(SCA)

InformatIon (SCI)

Measure(SCM) :

Considerations of allegations for SCMs 1, 4, 5, and 18 should be limited to substantiated allegations to ensure that the ROP is based on confirmed facts. However, the number of confirmed allegations for most sites is very low and interpretation of such a small number will be difficult.

While NRC allegations are a source of insights about a licensee's SCWE that are already fully considered and analyzed. Additional inclusion in the ROP would be redundant.

The NRC has also recognized the difficulty in interpreting allegation numbers -

as both an increase and a decrease may be interpreted as a symptom of a declining safety culture.

It would be difficult for NRC inspectors to use or comprehend the trends in SCM-1 and SCM-2 unless they have full access to a licensee's Employee Concerns Program (ECP) and they fully understand the past trends of the site being evaluated.

SCM-2 may not be an indication of poor safety culture performance. Some individuals, because of their personality profile, will only submit anonymous concerns irrespective of the SCWE. It is simply their preference. Submitting an anonymous concern could be a positive aspect of safety culture because the individual is not afraid to submit a concern by their method of choice.

SCE-2 Personnel responsi-SCI-3 Management ensures employees SCM-3 Percentage of personnel bility for SCWE understand their responsibility to raise who have received initial SCWE issues, challenge unsafe acts, participate training in resolution of issues, and clearly com-municate issues to management Comments SCE-2 should also recognize Management's responsibility for SCWE" in addition to "Personnel responsibility for SCWE". SCM-3 would not be a valid measure for an individual's responsibility or their understanding of their responsibility.

Most general employee training programs already include some elements of SCWE training. The recommendation is also contained in the NRC's 2005 'Guidance for Establishing and Maintaining a Safety Conscious Work Environment." Notwithstanding the desirability of providing such training, the assumption that somehow, without training, a SCWE does not exist should not be used as a basis for a measure until it can be proven. The percentage of people who receive the training may be an indicator of management's commitment to a For ease of discussion each 'Safety Culture Attribute,' 'Safety Culture Element,' "Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

I

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 3 Safety Culture'Attribute Safety Culture Element (SCE)

Potential Safety Culture Inspection Potential Safety Culture-(SCA)

Information (SCI)

Measure (SCM)

SCWE, but what influences the safety culture is the effectiveness of the training and extent to which workers and supervisor apply what they learned.

SCE-3 Questioning attitude SCI-4 Management actions and commu-SCM-4 Annual total number of nication encourage challenging unsafe NRC allegations.

acts, voicing dissenting views, raising safety issues, and reporting anomalies SCI-5 Personnel at all levels of the orga-SCM-5 Annual number of NRC nization are aware of zero tolerance for allegations of chilling effect retaliation Comments SCM-4 and SCM-5 have the same problems with ambiguous interpretation and small numbers as does SCM-1 (See comments above).

Considerations of allegations for SCMs 1, 4, 5, and 18 should be limited to substantiated allegations to ensure that the ROP is based on confirmed facts. However, the number of confirmed allegations for most sites is very low and interpretation of such a small number will be difficult.

While NRC allegations are a source of insights about a licensee's SCWE that are already fully considered and analyzed. Additional inclusion in the ROP would be redundant.

The NRC has also recognized the difficulty in interpreting allegation numbers -

as both an increase and a decrease may be interpreted as a symptom of a declining safety culture.

The Inspection information and measures for SCE-3 may not fully support an evaluation of a questioning attitude. Having a questioning attitude is an important part of safety culture. Not having a question attitude can be related to more things than a poor SCWE. Davis Besse personnel did not have a questioning attitude for several reasons outside of their SCWE that would not be assessed by these measures. If you have weaknesses or indicators in several of these areas, then there may be a safety culture issue. Monitoring all these issues related to a questioning attitude is necessary to gauge status of safety culture in this area.

Unless a more comprehensive measure can be found, insufficient information would be available to identify safety culture problems like For ease of discussion each 'Safety Culture Attribute,' 'Safety Culture Element,' 'Potential Safety Culture Inspection Inforrnation,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 4 Safety Culture Attrbute;'

Safe CultureEloment(SCE)

P^liotential Safety Culture Inpec'tion Potential Safety Culture (SCA)

____-___l_:_-_-:___-__;-_____ J Information (SCI)

Meas=re (SCM) those at Davis Besse before an event.

SCA-2 Organizational SCE-4 Organizational learn-SCM-6 Ratio of industry OE re-learning & assessment ing & assessment ports evaluated versus total screened SCI-6 Searches of relevant OE are con-ducted when making significant modifica-tions to procedures or equipment SCI-7 Evaluations of OE are used to de-SCM-7 Annual number of report-velop appropriate corrective actions which able events or significant condi-are implemented in a time frame corm-tions adverse to quality attributed mensurate with their safety significance to inadequate action to previous OE reviews Comments SCM-6, although easily measurable, is too simplistic a measure of the effectiveness of an OE program and process. What is important is the effectiveness of the screening and evaluation process and what events it prevents. Events "prevented" are difficult or impossible to measure.

Determining inadequate action to previous OE reviews will require significant time and research for NRC evaluators to get an accurate picture. For example: How would the NRC score SCM-6 or SCM-7 when you have many generic type issues associated with digital systems, where failures are often very specific to a particular vendor or a particular revision to firmware, software or card revision? Is this an indication of declining safety culture or is this a vendor performance issue? This is a matter for the licensee to consider, but the impacts are mostly outside of the safety culture. These measures would be difficult to interpret as a numeric indicator of a safety culture.

SCE-5 Self-assessment pro-SCI-8 Self-assessments are of appropri-SCM-8 Annual number of repeat cess ate scope (e.g. identify latent conditions),

findings in self-assessments For ease of discussion each 'Safety Culture Attribute,- 'Safety Culture Element,' 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 5 Safety Culture Attibute Safety Culture Element (SCE)

Potential Safety Culture Inspection PotentIal Safety Culture

'(SCA):

InfomatIo (SCI)

Measure (SCM) are self-critical, and appropriate actions are taken SCI-9 Periodic assessments are con-ducted to evaluate the effectiveness of internal and external oversight groups Comments SCM-8 could be an indication of a strong, in-depth and rigorous reporting system in self-assessments and therefore a positive safety culture performance, or it could be indicative of a poor self-assessment process and safety culture.

However, the timing, nature, and subject of self-assessments is influenced by other factors such as outage schedule, industry events, and regulatory changes, which may or may not be related to safety culture. Hence, any kind of trend assessment would be difficult.

SCE-6 Problem identification SCI-10 Complete and accurate identifica-SCM-9 Percentage of self-identi-

& resolution/corrective action tion of the problem in a timely manner fled SCAQs and CAQs versus program commensurate with its significance and those that are self-revealing or ease of discovery identified by an external organi-zation SCI-1 1 Classification and prioritization of the resolution of the problem commensu-rate with its actual or potential safety sig-nificance SCI-12 Management appropriately chal-lenges the effectiveness of root cause evaluations (e.g. Corrective Action Re-view Board (CARB), Plant Operating Re-For ease of discussion each 'Safety Culture Attribute," 'Safety Culture Element," Potential Safety Culture Inspection Information," and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 6 Safet Culture Atbute p Safety Culture Element (SCE)

Potential Safet Cultue Inpecti;n Pential Safety Culture (SCA)

Iformation (SCI)

Measure (SCM) view Committee (PORC)

SCM-10 Average time for com-pleting corrective actions for SCAQs and CAQs Comments If SCM-9 is to be used as a measure, what is an 'external" organization would need to clarified. Oversight, NRC or INPO identified issues are easy to identify as external organization but what about an oversight board or another department? If another department, such as maintenance, identifies an operations issue is this considered identified by an external organization? Also, the identification of issues by outside organizations could mean the "outside" organization is a site strength or a positive aspect of safety culture or a weakness in the self-assessment or oversight processes.

SCI-1 0 may be an indicator about the accuracy and timeliness for the identification, but there appears to be no measures for this. It would be difficult to assess this indication. If an issue is easy to discover and it takes 2 months to discover the issue, is this satisfactory performance compared to discovering a latent hard to discover issue that has been around for 10 years? How to gauge timeliness and accuracy of identification is not clear.

The metric for SCI-1 would be very subjective. Classification at the proper level is not as important as doing something with the safety issue in a timely manner. The safety significance of issues can be hard to determine unless an event occurs.

SCI-12 could be subjectively evaluated by attending the selected meetings and observing management challenges and coaching. This could be a valuable input to measuring safety culture symptoms such as complacency, or lack of a questioning attitude, or lack of coaching or lack of accountability. However, this type of safety culture observation should not be restricted to root cause but should include daily meetings and pre-job briefs adequacy. It is fairly easy to observe behaviors that do not support a strong safety culture.

Observing these activities (daily meetings and pre-jobs) could also be a valuable input towards the evaluation of the site's safety culture.

SCM-10 average time for corrective actions can be affected by performance of outside contractors, regulators, vendors etc. and if this measure is used, this would have to be factored in. For example if the NRC takes 1 year to review a technical specification change (a corrective action) the metric of average age could be affected.

For ease of discussion each 'Safety Culture Attribute," 'Safety Culture Element," 'Potential Safety Culture Inspection Information," and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

so r,

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 7 Safety Culture Attribute Safety Culture Element (SCE)'-

Potential Safety C iture Insption PotentIal Saf Culture (SCA)

Inf r ationure (SCM) '

SCE-7 Continuous learning SCI-13 Site training program incorpo-environment rates new and emerging issues SCI-14 A process for knowledge transfer exists for the transfer of critical informa-tion and decision making SCM-11 Percentage of opera-tors who fail requalification exam-ination Comments SCI-13 and SCE-14 could be good input for the evaluation of the site's safety culture. However, the measure is not whether or not the training program incorporates new and emerging issues, but whether or not the training provided is effective in preventing events. The volume of the training program is not important, it is the correct choice of subject and the effectiveness of the training iswhat is important.

SCM-1 1 may not provide the necessary input into site culture. Failing a requal exam in many cases could be a positive indicator for safety culture status. Developing a good exam process that is challenging and points out weaknesses in operator performance could be a positive aspect of the site's safety culture.

SCE-8 Benchmarking SCI-15 The results of benchmarking ac-SCM-12 Number of improve-tivities are evaluated and specific recom-ments implemented per mendations are developed, implemented, benchmarking activity and communicated Comments Benchmaking is part of a learning organization, a desirable trait for safety culture, however it s not a regulated activity.

Further, the "state of the art," hence the value of benchmarking, in any area or overall does not advance evenly hence the number of For ease of discussion each 'Safety Culture Attribute," "Safety Culture Element,' 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

- V Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 8

-Safety CultureAttribute Safet CultureElement(SCE)

PotentiaSfet CureInspetio Potential Safety Culture (SCA)

Information (SCI)

Measure (SCM) improvements may vary wildly independent of a safety culture.

How would the NRC differentiate benchmarking from tours of other facilities. Monitoring the number is not as important as monitoring the quality of actions taken from a benchmarking trip. There will always be some benchmarking trips that do not result in site improvements or enhancements.

Might this measure discourage a licensee for taking benchmarking trips unless it had some assurance that improvements would likely result?

SCA-3 Work Planning SCE-9 Work control SCM-1 3 Number of preventive and Human Perfor-maintenance deferrals mance SCM-14 Percentage of correc-tive maintenance versus preven-tive maintenance SCI-16 Work planning and coordination considers the sequencing of system avail-ability and limits system unavailability SCI-17 Work planning and coordination ensures personnel are qualified, have access to the right procedures and job aides, and have appropriate instructions (e.g. expected outcomes, job conditions, hold points, contingencies for work, and stop work criteria) and equipment Comments For ease of discussion each 'Safety Culture Attribute,' 'Safety Culture Element,' 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 9 Safety Culture Attribute':

'Safety Culture Element (SCE) {I Potential Safety Culture Inspection

'Potential Safety Culture (SCA).

InfoVa;ition (SCI) - -

Measur SCM)

Variations in different plants CAP processes would make SCM-13 and 14 ripe candidates for inaccuracies and inconsistencies in reporting, which may not always be obvious. Likewise, over time periods of less than a few years, they are strongly affected by refueling intervals and schedules and other extended planned outages such as those for major turbine work or steam generator replacements.

Unavailability hours for safety systems are being tracked and might be a more useful metric for this element of work control. Rework, PMT failures, and Repeat issue numbers would also provide an indication of the strength of work control processes or human performance but is susceptible to inconsistent reporting.

All of these measures are already regulated and measured more effectively under the Maintenance Rule and other regulations.

SCE-10 Systematic decision SCI-18 Changes that are screened per making 10CFR50.59 appropriately consider safety issues and are based on conserva-tive assumptions SCI-1 9 Steps are taken to ensure that sufficient design margins are maintained when making changes to plant equip-ment, procedures, and personnel SCM-15 Number of NRC find-ings related to inadequate sys-tematic decision making (e.g.

operability evaluations, 50.59 evaluations)

SCI-20 There is evidence of interdepart-mental communication, coordination, co-operation, and decision-making at all lev-els of the organization (e.g. minimize con-For ease of discussion each 'Safety Culture Attribute," 'Safety Culture Element," 'Potential Safety Culture Inspection Information," and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

.I' Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 10 SafetyCultureAttibute Safety Culture Element (SCE)

P!-

otential Safety Culture Inspection Pote Safe ultue (SCA)

I formation (SC!)

t-Measr SCM) flict and enhance effectiveness of activi-ties)

Comments Efforts taken to increase design margins rather than just to maintain design margins should be included as an inspection item. Living with degraded margins should be an important factor in assessing safety culture. However, these factors would not be easy to measure.

Hesitation to shutdown the plant or lack of management reinforcement on operator conservative decision making should be an input into the assessment of this element, but it is not clear how it could be measured.

SCI-20 fits better under the next element (Conduct of Work). Rework may also fit better under SCE-1 1.

SCE-11 Conduct of work SCI-21 Procedure compliance is commu-SCM-16 Percent of condition (including maintenance, oper-nicated to personnel; personnel under-reports that are associated with ations, and engineering) stand which procedures require verbatim personnel not following proce-compliance, and such procedures are dures appropriately followed SCI-22 Human performance is closely SCM-17 Trend of human perfor-monitored and assessed, and significant mance error rate human performance issues are communi-cated to station personnel SCI-23 Appropriate interfaces are main-tained with offsite organizations (e.g. grid operators) that could impact nuclear sta-tion operations Comments For ease of discussion each 'Safety Culture Attribute,- 'Safety Culture Element,' 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

4V Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 11 Safety CultureAtbue Safety Culture Element (SCE) 1 Potential Safety Culture Inspetion Potential Safety Culture (SCA):

_Information (SCI)

Measure (SCM)

Procedure non-compliance trend codes on event reports, which are a typical way for plants to measure certain aspects of human performance, could identify many issues of no consequence and hence would be difficult to evaluate. A trend or count of the number of procedure compliance issues "with consequence" is a candidate for a SCM SCA-4 Organizational SCE-12 Safety policies SCI-24 There is policy on commitment to Safety Accountability safety over production with evidence that it is reinforced and communicated SCM-18 Annual number of pro-duction over safety concerns raised to the NRC allegation pro-gram or the ECP SCI-25 Corporate and plant nuclear oversight groups perform effective as-sessments Comments Considerations of allegations for SCMs 1, 4, 5, and 18 should be limited to substantiated allegations to ensure that the ROP is based on confirmed facts. However, the number of confirmed allegations for most sites is very low and interpretation of such a small number will be difficult.

While NRC allegations are a source of insights about a licensee's SCWE that are already fully considered and analyzed. Additional inclusion in the ROP would be redundant.

The NRC has also recognized the difficulty in interpreting allegation numbers -

as both an increase and a decrease may be interpreted as a symptom of a declining safety culture.

SCE-13 Accountability and incentive programs SCI-26 Managers are present during crit-ical activities and demonstrate a proper For ease of discussion each 'Safety Culture Attribute,' 'Safety Culture Element,' 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

V.

Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 12 Attribute Safety Culture Element (SCE)

Potentlal Safety Culture Inspe ti P otential Safety Culture;-

(SCA)

Informatin (SCI)

Me asure (SCM) safety focus SCI-27 Incentive programs reward safety behaviors and achievements Comments A potential metric might be the number and quality of observations in the observation programs. However, since most aspects of an observation process are not regulated activities and vary from site to site, a comparison and basis for determining what a change meant would be difficult. Likewise, outages and other major plant events would be expected to strongly influence the number of observations and their individual importance and nature.

The number of hours that mangers are present might be a positive safety culture indicator (management involvement) or a negative indicator (lack of trust in on-shift personnel). "Manager's presence" is also not a regulated activity.

Compensation of employees is not a regulated activity and should not be in the ROP. The cultural question is how a compensation program, including incentives, influences the culture and behavior.

SCE-14 Adequate resources SCI-28 Adequate resources are provided SCM-19 Engineering backlog to maintain plant calculations, drawings, trend (e.g. FSAR updates, etc.)

FSARs, and other design and licensing basis documents SCI-29 Working hours are within NRC SCM-20 Annual number of ap-guidelines proved deviations from the work-ing hours guidance Comments Staffing levels, staffing transition, and maintaining adequate experience levels in engineering and in other areas can be evaluated by looking at overtime hours, number of human performance events, and failure to implement change management or succession planning.

For ease of discussion each 'Safety Culture Attribute," 'Safety Culture Element," 'Potential Safety Culture Inspection Information,' and 'Potential Safety Culture Measure' had been identified as a numbered SCA-, SCE, SCI and SCE as shown.

-I Specific Comments on Elements of the NRC's Safety Culture Attributes Table Page 13 Safety Culture AttributeE-Safety Culture Element (SCE)

' Potential Safety Culture Inspecton Potential Safety Culture (SCA)

I Informatlon (SCI)

Measur (SCM),

Metrics could be number of year of experience in a particular group or department or retention trends of critical resources.

Given a site's culture and long term manning expectations, SCM-20 should consider only the trends in approved deviations from the working hours guidance. SCM-20 would be so strongly influenced by outages and external events that it might be meaningless.

SCE-15 Organizational SCI-30 Change process and basis of change management decisions for major organiza-tional/resource changes are communi-cated to staff, as appropriate SCI-31 There is a systematic process for evaluating the impact of organizational changes and evidence that the process is used Comments See comments above.

SCI-31 should look at the effectiveness of the process and how its application avoids certain kinds of event. However, events that 'don't happen" as a result of good change management would be hard to identify. Given that the drivers for major, systematic changes are often externally driven (changes in regulation for example) trending would be problematical.

For ease of discussion each "Safety Culture Attribute," 'Safety Culture Element," 'Potential Safety Culture Inspection Information," and 'Potential Safety Culture Measure" had been identified as a numbered SCA-, SCE, SCI and SCE as shown.