NUREG/CR-7295, Human Factors in Nondestructive Examination

NUREG/CR-7295, Human Factors in Nondestructive Examination

ML22083A071
Person / Time
Issue date:	03/31/2022
From:	Harrison J, Stephanie Morrow, Carol Nove, Sanquist T Office of Nuclear Regulatory Research, Pacific Northwest National Laboratory
To:
Dickey K
References
PNNL-32505 NRUEG/CR-7295
Download: ML22083A071 (55)

Text

NUREG/CR-7295 PNNL-32505 Human Factors in Nondestructive Examination Office of Nuclear Regulatory Research

NUREG/CR-7295 PNNL-32505 Human Factors in Nondestructive Examination Manuscript Completed: October 2021 Date Published: March 2022 Prepared by:

T Sanquist, Pacific Northwest National Laboratory S Morrow, U.S. Nuclear Regulatory Commission J Harrison, Pacific Northwest National Laboratory C Nove, U.S. Nuclear Regulatory Commission Pacific Northwest National Laboratory 902 Battelle Blvd, Richland, WA 99354 Carol Nove, NRC Contracting Officer Representative Office of Nuclear Regulatory Research

ABSTRACT This report summarizes key findings from a five-year research program investigating the different ways human factors influence nondestructive examination (NDE) outcomes. The research team reviewed specific incidents and events involving manual and encoded ultrasonic testing and conducted detailed interview studies with NDE examiners. The results from these studies reveal a range of organizational, group, environmental, task, and individual factors that combine to influence NDE reliability. The research team integrated these findings with contemporary concepts of latent organizational error to illustrate how multiple interacting factors can lead to suboptimal outcomes when performing qualified examinations. This information provides a technical basis for understanding the precipitating factors that can lead to human error in NDE and various system-level and person-level interventions that can mitigate errors.

System-level mitigations include organizational approaches to information accuracy, inspection scheduling, data review and interpretation, workforce skill development, and change management. Person-level mitigations include shift scheduling to avoid fatigue, reducing distraction and time pressure, individual skill development through training and practice, knowledgeable team composition, cognitive aiding, and individual data review processes.

Human error is a systems problem, which can be reduced by various interventions to manage the inherent variability in organizational processes and cognitive functions at the individual level.

iii

FOREWORD In the 1980s, the nuclear industry and the Nuclear Regulatory Commission (NRC) were confronted with several challenges with regard to ultrasonic examinations of piping welds. The ultrasonic examinations at the time were not capable of reliably finding cracks in austenitic materials, most notably in the recirculation piping in boiling water reactors. A common way of discovering large circumferential cracks in Class 1 welds was via leakage after the cracks propagated through the entire thickness of the pipes.

Human factors studies in the 1980s found that ultrasonic examination personnel were often working under conditions that reduced their effectiveness in the field. Excessive noise, high temperatures, fatigue, distractions, time pressures, management pressures, and poor equipment design were found to be issues. These human factors studies made recommendations for future work to address and correct these conditions.

These identified human factors issues were considered a lower priority than the problem that ultrasonic inspection procedures and equipment were often incapable of reliably finding cracks in austenitic welds even under ideal conditions. To deal with issues regarding procedures and equipment, the NRC and industry worked to implement performance demonstration testing.

This work resulted in American Society of Mechanical Engineers Boiler and Pressure Vessel Code,Section XI, Appendix VIII. The performance demonstrations, coupled with advancements in ultrasonic examination technology, provide assurance that the ultrasonic examinations are capable of finding flaws. Today, the NRC and industry have confidence that ultrasonic equipment, procedures, and personnel that meet the requirements of Appendix VIII will be able to find flaws in components under ideal conditions.

Conditions in the field are often not ideal, however. Operational experience in the past two decades has shown that flaws missed in examinations of nuclear reactor components were, in fact, detectable by the examination equipment and procedures. Human factors issues such as time pressures, poor communication between licensees and contractors, and cognitive errors in data analysis now often play larger roles than technology in missed detections. This operating experience motivated the NRC staff to take up some of the recommended human factors work that had been described in the 1980s and look into the human factors issues that can result in failed examinations today.

Given the evolution and improvements in nondestructive technology and qualifications, the NRC is working to evolve its evaluations from focusing primarily on technological shortcomings and look more at the fundamental organizational, cognitive, and environmental factors that can lead to a failed examination. This NUREG/CR provides key insights into the human factors issues that can hinder an examination; it will be used to guide the NRC through future evaluations of inspection issues.

Stephen Cumblidge Materials Engineer U.S. Nuclear Regulatory Commission v

TABLE OF CONTENTS ABSTRACT ...............................................................................................................................iii FOREWORD ..............................................................................................................................v LIST OF FIGURES ....................................................................................................................ix LIST OF TABLES ......................................................................................................................xi ABBREVIATIONS AND ACRONYMS .....................................................................................xiii 1 INTRODUCTION .................................................................................................................1-1 2 PURPOSE AND SCOPE.....................................................................................................2-1 3 TERMINOLOGY AND CONCEPTS ....................................................................................3-1 4 REVIEW OF NDE INCIDENTS/EVENTS.............................................................................4-1 4.1 Manual Conventional UT ..............................................................................................4-1 4.2 Encoded UT .................................................................................................................4-2 5 NDE PERFORMANCE INFLUENCING FACTORS .............................................................5-1 6 NDE FUNCTIONAL TASK STRUCTURE AND PERFORMANCE INFLUENCES ..............6-1 7 NDE OUTCOME/ERROR AND CONSEQUENCE MODELING ..........................................7-1 8 NDE AND THE LATENT FAILURE MODEL .......................................................................8-1 9 NDE ERROR REDUCTION APPROACHES .......................................................................9-1 9.1 System-level Mitigations ...............................................................................................9-1 9.2 Person-level Mitigations ...............................................................................................9-2 9.3 Error Management Principles .......................................................................................9-3 10

SUMMARY

AND CONCLUSIONS ....................................................................................10-1 11 REFERENCES ..................................................................................................................11-1 vii

LIST OF FIGURES Figure 5-1 Categories of NDE Performance Influencing Factors ........................................5-1 Figure 6-1 General Functional Structure of NDE Tasks ........................................................6-1 Figure 8-1 Organizational and Operational Defenses and Error-Inducing Factors UT Inspection and Influences over Time .................................................................8-2 Figure 8-2 Examiner Cognitive Processes Are Affected by Local Conditions That Can Increase the Likelihood of Error .................................................................8-3 ix

LIST OF TABLES Table 5-1 NDE Performance Influencing Factors and Descriptions ...................................5-1 Table 7-1 Potential Outcome/Error Types, Precipitating Factors, and Consequences for Manual and Encoded UT Functions .............................................................7-1 xi

ABBREVIATIONS AND ACRONYMS ASME American Society of Mechanical Engineers BPV Boiler and Pressure Vessel CASS Cast austenitic stainless steel DM Dissimilar metal EPRI Electric Power Research Institute ISI inservice inspection NDT nondestructive testing NPP Nuclear power plant NRC Nuclear Regulatory Commission OE Operating experience PDI Performance Demonstration Initiative PNNL Pacific Northwest National Laboratory POD Probability of detection UT ultrasonic testing xiii

1 INTRODUCTION Nondestructive examination (NDE) plays a vital role in ensuring the safety of nuclear power plant (NPP) operations. It is used to detect flaws and deficiencies in steam generators, pipe welds, vessels, valves, pumps, and other critical components in an NPP (EPRI 1988). The U.S.

Nuclear Regulatory Commission (NRC) and the nuclear industry have devoted considerable attention to improving the reliability of the inspection process for NPP components. These efforts have taken place during times of emerging materials degradation mechanisms and significant changes in inspection technology.

There has been substantial progress in improving NDE reliability over the past two decades through the development of rigorous qualification processes, including performance demonstrations for NDE equipment, procedures, and personnel, via the requirements of the American Society of Mechanical Engineers (ASME) Boiler and Pressure Vessel Code (BPV Code),Section XI, Appendix VIII. Although performance demonstration helps to ensure that equipment, procedures, and personnel are capable of reliably detecting flaws in a formal testing environment, notable failures have occurred during application in the field. In each case, the equipment and procedures were physically capable of obtaining discernable signals from the flaws.

These recent events suggest that robust techniques and qualifications are necessary but do not guarantee 100% detection for field examinations. The effective application of NDE is dependent on the personnel performing the examination, the design of the task, along with the environmental and organizational conditions within which personnel carry out the task. One aspect of understanding NDE reliability in the field is to consider the role and impact of human factors, which is broadly construed to address influences from the individual examiner to the organizations involved.

Human factors research in NDE was initiated by the NRC in the early 1980s as a result of concerns about human reliability in ultrasonic testing (UT) after the discovery of a defect at the Edwin I Hatch NPP. Inspection requirements for NPPs had been developed on the basis of prescriptive standards from other industrial manufacturing processes and experience with fatigue cracking (Doctor et al. 2013). Subsequent experience in the United States and internationally suggested that human performance in UT was quite variable, depending on many factors, including the nature of the defect, the equipment, examiner experience, etc. A number of systematic studies known as mini-round robins, in which multiple examiners evaluated multiple test specimens, showed considerable variation in inspection results (EPRI 2008). Early studies by Pacific Northwest National Laboratory (PNNL) concluded:

Although there is an increasing awareness of the importance of human performance in NDT [nondestructive testing], no systematic evaluations of the variables likely to influence the performance of technicians and/or the man-machine system have been conducted to date. An urgent need exists for systematic, multi-variable evaluations in the NDT field, particularly for those areas where significant demands are placed on the technician, and where the ISI [inservice inspection] results are of critical importance to the integrity of nuclear power plant components (Spanner et al. 1986).

Despite the suggestion of Spanner et al. (1986), relatively little work was directed at human factors in NDE between 1986 and 2016, though more recent findings of flaws at the VC Summer, Ringhals, Duane Arnold, and North Anna plants suggest that human factors remain a 1-1

concern (Doctor et al. 2013). Most research on the reliability of NDE has tended to focus on quantifying probability of detection (POD) in blind performance testing of samples. While sample-based blind testing tends to yield quantitative results, performance is highly variable (Heasler et al. 2003; Meyer et al. 2019; Wheeler et al. 1986). The limited work that has looked at human factors in NDE is aptly captured by Spanner et al. (1986) who state that no single human performance factor is responsible for the wide performance variations (observed). The principal industry focus on human reliability in the past 30 years has been on development and implementation of the Performance Demonstration Initiative (PDI). This initiative, administered by EPRI to meet the requirements of the ASME BPV Code,Section XI, Appendix VIII, qualifies the capability of procedures, personnel, and equipment to detect known flaws through blind testing on samples with known flaws in a laboratory setting.

In 2016, based on the continuing occurrence of apparent human performance issues in ultrasonic NDE, the NRC initiated research projects to address human factors and to identify potential mitigations that go beyond traditional training certification and performance demonstration qualifications. These projects entailed reviews of relevant scientific and technical literature, field research focusing on analysis of tasks, and interviews with subject matter experts (D'Agostino et al. 2017; Sanquist 2020; Sanquist and Harrison 2021; Sanquist et al.

2018). This NUREG/CR presents a synopsis of that work.

1-2

2 PURPOSE AND SCOPE The purpose of this document is to synthesize the results of NRCs human factors projects executed between 2016 and 2021 that have focused on specific NDE and ultrasonic testing issues. The goal is to provide a high-level synopsis of problems addressed, research approaches employed, findings obtained, and mitigation strategies identified. The audience for this report is primarily a non-research communityregulators, vendors, and utilities. The work described in this report can be used to anticipate and understand the general classes of problems that are likely to be encountered by the NDE community in the future. Application of known research methods and findings from the current research program can be used to address those issues.

This report is based on four specific projects that span various problem areas and methods.

The initial project consisted of a literature review to identify empirical research addressing human factors in NDE (D'Agostino et al. 2017). The second project included a task analysis and interviews with more than thirty examiners who conducted manual UT (Sanquist et al.

2018). This was followed by a literature review addressing the experience requirements for skill development and focused on fundamental principles of human learning and memory as they pertain to developing and maintaining expertise (Sanquist 2020). Most recently, research was conducted to analyze tasks and interview examiners who conducted encoded UT (Sanquist and Harrison 2021). Taken together, this body of research provides a basis for:

• integrating high-level conclusions about performance influencing factors that affect NDE

• identifying likely problems to be encountered in the future

• applying methods for addressing those problems

• developing specific event and more general operational guidance.

The individual project reports provide specific details on methods and findings. This report integrates the results of these projects in general conclusions that are applicable to the entire range of NDE methods used in NPPs.

We start this report with a review of NDE incidents and events that have occurred over the past 10-year period. This is followed by a discussion of how suboptimal NDE outcomes are complex events that generally do not involve a single point failure, but instead are the result of numerous interacting factorsorganizational, technical, and individual. We then discuss the performance influences that have been identified by examiners, vendors, and utilities as affecting the various functional phases of the examination process. The analysis is expanded by an error modeling approach that illustrates the linkage between specific precipitating factors, types of errors, and potential consequences. The results are then synthesized into a set of error management principles and specific mitigation strategies for particular types of errors.

2-1

3 TERMINOLOGY AND CONCEPTS While we use the term error in this report, contemporary thinking about incidents and events suggests that error should not be construed as a human fault (Reason and Hobbs 2003; Woods et al. 2010). This term is highly problematic in that it tends to reduce a complex chain of events to a single cause with no explanatory value and further focuses remediation on traditional, and often ineffective, approaches such as additional training, procedures, oversight, and other complexities that can introduce additional vulnerabilities. Instead, error encompasses conditions that may be present at the organizational level (e.g., specific plant/utility practices), specific occurrences of indications being missed or misinterpreted by examiners, and consequences resulting from interactions among these factors. A more useful way of thinking about NDE failures, whether they result from improper planning, examiner attentional lapses due to fatigue, time pressure, or other pre-conditions, is to consider them as undesirable outcomes. For example, an examiner may do everything according to procedure, examine a weld that is properly surfaced, interpret the results properly, but the outcome would still be an error because of improper paperwork directing the examination of the wrong component (this has happened). While we will continue to use the term error in this document as a shorthand expression, its use is not intended to imply human failure, but instead an event or series of events that may result in misinterpretation or failure to detect a flaw.

NDE is best considered as a complex sociotechnical system that involves multiple layers of performance influencing factors, from organizational and environmental variables to more focused elements, such as differences between individuals, task structure, and knowledge and skill of the examiner. NDE activities can be conceptualized as involving system-level variables and person-level variables (Reason 2000). An example of a system-level variable would be the work experience hours required for various levels of examiner certificationthese requirements are established through a complex and lengthy organizational process involving utilities, vendors, regulators, and industry groups. The system-level requirements in turn affect the knowledge and skill of individual workers and their practical field experience. The discussion in Section 4 can be useful in assessing the system-level and person-level variables that can act as precipitating factors leading to undesirable NDE outcomes.

3-1

4 REVIEW OF NDE INCIDENTS/EVENTS 4.1 Manual Conventional UT Operating experience (OE) and event reports documented in the NRC white paper on Improving Effectiveness and Reliability of NDE (NRC 2016) provide several case examples of manual UT errors that can be analyzed in terms of interacting system- and person-level influences.

A very well-documented reliability issue occurred at North Anna Unit 1, in which manual UT missed five large, axially-oriented flaws in a steam generator primary inlet dissimilar metal (DM) weld. These flaws were later detected as a result of through-wall leakage during a machining process to prep the weld for a full structural weld overlay. The specific configuration involved an outside diameter taper with a design that was not included in the generic PDI specimen set; thus, the licensee used a site-specific qualification with open mockups to address the configuration. The issues discovered in the event review included:

• UT probes for the site-specific examination were likely not tested after receipt by the utility to ensure proper impingement angle at the inside diameter of the pipe.

Subsequent modeling revealed the impingement angle to be 30 degrees, rather than the 40 degrees specified by the utilitys site-specific procedure (Anderson et al. 2012; NRC 2016).

• Modeling showed that the probes were actually limited in their ability to detect interior diameter flaws in a manual real-time examination.

• The technical justification for the site-specific mockups allowed extremely wide latitude for PDI procedure deviation that essentially undermines the intent of Appendix VIII to ensure that all inspections go through blind performance demonstrations to prove they are effective and robust (Anderson et al. 2012).

• A team scanning approach was used, which is not currently addressed in the ASME Code or qualified via the PDI.

This event illustrates a confluence of circumstances involving organizational factors of planning and procedure qualification that led examiners to miss significant flaws because the technique being applied was not adequately qualified.

At Duane Arnold Energy Center Unit 1, intergranular stress corrosion crack indications in a recirculation riser safe end-to-nozzle DM weld were detected in 2007, and through review of prior inspections were discovered to have been present in the 1999 and 2005 inspections.

Review of these findings (PNNL 2007) led to a number of conclusions about the overall process and procedure, including:

• Scanning with an automated device with gel-type couplant may have gummed-up the transducer wedge, resulting in poor coupling between the transducer and the inspection surface, thus reducing signal quality.

• The probe movement was made in the forward and reverse raster motions. While this saves time, it can reduce signal quality due to different mechanical pressures and wiping of couplant from the surface.

• Software limitations on the device precluded thorough investigation of geometrical and flaw responses, and pixilation of the images does not provide adequate resolution for depth-sizing, and in some cases, for determining accurate spatial positions of 4-1

responses. This can make distinguishing cracks from geometrical reflectors very challenging.

• The PDI-qualified procedure contained vague guidance regarding signal characterization, and the difficult field conditions were insufficiently represented in the original PDI qualification.

• Although, not certain, it also appears that the 2005 data was not independently reviewed, since some of the issues noted could have been identified and addressed as a result of an independent review (NRC 2016).

This event illustrates a range of human factors issues associated with the realism of procedure qualification, field procedure execution, inspection procedure clarity, and equipment design problems.

A human performance error in utility planning is illustrated by an event at R.E. Ginna Power Plant that was discovered to have occurred in 2008. NRC review of documentation submitted by the plant in a relief request revealed that an inspection had been performed using a stainless steel to cast austenitic stainless steel (CASS) procedure on a CASS-to-CASS component.

Further investigation revealed that the sites NDE staff had originally populated the ISI list with incorrect informationthey had identified the weld incorrectly. The vendor reviewed historical data, which was based on the use of the correct CASS calibration block, but the incorrect procedure was performed nonetheless. Post-examination reviews of the weld by the vendor and site personnel did not reveal the error. This event illustrates human and process errors at multiple time-points. The specific procedure execution was correct, but not appropriate for the component examined. This appears to be an issue partially related to assuming that information received is correct and failing to notice discrepancies throughout the process.

An event at Point Beach Nuclear Plant in 2011 involved NRC inspectors observing an NDE examiner failing to perform additional scans required by NDE-173, PDI Generic Procedure for the Ultrasonic Examination of Austenitic Piping Welds. When this oversight was pointed out, the examiner performed the additional required scans. The procedure in use was issued as informational use, a type of designation that allows the examiner to rely on memory, rather than following the specific steps in a written procedure in their presence. The procedure was an updated version, and failure to recall the most recent steps would be considered a memory lapse resulting from informational use procedures.

4.2 Encoded UT Recent OE also suggests the need for a better understanding of the human factors associated with encoded UT examination reliability. For example, in May of 2013, a review of UT data from the Shearon Harris NPP revealed an indication that was missed during an inspection in 2012. A root cause analysis determined that the indication was challenging to detect and that there was little the licensee could have done directly to cause the analysts to miss the indication.

However, a special inspection report (NRC 2013) noted that analyst working conditions, including tight quarters, noise, distraction, and fatigue from long work hours, may have contributed to the missed indication. Independent review of the data, which might have identified the indication, was not performed during this inspection.

In November of 2018, Palisades found leakage during visual inspection of its reactor pressure vessel head. Further analysis of historical data revealed that the leak was caused by primary water stress corrosion cracking in a control rod drive mechanism penetration tube. The crack had grown over the years and was missed by inspectors in eight inspections over the course of 4-2

11 years (ultrasonic data collected in 2007, 2009, 2010, 2012, 2014, 2015, 2017, and 2018).

Additional information provided by the NRCs (2019) review of the vendor root cause analysis suggested that examiner training that focused on outer diameter flaws led to a biased mind-set among analysts to discount the possibility of inner diameter cracking, and that the practice of comparing current inspection data with the most recent past examination obscured substantial growth in the flaw. As with the Shearon Harris event, independent review was not conducted.

The root cause analyses of both events revealed that the flaw indications were represented in the recorded data, and that the procedures used were qualified to reveal such flaws. Human factors affected analyst performance, including the misconception that the indication for Palisades was caused by geometry on the inner diameter, and for Shearon Harris, suboptimal inspection conditions and fatigue resulting from long work hours. Other cognitive and work practice factors and potential mitigations are discussed in the NRC inspection reports (NRC 2013; NRC 2019). It is noteworthy that both events occurred during reactor head exams, which tend to receive more attention from the utility due to the potential impact on the outage schedule; this can lead to implicit or explicit time pressure among the examiners.

4-3

5 NDE PERFORMANCE INFLUENCING FACTORS NDE can be conceptualized as a complex system of functions and influences that occur over an extended period of time, culminating in a component examination. The events reviewed above show that there is generally no single point of failure in the NDE process, but instead a confluence of multiple factors. Our research has shown that these influences cluster into five general categories as shown in Figure 5-1. Descriptions for each of these performance influencing factors (PIFs) are provided in Table 5-1.

Figure 5-1 Categories of NDE Performance Influencing Factors Table 5-1 NDE Performance Influencing Factors and Descriptions Category PIF Description Pre-job Preparation Activities performed by the vendor to get ready for an Task exam Characteristics Equipment Specific equipment and material used to perform exam and its availability 5-1

Table 5-1 NDE Performance Influencing Factors and Descriptions (cont.)

Category PIF Description Procedure Content and/or nature of a written procedure used in an exam Time Pressure Temporal constraints due to specific exam performance or the overall inspection schedule Task Complexity Factors such as ambiguity in assessing or executing the task, the degree of mental effort or knowledge involved, whether special sequencing or coordination is required, or whether the task requires sensitive and careful manipulations Knowledge/Experience What the examiner knows, level of experience on the job, and certifications and qualifications Examiner Process How the examiner executes the specific task (reporting comments, etc.)

Individual Motivation/Attitude/Personality Characteristics of the person Differences Physical Abilities Lifting capacity, flexibility, reach, dexterity, etc.

Cognitive Factors Attention, perception, memory, spatial ability Workload/Stress/Fatigue Pace, intensity, and duration of exam, work shifts or assignments Team Coordination Peer interaction while doing the inspection; this Group should be focused on performing the exam Characteristics Team Cohesion Familiarity of inspection team with one another and the impacts upon exam process Accessibility of Component Location, reachability Lighting Visibility Physical Noise Ability to hear while doing exam Environment Radiation Task-specific dose, cumulative dose Temperature and Humidity Heat, dehydration, glasses fogging, etc.

Utility Planning Activities conducted by the utility to prepare for exam Organizational Culture Norms and expectations in the work environment (the feeling of the workplace)

Supervision Oversight of the NDE processeither directly by vendor, utility, or regulator Organizational Training Comments about types and quality of training, Factors required and optional practice on samples Vendor-Utility Interactions Working relationships between the two parties, including developing work packages, documentation requirements, expectations for reporting indications, role played by vendor (sometimes as in-house NDE planner), etc.

Industry Challenges Work force availability, work opportunities 5-2

The primary focus of the human factors research has been to understand the interaction and impact of these PIFs upon examinations. The next section describes the consolidated results of our interviews with 37 NDE examiners (comprising Level II and Level III qualified examiners employed by vendors and utilities). More detailed descriptions can be found in the original research reports (Sanquist and Harrison 2021; Sanquist et al. 2018).

5-3

6 NDE FUNCTIONAL TASK STRUCTURE AND PERFORMANCE INFLUENCES The process of performing NDE is represented as a series of high-level functions, tasks, and subtasks that occur over a period of time. The high-level functional structure is shown in Figure 6-1. This structure is based on integration of procedure documents and interview results. The time span represented in the figure covers the years and months of planning for inspection, leading up to more immediate preparation, examination, data interpretation, and reporting.

DATA REPORT PREPARE FOR CONDUCT PLAN INSPECTION INTERPRETATION EXAMINATION EXAMINATION EXAMINATION AND EVALUATION RESULTS Figure 6-1 General Functional Structure of NDE Tasks Each function comprises multiple tasks (See Sanquist and Harrison (2021) and Sanquist et al. (2018) for further details)

The functional flow of inspection activities depicted in Figure 6-1 incorporates a wide time window. Planning generally occurs months to years in advance of conducting the examination.

Preparation for the examination takes place from when the NDE team arrives on site (one to two weeks prior to a specific examination) to when all pre-job briefs have occurred, and the components are ready to be inspected. Conducting the examination occurs once the examiners have located and verified the components for examination. Data interpretation and evaluation occurs after the exam is conducted for encoded procedures, while for manual UT it occurs at the same time as data collection (conducting exam).1 Reporting occurs after the examination is completed, resulting in an inspection report document. The following paragraphs report high-level conclusions from the analysis of examiner interview data (Sanquist and Harrison 2021; Sanquist et al. 2018).

Planning Examination: For the Planning Examination function, there was agreement across interviewees that developing the inspection requirements properly was highly important.

Without proper requirements at the beginning, the remaining functions and tasks would be of little value. Elaboration of this point indicated that knowledge of operating experience, good plant and component drawings, specification of equipment to ensure availability, and well-structured work packages were important in specifying inspection requirements for the vendor.

It was also stressed that getting experienced people who work well together was a key aspect, and that sufficient lead time to accomplish planning led to better definition of the inspection requirements.

The performance influencing factors most affecting the Planning Examination function appeared to reflect a perspective from inspection vendors who had encountered surprises at the 1

There may be some non-real time interpretation associated with manual UT when additional test results are needed for evaluating flaw indications that are seen during the examination.

6-1

inspection site. Examples included missing data in the work package (utility planning) and obstructions that were not noted (accessibility of component). The utility NDE planner perspective was reflected in comments indicating a desire to focus the examinations in a limited area (radiation exposure), and to get examiners with knowledge of the plant and component (knowledge and experience), which is sometimes not possible due to schedule compression (time pressure).

Preparing for Examination: The Preparing for Examination function showed general consensus that the pre-job briefing and equipment calibration were the most important tasks.

Missed information in the briefing or erroneous calibration would have considerable downstream consequences on conducting the examination. Of the two tasks, it appeared from the interviews that pre-job briefings were considered more important because calibration is more of a skill-based procedure under the control of the individual examiner. Pre-job briefings, however, are affected by numerous other sources of variance: the skill and experience of the site NDE lead responsible for the briefing, the quality of the planning documents leading up to the pre-job briefing, and the overall pace of work established by the site.

Discussions of performance influencing factors within the Preparing for Examination function showed more variability than the Planning Function. Numerous factors were mentioned, including erroneous planning information, time pressure, complex procedures, and lack of experienced personnel. The influencing circumstance that tended to recur in this portion of the interview was the examiner finding a different configuration at the component than was described in the pre-job briefing, and presumably what they had prepared for in terms of equipment and calibration. This result shows the inherent difficulty in trying to isolate important or critical tasks and/or factors in the ongoing flow of sequential work; poor or erroneous information in the planning process is perpetuated into the pre-job briefing, which can then affect circumstances at the exam location.

Conducting Examination: In the Conducting Examination function, there was complete consensus that verifying conditions at the exam location was the most important task. This finding reflects earlier comments that erroneous or incomplete planning information can adversely affect the quality of the examination. The finding also reflects a focus on sources of variability that are outside the examiners direct control. Performing the manual UT scan by manipulating the probe and interpreting signals is a skill-based activity, dependent on the examiner. Conditions at the exam location and their descriptions in the work package are based on numerous utility planning inputs, which the examiner should ensure are correct prior to conducting the scan.

The performance influencing factors mentioned most often for the Conducting Examination function were the weld profile or condition not being as expected, the physical environment in which the scan is performed, and general issues of component access. Differences in planned versus actual weld profile can potentially disrupt a UT examination due to the need for better surface preparation. Alternatively, limited examinations or use of different probes may be warranted. The ability to access the component can affect both examination coverage and the complexity of conducting the scanfor instance, for manual exams, how the body is positioned, how and when notes are taken, and how examiners share scanning subtasks.

Data Interpretation and Evaluation: This function is specific to encoded UT examinations, since the data are generally reviewed after a scan is completed, in an area outside of the location where the examination was performed. There was general agreement that a separate, dedicated physical space for reviewing and interpreting data was very important to reduce 6-2

distractions and interruptions that could lead to missing important information in the complex displays. Further, several examiners indicated that they made it a practice to start over with data evaluation if interruptions of their analysis process occurred. One vendor specifically implemented independent double review of data. There was also discussion of comparing current data with all past results rather than just comparing it to the most recent exam to discern potential growth of subtle flaws; this practice could help mitigate the cognitive bias of accepting previous results uncritically. However, there was no standard practice described across vendors.

Reporting Examination Results: In the Reporting Examination Results function, the task most frequently mentioned as important was the debriefing between the utility representatives and the NDE team. This interacts with the organizational culture factor, so these will be discussed together. The qualitative aspect of reporting involves who is debriefed and when. Vendor interviewees indicated that they had the practice of discussing potential indications among their own team, initially, prior to engaging the utility NDE leads. This can vary from site to site, depending on working relationships and organizational culture. Procedurally, there was consensus that the flaw characterization flowcharts in specific examination procedures are very useful tools for evaluating indications and provide a structure for subsequent decisions, e.g.,

whether to request radiography images or to have another team perform a scan. The utility NDE lead perspective on Reporting Examination Results indicated that they want to be involved in reviewing any indications and potentially perform a re-scan themselves. It was also clear that the utility NDE leads want to closely manage distribution of information about indications until there is some diagnostic clarity. This is for purposes of limiting the spread of erroneous information (rumor control) and avoiding intervention from the outage control center when it may be unnecessary.

6-3

7 NDE OUTCOME/ERROR AND CONSEQUENCE MODELING The event descriptions and performance influence data provided above show that human factors issues can occur at multiple points in time through the planning, preparing, conducting, interpretation, and reporting functions of UT exams. This suggests that a broader focus on human performance across the time span and multiple functions within the examination process can help to identify areas that can lead to failures to detect, misinterpretation of indications, false calls, limited coverage, and other problems. Signal detection and interpretation occurs in a larger context, which is error-prone as well. A selective analysis of these potential vulnerabilities and potential consequences based on the interview results and other technical documentation is discussed in Sanquist et al. (2018) and Sanquist and Harrison (2021) and presented below in Table 7-1.

The task analysis and examiner interviews provided the basis for identifying potential errors, precipitating factors, and consequences within each functional task in the UT exam process (Plan, Prepare, Conduct, Interpret, and Report). The inputs to the following error analysis were also supported by operating experience. An incorrect component type was used at Ginna, procedure steps were not performed at Point Beach, incomplete examination coverage occurred at Duane Arnold, and multiple problems were combined at North Anna. Overreliance on previous analysis occurred at Palisades for an encoded exam.

Table 7-1 Potential Outcome/Error Types, Precipitating Factors, and Consequences for Manual and Encoded UT Functions Selective Precipitating Factors Outcome/Error Type Potential Consequences Planning Examination Plant drawings wrong Incorrect inspection requirements Flaw undetected, false positive on wrong component, need to re-do exam on proper component, additional examiner dose Poor communication with Component to be inspected not Increased examiner workload, stress, craft specialties properly prepared fatigue; time pressure Wrong requirements, poor Inspection vendor with wrong training Increased examiner workload, stress, communication with vendor and certifications fatigue; time pressure Overscheduling to reduce Schedule conflicts with other Exam delayed, not performed, or time outage length maintenance procedures pressure upon examiner Preparing for Examination Time pressure, incomplete Incomplete or erroneous pre-job Conditions or procedure execution not or inaccurate information briefing as expected used in planning Incorrect file recalled on Need to repeat calibration Increased examiner workload, stress, instrument, new shift, fatigue; time pressure temperature change Memory lapse, incomplete Material left behind (e.g., probes, Need to exit exam area to retrieve pre-job briefing etc.) necessary items, exam delayed, time pressure 7-1

Table 7-1 Potential Outcome/Error Types, Precipitating Factors, and Consequences for Manual and Encoded UT Functions (cont.)

Selective Precipitating Factors Outcome/Error Type Potential Consequences Overscheduling other work Excessive wait time to enter Inefficient use of examiner time, in same area; many other controlled area fatigue, delay of other procedures crew in health physics area Analyst does not confirm Incomplete data due to loss of Flaw may not appear in data and will acceptable data and probe couplant or probe liftoff in small not be detected contact in pre-scan areas Reversing index and scan Data image out of phase Locations of indications incorrect and axes in setup misinterpreted Skewed encoder mounting Response characteristics may be Missed flaw detection present outside of the expected region; incomplete coverage Conducting Examination Work package error, wrong Wrong component examined Flaw undetected, false positive on drawings, prepped wrong wrong component, need to re-do component exam on proper component, additional examiner dose Surface conditions poor, More time required at pipe than Stop exam, re-surface, do exam with undocumented accessibility planned limitations, additional dose, time problems pressure Obstructions, surface prep Incomplete exam coverage Flaw undetected Noise, heat, visibility Erroneous data recording Missed indications, erroneous documentation Informational use procedure Procedure steps left out Failure to execute procedure as written, need to re-do exam Data Interpretation and Evaluation Analyst does not confirm Incomplete data due to loss of Flaw may not appear in data and will 100% acceptable data and couplant or probe liftoff in small not be detected probe contact areas Analyst does not follow all Systematically dismissing an Flaw mischaracterized as geometry or procedural steps to indication as geometry without a another anomaly evaluate UT image complete evaluation response Overreliance on previous Analyst accepts previous Flaw mischaracterized as geometry or data analysis interpretation of data another anomaly Interruptions or distraction Less attention devoted to data upon Missed flaw detection during data interpretation resumption of task Inspection gate selection Response characteristics may Missed flaw detection too tight present associated responses outside of the expected region Reporting Examination Large number of welds Documentation inaccurate or Critical information about conditions examined before incomplete left out of report documentation, inadequate note taking, poor team coordination at weld 7-2

Table 7-1 Potential Outcome/Error Types, Precipitating Factors, and Consequences for Manual and Encoded UT Functions (cont.)

Selective Precipitating Factors Outcome/Error Type Potential Consequences Varying formats across Documentation not to plant Information left out of report, re-work utilities, varying coverage standards of reports, need to re-do exam calculator methods, examiners unfamiliar with plant expectations Premature communication Plant personnel escalate unimportant Unnecessary oversight from outage with non-NDE personnel finding control center prior to full evaluation Lack of post-job briefing, Failure to escalate finding of Time pressure, failure to properly inadequate communication potential flaw in timely manner characterize indication between vendor and utility The error analysis presented in this section is meant to illustrate an approach that might be applied more comprehensively across a range of NDE techniques to systematically identify types of organizational and operational errors. The lists of error types, precipitating factors, and consequences are not exhaustive, and at this point should be considered as demonstrating an analytic approach that could be extended. Such an extension would need to involve subject matter experts in the specific NDE techniques of concern and should also focus on identifying potential mitigations.

7-3

8 NDE AND THE LATENT FAILURE MODEL The NDE incidents and events described above and the outcome/error/consequence analysis clearly portray the interaction between system-level organizational factors, such as work scheduling or standard vendor practice, with person-level performance of individual examiners.

This section discusses contemporary thinking about complex process incidents, events, and failures, and shows how NDE can be viewed in this context.

Various types of error have been the topic of considerable study in human factors engineering for many years (Fitts and Jones 1947; Reason 1990; Reason and Hobbs 2003). Early focus on this topic addressed error at the level of task executionas in making wrong control selections, misreading indicators, or skipping steps in procedures. Assessment of non-detection errors in industrial inspection tasks (Reason and Hobbs 2003) suggest the following classes of error that are focused at the level of task execution:

• Inspection interrupted before reaching defect

• Inspection completed but examiner was distracted, preoccupied, tired, or in a hurry

• Examiner did not expect to find problem in that location

• One defect located, but adjacent one missed

• Poor lighting, dirt, grease.

Additionally, Reason and Hobbs (2003) identify two common sources of error that can cut across multiple tasks:

• Inadequate rest breaks

• Unsatisfactory access to job location.

These latter elements reflect how organizational systems can create conditions that contribute to problems at the level of individual task performance. Both of these factors were mentioned in the examiner interviews regarding organizational factors that influence performance.

The most significant theoretical work concerning performance error is that of Reason (1990). A key element of Reasons theoretical approach is the linkage between the human operator and organizational error. The basic concept is that human operators will have lapses of memory or attention2 and latent errors within the organizational structure can lead to or exacerbate errors at the individual level.3 A conceptual view of the interaction between layers of defense, system, and person-level variables is shown in Figure 8-1. This figure captures how the layers of defense in NDE are manifested over time, with some processes such as ASME Code implementation and inspection planning taking place over a multiple year period; more immediate person-level influences occur close to or at the time of examination.

2 The speed and accuracy of memory and attention are influenced by individual, task, and organizational variables documented in the basic human performance research literature.

3 An example of an organizational variable that is a latent error would be the clustering of inspections during the spring and fall seasons, which leads to time pressure, competition for resources, and lack of work for examiners during other seasons.

8-1

Figure 8-1 Organizational and Operational Defenses and Error-Inducing Factors UT Inspection and Influences over Time The interaction between latent organizational factors and their direct impacts upon the examiner is well-captured by Reason and Hobbs (2003) in the following quote:

Latent conditions arise from the strategic decisions made by designers, manufacturers, regulators and top-level managers. These decisions relate to goal setting, scheduling, budgeting, policy, standards, the provision of tools and equipment, and the like. Each of these decisions is likely to have some adverse consequences for some part of the system (under-manning, shortage of resources and so on).

Within the workplace, the local effects of these decisions turn into error- and violation-producing conditions such as time pressure, high workload, the wrong tools, inadequate skills and experience, and so on. These local factors, in turn, interact with human psychology to cause unsafe acts, or active failureserrors and violations that have a direct impact upon the system. Such unsafe acts can penetrate some or all of the layers of defence. (Reason and Hobbs, 2003, p.77-78).

As shown in Figure 8-2, various system-level factors such as time pressure, work schedule, and training directly influence the cognitive processes that examiners use during the collection and interpretation of examination data.

8-2

Figure 8-2 Examiner Cognitive Processes Are Affected by Local Conditions That Can Increase the Likelihood of Error (Local factors are shown in the two columns as input to examiner cognition.) Figure is based on Reason and Hobbs (2003).

The basic cognitive model portrayed in Figure 8-2 is supported by extensive human factors and cognitive psychology research. The general process is that rapid and often unconscious cognitive processes, such as attention, perception, and memory, are influenced by external events and circumstances; thus, there is ongoing variability in human mental processes that are only partly under individual control.

8-3

9 NDE ERROR REDUCTION APPROACHES The complex sociotechnical system comprising NDE has many sources of variability that can either enhance or degrade examination outcomes. This may encompass organizational elements, such as organizational culture and the tendency of some outage control centers to exert schedule pressures on inspection teams, to more individual fluctuations in examiner perceptual attention resulting from distractions, fatigue, or stress. It is important to recognize that variability is an inherent part of all systems, and thus attempts to eliminate all error will not succeed. However, it is possible to understand how variations in NDE circumstances at the system and person levels occur and interact, and thus be able to reduce vulnerabilities and the likelihood of error. Humans are generally poor at being able to predict when and where specific errors will occur but good at understanding the circumstances that can either lead to or reduce errors.

This section discusses mitigation approaches for various kinds of situations; the mitigations are based on human factors research that has demonstrated their utility in other complex circumstances, such as transportation, medicine, and process control (Reason and Hobbs 2003; Strauch 2017; Woods et al. 2010), and upon the findings of the NDE-specific human factors research performed in this NRC program (D'Agostino et al. 2017; Sanquist 2020; Sanquist and Harrison 2021; Sanquist et al. 2018). We divide the mitigations into two general categoriesthose that are implemented at the system/organization level and primarily affect processes and procedures of the utility and vendor, and person-level approaches that have their impact closer to the examiner and task performance. It should be noted that these categories are not mutually exclusive and that system-level mitigations can also directly affect human performance at the person-level; for example, person-level impacts such as reducing distractions or fatigue can be achieved by organizational changes in communications and scheduling.

9.1 System-level Mitigations System-level error reduction involves changes in organizational processes and procedures upstream from the specific tasks of component examination. The ownership of system processes can involve industry organizations, regulatory agencies, utilities, specific plants, and NDE vendors. Sometimes multiple stakeholders are engaged in system-level processes, such as ASME Code development and modification. The following specific mitigations are based on organizational performance influencing factors identified in examiner interviews and approaches that have been successful in other industries and applications.

• Information Accuracy - Ensuring that inspection-related information is correct can enhance the examinations team confidence in inspection requirements, plant drawings, and component locations and labeling. Inaccuracies in these fundamental elements leads to increased time to ensure that the examination location is correct and appropriate tools are assembled for the work.

• Communications and Preparation - Proper preparation of the examination site was frequently mentioned during interviews as a necessary and desirable element for obtaining sufficient coverage and reducing unnecessary time in the controlled area. In the worst case, craft specialties have failed to remove insulation or provide scaffolding to reach the component. Communication with craft specialties to ensure appropriate site preparation can reduce these problems.

9-1

• Inspection Scheduling - In the planning process for inspection, utilities, and plant NDE personnel should try to budget sufficient time for preparation, examination, and review of findings. Compressed inspection schedules can lead to narrow time windows for task completion, which can increase explicit or implicit time pressure and increase vulnerability to errors. Compressed schedules can also place additional burdens on the vendor inspection crew, such as working longer hours or during irregular times, which can in turn lead to fatigue.

• Data Review and Interpretation Process - Implement standard independent review of examination results to confirm findings that are made by a single examiner/reviewer.

This might consist of review by other vendor staff, or by plant NDE personnel. Selective comparison of current exam results with more than just the immediate prior exam history can be useful in identifying subtle flaws that have grown but would not be detectable without a longer look-back period.

• Skill Development - ASME Code should realistically address training and experience requirements, in terms of classroom, laboratory and field experience. Expertise takes many years to develop, and increased opportunities for targeted training and practice appear warranted. For instance, targeted training to overcome the cognitive bias of only considering outside diameter flaws, which occurred in the Palisades event.

Continual review of standard knowledge and practice is warranted, given plant aging and the potential for new failure mechanisms that have not been previously seen.

• Change Management and Evaluation - It is a common experience that introducing new technologies, processes, and procedures with the intention of reducing one type of human factors problem generally results in new, unanticipated issues. Any changes proposed in ASME Code, qualification testing procedures, or utility and plant processes should be based upon a detailed human factors technical basis, including appropriate task and error analysis using techniques described in this NUREG/CR.

9.2 Person-level Mitigations Person-level error reduction mitigations are aimed at creating conditions that are more conducive to reliable attentional, perceptual, and decision-making processes in the individuals conducting and interpreting exams. A variety of contextual factors surrounding the examination process can create various stressors, which potentially degrade the examiners cognitive processes. By taking steps to control these influences, such stressors can be reduced and create conditions more conducive for focused attention, perception, and decision-making.

• Shift Scheduling - Long shifts and round-the-clock work is common in NDE. Research indicates that fatigue develops with longer time-on-task, and particularly during night work. Interventions for this problem entail taking breaks during long, monotonous tasks, and ensuring that schedules use forward rotation if crews are to be varied on shifts over a long outage (e.g., moving from day to afternoon to night). Alternatively, selection of personnel who experience fewer difficulties during night work would be appropriate if such individuals can be identified.

• Reduce Distraction and Time Pressure Stress - Maintaining focused attention on sensitive tasks is a requirement of many jobs and everyday activities, and it is particularly important in tasks involving visual search for subtle signals that are infrequentone of the basic tasks of NDE. Any circumstances that reduce an examiners focused attention while scanning and reviewing data can potentially reduce performance accuracy. Distraction and time pressure can result from too-frequent communication with health physics and NDE oversight personnel during scanning as 9-2

well as other staff in the workspace used for data review and interpretation.

Distractions and time pressure stress should be minimized by reducing communication with the field examination team to essential interactions and limiting frequent reminders of stay time limits. Analysts should be provided with dedicated space for review and interpretation of data that is separate from other staff members and sufficient time to ensure accurate work.

• Skill Development - The voluminous literature concerning expertise shows that experience is essential for building domain-specific knowledge, and that there is no upper bound on how much experience is appropriate; experts keep improving. More junior personnel should be provided with sufficient opportunities to gain field experience, implementing a range of exams appropriate to their qualification level. This can be gained by on-the-job training, use of practice samples, and potentially through perceptual learning with simulators. These approaches provide critical feedback to trainees.

• Exam Team Composition -Teams should be assembled of members who are experienced working with each other. Having teams whose members are experienced working with each other provides a level of group communication and cohesion that can facilitate shared awareness when conducting exams. Junior team members can anticipate requirements without being specifically instructed and can serve as valuable backup.

• Cognitive Aiding - The one study of decision aiding available for NDE (Harris 1992) suggests that a simple checklist that reminds examiners of key aspects of the A-scan such as rise time, the signal walk, and persistence with probe skewcan facilitate flaw detection with subtle indications. Often, this type of more focused analysis is built into procedure flowcharts for characterizing flaws.

• Complete Read of Data Without Interruption - This practice for encoded UT was identified by one respondent as their individual standard procedure in order to ensure that they devoted full attention to a set of data. If an interruption happened, they started over. Generally, respondents reported being able to review data from an entire encoded scan within 90 minutes. The medical image interpretation literature (reviewed in Sanquist and Harrison (2021)) suggests that physicians who are interrupted in the midst of reading an image tend to spend less time on that image when they resume, resulting in more discrepancies occur. Completing a full reading of the dataset without distraction can reduce this potential.

9.3 Error Management Principles The foregoing discussion and analysis shows that error is an inherent part of complex systems and that, while mitigations can be implemented to reduce the overall likelihood of undesirable NDE outcomes, such problems can never be entirely eliminated. Our brief review of the latent error model and the application of this concept to NDE reflects current thinking in human factors, i.e.,

the label error should be the starting point for investigation of the dynamic interplay of larger system and contextual factors that shaped the evolution of the incident (Woods et al. 2010).

In other words, what is labeled as an error is really a consequence, rather than a cause.

Reason and Hobbs (2003) have formulated several error management principles that apply to virtually all circumstances of people working in complex systems. The following are several of the Reason and Hobbs error management principles with examples of how they relate to NDE:

9-3

1. Human error is both universal and inevitable - Variability exists in human performance, whether in rapid unconscious cognitive processes such as attention and perception, or in more overt activity such as communication and process implementation. In NDE, this is illustrated by the high level of performance variability among experienced examiners in round robin studies (Heasler et al. 2003; Meyer et al. 2019; Wheeler et al. 1986). Errors will continue to occur, but conditions can be changed or monitored to reduce their likelihood and severity.

2. Errors are not intrinsically bad - Errors do present learning opportunities and the potential for change. For example, the missed flaw at Palisades was partially due to flawed assumptions regarding inner diameter cracking and a data review look-back period that was too short. These issues are likely present across the NDE enterprise and can be changed.

3. You cannot change the human condition, but you can change the conditions in which humans work - This principle is a corollary to Principle #1: human attention, perceptual, and memory capabilities and limitations are relatively fixed, but process and procedural changes can be made to accommodate them; knowledge can be increased by training and experience. This includes identifying potential error traps and altering the situation through controllable means. For example, the Shearon Harris event shows the potential utility of dedicated analysis space and independent review of findings.

4. The best people can make the worst mistakes - Errors are not limited to junior or inexperienced personnel; they can occur at all levels of experience and knowledge. This is why it is important to focus on situational conditions and redundant defenses. This principle is illustrated by the Palisades event in which experienced examiners attributed a flaw to geometry, which was subsequently accepted by other analysts over a period of years.

5. Errors are consequences rather than causes - Errors are often the result of work-related circumstances, so analysis of the conditions leading to the error can help to change conditions to limit or reduce their potential recurrence. Examples of this include distraction resulting from interruptions during data review and too much input from health physics during key parts of examinations.

6. Many errors fall into recurrent patterns - Event and incident analyses reveal that errors tend to happen for predictable reasons, such as knowledge gaps, fatigue, time pressure, etc. Focusing on reducing these and similar controllable elements can reduce error likelihood. Examination time pressure can lead to missing flaws due to fatigue, and distraction can disrupt focused attention during data analysisboth of these factors were repeatedly mentioned by examiners.

7. Safety-significant errors can occur at all levels of the system - Errors are not restricted to the task execution end of work activities, but can occur throughout the complex system required to accomplish NDE, from examiner, to supervisor, to vendor and utility management, to code and standard organizations. This is shown by the event at Ginna in which the ISI list had been incorrectly populated by site staff, leading to use of the wrong procedure. It is generally true that the higher the level of the error, the more consequential it can be.

8. Effective error management aims at continuous reform rather than local fixes -

Systemic change and continuous monitoring are more effective long-term approaches than patches that address a localized problem. Examples are provided by activities such as those conducted by the Nuclear Industry Focus Group, which addressed DM weld exams (EPRI 2013); and a variety of other efforts aimed at improving NDE reliability (EPRI 2016).

These principles are meant to provide a guiding philosophy to the overall organizational approach to addressing NDE reliability and over time can be implemented within and across the multiple entities comprising NDE/ISI operations.

9-4

10

SUMMARY

AND CONCLUSIONS It has been recognized for many decades that human factors is a key element affecting the outcome of NDE. Most of the focus on the human element has been on performance demonstration testing to establish qualified procedures and personnel. Despite improved inspection outcomes resulting from this approach, a variety of suboptimal NDE outcomes have been seen in the past 10 years that have clearly had a human factors component. The NRC initiated research in 2016 to more fully understand the factors affecting human performance in NDE; this report has summarized and synthesized the findings from that research.

The principal findings of the research are as follows:

• NDE is affected by a range of factors encompassed by a sociotechnical systems model, incorporating organizational, environmental, task, individual, and group variables.

• NDE error can be investigated in a structured way by means of task analysis, which entails developing functional descriptions of the tasks that occur over an extended period of time, from planning through reporting.

• Interviews have identified key performance influencing factors that affect each of the NDE functional phases and specific tasks, with particular emphasis on organizational and team variables and examiner and team knowledge.

• NDE can be represented as an organizational system with multiple layers of defense, and error modeling can be used to identify specific error-precipitating factors and consequences.

• A number of specific error mitigation approaches were identified and discussed, as well as broad error management strategies.

• It is generally recognized that analysis and remediation of undesirable NDE outcomes needs to focus on changing the conditions of work rather than fixing the worker.

Addressing human factors in NDE is part of the wider-scope effort of continuous quality improvement and system vulnerability analysis. By framing NDE as a complex operational system affected by a range of interacting variables, an enriched understanding of the conditions affecting examiner performance can be achieved. As a result, a variety of targeted and systemic interventions can be implemented to reduce the overall likelihood of error.

10-1

11 REFERENCES Anderson, M T, A A Diaz, and S R Doctor. 2012. PNNL-21546. Evaluation of Manual Ultrasonic Examinations Applied to Detect Flaws in Primary System Dissimilar Metal Welds at North Anna Power Station. Pacific Northwest National Laboratory. Richland, Washington.

D'Agostino, A, S Morrow, C Franklin, and N Hughes. 2017. Review of Human Factors Research in Nondestructive Examination. U.S. Nuclear Regulatory Commission. Washington, D.C.: ADAMS Accession No. ML17059D745.

Doctor, S R, S E Cumblidge, T T Taylor, and M T Anderson. 2013. NUREG/CR-7165; PNNL-19014, Rev. 2. The Technical Basis Supporting ASME Code,Section XI, Appendix VIII:

Performance Demonstration for Ultrasonic Examination Systems. U.S. Nuclear Regulatory Commission. Washington, D.C.: ML13144A107.

EPRI. 1988. EPRI Report NP-6052. Human Performance in Nondestructive Inspections and Functional Tests. Electric Power Research Institute (EPRI). Palo Alto, California.

EPRI. 2008. EPRI Report 1016969. Nondestructive Evaluation: A Review of NDE Performance Demonstrations - NDE Round Robin Report. Electric Power Research Institute. Palo Alto, California.

EPRI. 2013. EPRI Report 3002000041. Nondestructive Evaluation Improvement Focus Group Extent of Condition Actions in Response to North Anna Dissimilar Metal Weld Operating Experience. Electric Power Research Institute. Palo Alto, California.

EPRI. 2016. EPRI Report 3002009080. Nondestructive Evaluation: NDE Reliability Update: A Guide for the Review and Implementation of Recent U.S. Nuclear Power Industry NDE Reliability Guidelines and Reports. Electric Power Research Institute. Palo Alto, California.

Fitts, P M, and R E Jones. 1947. Memorandum No. TSEAA-694-12. Analysis of Factors Contributing to 460 "Pilot-Error" Experiences in Operating Aircraft Controls. Army Air Forces Headquarters, Air Material Command, Engineering Division.

Harris, DH. 1992. TR-100412. Effect of decision making on ultrasonic examination performance.

Electric Power Research Institute. Palo Alto, California.

Heasler, P G, S R Doctor, and D A Jackson. 2003. NUREG/CR-6795 PNNL-13873. A Comparison of Three Round Robin Studies on ISI Reliability of Wrought Stainless Steel Piping.

Meyer, R M, A E Holmes, and B Lin. 2019. PNNL-SA-140496. Analysis of Empirical Probability of Detection Data for Dissimilar Metal Welds.

NRC. 2013. Shearon Harris Nuclear Plant - NRC Special Inspection. Report 05000400/2012010. U.S. Nuclear Regulatory Commission. ML13192A154.

NRC. 2016. White Paper on Improving the Effectiveness and Reliability of Nondestructive Examination. U.S. Nuclear Regulatory Commission. Washington, D.C.: ML16306A354.

NRC. 2019. Nuclear Regulatory Commission Vendor Inspection Report of Framatome Inc. No.

99901300/2019-201. U.S. Nuclear Regulatory Commission. ML19261A188.

PNNL. 2007. Technical Evaluation Report, Review of Ultrasonic Data for Flaws Detected at Duane Arnold Energy Center, Unit 1, JCN-J3218, Task 6, TAC MD4630. Richland, Washington. Pacific Northwest National Laboratory: ML071760259.

Reason, J. 1990. Human Error. Cambridge, UK: Cambridge University Press.

Reason, J. 2000. "Human error: models and management." BMJ, no. 320: 768-770.

Reason, J, and A Hobbs. 2003. Managing Maintenance Error: A Practical Guide. Boca Raton, Florida: CRC Press.

11-1

Sanquist, T F. 2020. PNNL-29761. Nondestructive Examination (NDE) Training and Qualifications: Implications of Research on Human Learning and Memory, Instruction and Expertise. ML20079E343.

Sanquist, T F, and J Harrison. 2021. PNNL-31245. Human Factors of Encoded Ultrasonic Examinations in Nuclear Power Plants. ML21124A141.

Sanquist, T F, S Morrow, A D'Agostino, N Hughes, and C Franklin. 2018. PNNL-27441. Human Factors in Nondestructive Examination: Manual Ultrasonic Testing Task Analysis and Field Research. NRC Adams ML18176A055.

Spanner, J C, R W Badalamente, W L Rankin, and T J Triggs. 1986. NUREG/CR-4436, PNL-5641, BHARC-400/85/016, Vol. 1. Human Reliability Impact on Inservice Inspection, Phase 1 Summary Report. U.S. Nuclear Regulatory Commission. Washington, D.C.

Strauch, B. 2017. Investigating Human Error: Incidents, Accidents, and Complex Systems. Boca Raton, Florida: CRC Press, Taylor & Francis Group.

Wheeler, W A, W L Rankin, R W Badalamente, and J C Spanner. 1986. NUREG/CR-4600, PNL-5757. Human Factors Study Conducted in Conjunction with a Mini-Round Robin Assessment of Ultrasonic Technician Performance. Pacific Northwest National Laboratory. Richland, WA.

Woods, D, S Dekker, R Cook, L Johanssesen, and N Sarter. 2010. Behind Human Error. Boca Raton, Florida: CRC Press, Taylor & Francis Group.

11-2

NUREG/CR-7295 PNNL-32505 Human Factors in Nondestructive Examination March 2022 T Sanquist, Pacific Northwest National Laboratory Technical S Morrow, U.S. Nuclear Regulatory Commission J Harrison, Pacific Northwest National Laboratory C Nove, U.S. Nuclear Regulatory Commission Pacific Northwest National Laboratory 902 Battelle Blvd, Richland, WA 99354 Division of Engineering Office of Nuclear Regulatory Research U.S. Nuclear Regulatory Commission Washington, D.C. 20555-0001 This report summarizes key findings from a five-year research program investigating the different ways human factors influence nondestructive examination (NDE) outcomes. The research team reviewed specific incidents and events involving manual and encoded ultrasonic testing and conducted detailed interview studies with NDE examiners. The results from these studies reveal a range of organizational, group, environmental, task, and individual factors that combine to influence NDE reliability. The research team integrated these findings with contemporary concepts of latent organizational error to illustrate how multiple interacting factors can lead to suboptimal outcomes when performing qualified examinations. This information provides a technical basis for understanding the precipitating factors that can lead to human error in NDE and various system-level and person-level interventions that can mitigate errors. System-level mitigations include organizational approaches to information accuracy, inspection scheduling, data review and interpretation, workforce skill development, and change management. Person-level mitigations include shift scheduling to avoid fatigue, reducing distraction and time pressure, individual skill development through training and practice, knowledgeable team composition, cognitive aiding, and individual data review processes. Human error is a systems problem, which can be reduced by various interventions to manage the inherent variability in organizational processes and cognitive functions at the individual level.

Nondestructive examination (NDE), human factors, human performance, ultrasonic testing, NDE reliability, performance demonstration, performance influencing factors

NUREG/CR-7295 Human Factors in Nondestructive Examination March 2022 PNNL-32505