ML18012A719
| ML18012A719 | |
| Person / Time | |
|---|---|
| Issue date: | 01/12/2018 |
| From: | D'Agostino A, Carmen Franklin, Niav Hughes, Stephanie Morrow, Carol Nove, Sanquist T Office of Nuclear Reactor Regulation, Pacific Northwest National Laboratory |
| To: | |
| Rezai A, NRR-DMLR 415-1328 | |
| Shared Package | |
| ML18002A510 | List: |
| References | |
| PNNL-SA-131498 | |
| Download: ML18012A719 (16) | |
Text
Exploring the Effects of Human Factors on Manual Ultrasonic Nondestructive Examination Tom Sanquist*, Stephanie Morrow**,
Amy DAgostino**, Niav Hughes**, Carmen Franklin**
- PNNL
- Nuclear Regulatory Commission Carol Nove - NRC COR PNNL-SA-131498
Research Goals Systematically evaluate the human factors that can affect ultrasonic testing (UT) examiners.
Understand the key differences between qualification performed in a laboratory environment vs. performance during field examinations.
Determine the highest priority human factors that may be impacting UT NDE performance in the field.
=
Background===
Nondestructive examination (NDE) is a critical maintenance function in nuclear power plants.
Across many industries, maintenance errors comprise the largest proportion of human factors-related operational problems (as distinct from operator error in system control).
It is now widely recognized in safety-sensitive complex systems, that error has its roots in the entire organization - not just a single individual performing an isolated task.
The human factors aspects of NDE, and ultrasonic testing in particular, has been studied primarily through the lens of round-robin performance testing and POD- the human-machine model.
The resulting focus on examiner performance has led to institutionalized practices:
ASME Code Requirements for Performance Demonstration (PD)
Specific procedure qualification Refresher practice These practices have resulted in significant progress in improving NDE reliability over the past two decades, but there may be additional opportunities to address a broader range of human factors that can impact NDE in the nuclear industry.
3
Back to the Future: The Need for a Systematic View of Human Factors in NDE Although there is an increasing awareness of the importance of human performance in NDT, no systematic evaluations of the variables likely to influence the performance of technicians and/or the man-machine system have been conducted to date. (Spanner, et al.,
1986/NUREG/CR-4436)
Relatively little work has been done over the past 30 years on human factors in NDE. Most research has focused on quantifying probability of detection (POD) in blind performance testing on samples, with less consideration of other factors that can influence outcomes in operational settings.
Our working principle: The reality of the examination process is that numerous people and organizations are involved over a considerable time period, and research is needed to better understand these performance influencing factors.
4
Cascading Effects of Errors The prevailing focus of the few human factors studies done between 1986 and now has been the examiner at the component - and the corresponding POD measures Previous research suggests 42-65% of human performance problems in nuclear power plant events are attributable to maintenance, calibration, and testing These maintenance errors tend to result from cascading effects of organizational issues, local condition variations, with impacts on task performance 5
Research Approach Part 1: Topic Characterization and Literature Review Part 2: Task Analysis and Field Study Part 3: Analysis and Prioritization 6
Sociotechnical Systems Model Organizational Factors Physical Environment Group Characteristics Individual Differences Task Characteristics Adapted from Neville Moray (2000) Culture, politics and ergonomics, Ergonomics, 43:7 7
Performance Influencing Factors Task Characteristics
- PreJob Preparation
- Equipment
- Procedure
- Time Pressure
- Task Complexity Organizational Factors Individual Differences
- Utility Planning
- Knowledge/Experience
- Organizational Culture
- Motivation/Attitude/Personality
- Supervision
- Physical Abilities
- Training
- Vendor/Utility Interactions
- Industry Challenges NDE
- Cognitive Factors
- Examiner Process Performance
- Workload/Stress/Fatigue Physical Environment Group Characteristics
- Accessibility of Component
- Team Coordination
- Temperature/Humidity
- Team Cohesion
- Radiation
- Noise
- Lighting 8
Research Status Part 1: Characterization and Literature Review - COMPLETE Literature Review report is publicly available at ADAMS Accession Number ML17059D745 (https://www.nrc.gov/docs/ML1705/ML17059D745.pdf)
Part 2: Task Analysis and Field Study - IN PROGRESS Part 3: Analysis and Prioritization - PLANNING STAGE 9
Task Analysis and Field Study: Methods and Participants Developed list of functions comprising manual UT from procedure reviews, prior studies of UT, and SME input Expanded functions to specific tasks and sub-tasks Developed structured interview protocol Completed analysis at EPRI to validate and expand task list with 11 subject matter experts Visited 3 nuclear power plants to observe UT NDE in the field Solicited subject matter expert participation via email Conducted face-to-face interviews with 9 subject matter experts Conducted two focus groups with 21 subject matter experts Reviewed detailed field notes and complied transcripts Performed content analysis of field notes Detailed methods discussed in tech report in preparation 10
Results: Manual Ultrasonic Testing Task Analysis - A Sample FUNCTION: A group of activities generally performed together to accomplish an overall goal.
REPORT PLAN FOR PREPARE FOR CONDUCT EXAMINATION EXAMINATION EXAMINATION EXAMINATION RESULTS TASK: A smaller unit of
- 1. Receive plant orientation and training behavior that provides
- 2. Review work package information about the
- 3. Verify that the component is within the procedure and personnel human-information qualification ranges processing demands,
- 4. Assemble equipment and materials communication
- 5. Calibrate equipment to procedure requirements,
- 6. Attend prejob briefing instrument interactions,
- 7. Coordinate examiner responsibilities etc.
- 8. Pack equipment and materials
- 9. Prepare to enter area for exam
- 10. Locate component 11
Content Analysis of Interviews - Theme Development Analysis of themes is meant to provide a comprehensive view of the data and portray patterns across the participants*
A theme is:
attributes, descriptors, elements or concepts an implicit thread that organizes a group of repeating ideas a common point of reference and portray aspects of how the participants understand the research questions The goal is to develop a comprehensive set of concepts:
captures the essence of interview and focus group responses, can be used to prioritize and design potential mitigations can specify additional programmatic research for technical basis and guidance development Preliminary analysis reveal 15 major themes and 53 associated issues
- Hsieh, H, Shannon, SE (2005). Three approaches to qualitative content analysis. Qualitative Health Research. 15(9): 1277 - 1288 Ryan GW, Bernard HR. Techniques to identify themes. Field Methods. 2003; 15(1): 85109 12
Results: Example Themes Derived from Content Analysis The extent and quality of utility planning can facilitate or impede examination. (69 comments)
The individual examiner is responsible for a range of functions and tasks beyond simply performing the scan, including work site assessment in relation to the work package, individual scan techniques within procedure parameters, and evaluation and reporting of indications. (49 comments)
Time pressure is pervasive and multi-faceted - when performing examination, while preparing, when evaluating indications and reporting. (36 comments)
Training and practice is a valuable use of time, although opportunities may be limited for entry-level people. There are diverse sources of practice samples available, but no apparent system for utilization. (27 comments)
Team coordination is a key element of performing a UT scan, and includes planning team composition, interactions at the inspection site and peer-check on scan completion and interpretation. (22 comments) 13
Next Steps Complete summary report on Part 2: Task Analysis and Field Study Plan for workshop in Spring/Summer 2018 to prioritize issues identified in field research Plan for future activities 14
We need your expertise for continuing human factors research input Anticipated needs:
Workshop participants Virtual SME panel Occasional telephone and email consults to clarify material
Contact:
Thomas Sanquist, Ph.D.
Pacific Northwest National Laboratory sanquist@pnnl.gov Stephanie Morrow, Ph.D.
NRC, Office of Research Human Factors and Reliability Branch Stephanie.Morrow@nrc.gov Amy DAgostino, Ph.D.
NRC, Office of Research Human Factors and Reliability Branch15 Amy.Dagostino@nrc.gov
We thank our subject matter expert participants and host utilities - we appreciate your continuing support Future research efforts will benefit from your involvement