ML23292A045
| ML23292A045 | |
| Person / Time | |
|---|---|
| Issue date: | 09/25/2023 |
| From: | Dickerson K, Grasso J, Niav Hughes, Wang I, Watkins H NRC/RES/DRA/HFRB |
| To: | |
| References | |
| Download: ML23292A045 (26) | |
Text
Perceived Digital Control Room Usability and the Role of Individual Affinity for Technology Kelly Dickerson, Michael Hildebrandt, Heather Watkins, John Grasso, Isaac Wang, and Niav Hughes Green
- NRC Mission
- Digital modernization
- Technology affinity defined
- Importance of usability Roadmap studies to safety
- Study results
- Implications and future directions
NRC Mission
- The U.S. Nuclear Regulatory Commission (NRC) regulates the commercial use of nuclear power in the United States.
- The review and oversight of commercial nuclear power plants includes examining the human factors engineering (HFE) processes used by licensees.
Digital Modernization Nuclear industry is pursuing a range of modernization activities.
Advanced reactors and future control rooms are expected to be partially or fully digital.
Modernization does not always have a positive impact on human performance.
Measuring Operator Impact Direct comparisons of analog A shift in demand source could and digital controls revealed be due to system-related or levels of workload were operator-related factors.
comparable, but sources of demand were different.
Individual Operator Differences l Response to Technology
- Skills and knowledge about the specific system of interest.
- Individual and Personality characteristics
- Openness
- Need for cognition
- Coping
Affinity for Technology Interaction Approaching Higher Affinity technology Avoiding Lower Affinity technology
Safety Implications of Individual Differences in Affinity
- Operator level of adaptation, coping, and personal attitudes towards technology can impact usability ratings during system development and perceived usefulness and ease of use of new technology in the control room.
- Affinity levels could impact use of computerized procedure systems, trust in automated systems, and training time required for fluency with digital systems.
Operators and New Technology NUREG/CR-6947 Research Areas Supporting Review of NPP Technology Roles of personnel and automation Staffing and training Normal operations management Disturbance and emergency management Maintenance and change management Plant Design and construction Human factors engineering methods and tools
Research Questions This Photo by Unknown Author is licensed under CC BY-SA
Methodological Highlights l Samples Population Sample size University students 36 Operators 36 Operator sample includes 3 distinct data collection sites - there were no differences between collection sites on any tests. Therefore, operators were combined into a single sample.
Methodological Highlights l Affinity for Technology Interaction Scale (ATI)
- ATI is a 9-item survey that measures the tendency to engage with or avoid technology.
- Practical appeal: publicly available norming data, is quick to administer, and free to use.
- Scientific appeal: norming data can be used for research, correlated with the system usability survey, is a low-risk way to address some nuclear-specific research gaps.
Methodological Highlights l Surveys
- Two versions of the ATI
- An unmodified version assessing personal use of technology.
- Modified version reduced to five items and assessing workplace or obligatory technology use.
- Two versions of the System Usability Survey (SUS)
- Standard 10 item version.
- Modified for use in a known plant simulator.
ATI Results Participant ATI Personal Use Context* ATI Professional Use Context*
Type Operators 4.32 (.14) 4.41 (.13)
Students 3.61 (.12) 3.24 (.12)
- No difference between personal and professional use contexts.
- Operators had significantly higher affinity for technology than students.
SUS Results Standard Modified SUS Type*
83.96 (2.80) 68.52 (3.68)
- SUS ratings were significantly higher for the group using the HAMMLAB simulator compared to their home plant simulator.
- Difference could be due to modification of the SUS or simulator usability.
Affinity and Usability Table ATI Personal Use Context ATI Professional Use Context contains r2 values Standard SUS -.059 -.156 Modified SUS .321 .136
- No correlation between operator affinity for technology and perceived system usability.
by Unknown Author is licensed under KEY TAKEAWAYS AND CONCLUSIONS
Key Takeaway 1: Validation of new measures takes time
- No correlation between SUS and ATI in a single study does not necessarily mean that individual differences in technology affinity are not important considerations for operator performance in the context of new technology in the control room.
Key Takeaway 2: Increased technological diversity means a greater need for methodological research.
- NUREG/CR-6947 Anticipated the need for new methods in human factors and HRA back in 2008.
- The results of this study the need for new tools for digital systems in the nuclear domain.
- And the challenge ahead in modifying existing tools to fit within a nuclear HF problem space.
Concluding Comment l Translational Human Factors
- New tools developed by researchers may not be useful to regulators or the industry.
- New tools developed by industry may be perceived as lacking rigor by researchers and the regulator.
- A translational human factors approach may be needed to bridge these gaps and bring about methodological innovations.
Questions?
ATI Scoring
- 9 items
- 6-point Likert scale where 1 = completely disagree and 6 = completely agree.
- Three negatively worded items are reverse coded.
- The ATI score is the average computed across all 9 items.
ATI Questions I enjoy tinkering with technical systems.
I enjoy testing the functions of new technical systems.
I primarily deal with technical systems because I have to.
When I have a new technical system in front of me, I try it out extensively.
I enjoy spending time becoming acquainted with a new technical system.
It is enough for me that a technical system works; I dont care how or why.
I try to understand how a technical system works exactly.
It is enough for me to know the basic functions of a technical system.
I try to make full use of the capabilities of a technical system.
Bold questions are the five items used in the professional or obligator use context version of the ATI.
SUS Scoring
- Participant SUS scores were calculated using the approach reported in the literature.
- Odd numbered items: 1 was subtracted from the raw score.
- Even numbered items: the raw score is subtracted from 5.
- The sum of these scores was multiplied by 2.5 to produce the total SUS score.
SUS Questions Standard SUS Modified SUS I think that I would like to use this system frequently. I enjoy using the system.
I found the system unnecessarily complex. I find the system unnecessarily complex.
I thought the system was easy to use. I think the system is easy to use.
I think that I would need the support of a technical person to be able to use this system.
I found the various functions in this system were well I think the various functions of the system are well integrated. integrated.
I thought there was too much inconsistency in this system. I think there is too much inconsistency in the system.
I would imagine that most people would learn to use this I would imagine that most people would learn to use the system very quickly. system very quickly.
I found the system very cumbersome to use. I find the system very cumbersome to use.
I felt confident using the system. I feel confident using the system.
I needed to learn a lot of things before I could get going with I needed to learn a lot of things before I could get going with this system. this system.
Helpful References
- Franke, T., Attig, C., & Wessel, D. (2019). A personal resource for technology interaction: development and validation of the affinity for technology interaction (ATI) scale. International Journal of Human-Computer Interaction, 35(6), 456-467.
- Higgins, J., & Fleger, S. (2012). Human Factors Engineering Program Review Model (NUREG-0711) Revision 3: Update Methodology and Key Revisions (No. BNL-96812-2012-CP). Brookhaven National Lab.(BNL), Upton, NY (United States).
- Lin, J., Matthews, G., Schreck, J., Dickerson, K., & Green, N. H. (2023). Evolution of Workload Demands of the Control Room With Plant Technology. Human Factors and Simulation, 57.
- Park, J., Kim, Y., & Jung, W. (2016). A framework to estimate task opportunities from the operational experience of domestic nuclear power plants. Safety science, 88, 146-154.
- Park, J., Kim, Y., & Jung, W. (2016). A framework to estimate task opportunities from the operational experience of domestic nuclear power plants. Safety science, 88, 146-154.
- OHara, J., Higgins, J., Brown, W., & Fink, R. (2008). Human Factors Considerations in New Nuclear Power Plants: Detailed Analysis (No. BNL-79947-2008). Brookhaven National Lab.(BNL), Upton, NY (United States).