ML21176A134

Social Scientists in an Adversarial Environment: the Nuclear Regulatory Commission and Organizational Factors Research

ML21176A134
Person / Time
Issue date:	02/03/2021
From:	Thomas Wellock NRC/SECY
To:
Wellock T
References
Download: ML21176A134 (17)

Text

~ Taylor & Francis

~ Taylor& FrancisGroup Nuclear Technology ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/unct20 Social Scientists in an Adversarial Environment:

The Nuclear Regulatory Commission and Organizational Factors Research Thomas R. Wellock To cite this article: Thomas R. Wellock (2021): Social Scientists in an Adversarial Environment:

The Nuclear Regulatory Commission and Organizational Factors Research, Nuclear Technology, DOI: 10.1080/00295450.2020.1826273 To link to this article: https://doi.org/10.1080/00295450.2020.1826273 Published online: 03 Feb 2021.

~~

Submit your article to this journal 13'

!,hi Article views: 71 rli View related articles 13' CrossMdrk View Crossmark data 13' Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=unct20

NUCLEAR TECHNOLOGY DOI: https://doi.org/10.1080/00295450.2020.1826273 8ANS I11> Check for updates I Social Scientists in an Adversarial Environment: The Nuclear Regulatory Commission and Organizational Factors Research Thomas R. Wellock*

U.S. Nuclear Regulatory Commission, Rockville, Maryland Received March 3, 2020 Accepted for Publication September 16, 2020 Abstract This paper examines the Nuclear Regulatory Commissions (NRCs) pursuit of social science research that could inform the oversight of nuclear power plant management. Perhaps no nuclear regulator has been as supportive of research on the intersection of organizational factors and reactor safety or as cautious in applying those findings to its regulations.

This dissonance was rooted in the NRCs long-held conviction that it should regulate power plants not people, which conflicted with its regulatory experience after the 1979 Three Mile Island accident (TMI).

Intrusive oversight of a licensees business, it was believed, would destroy its sense of ownership for safety. TMI challenged that understanding of the NRCs role, and a series of mishaps at other plants compelled the agency to cross the line between regulation and management. The NRCs relationship with industry became highly adversarial, and the agency turned to social scientists to help establish an objective basis to judge a licensees organizational culture. Behavioral experts joined plant oversight review teams and received generous funding to quantify the contribution of organizational factors to accident risk. Scores of scholars at national laboratories and a dozen universities contributed, but the NRC abandoned the research in the mid-1990s in the face of inconclusive research and industry resistance.

In need of a less controversial oversight program, the NRC abandoned direct assessment of plant management for a more quantitative approach that relied on plant performance indicators. When the 2002 Davis-Besse vessel head erosion event came perilously close to a significant loss-of-coolant accident, it raised questions about the appropriate role for the NRC in assessing a licensees safety culture. The NRC revised its oversight program to incorporate qualitative insights from its earlier research while still acknowledging the line between regulation and management. The NRC learned that while there were substantial cultural and technical obstacles to integrating safety culture insights with established management and regulatory practices, it was necessary to overcome them.

The agency found stability in its contentious oversight program only when it made appropriate room for safety culture expertise.

Keywords History of nuclear power, organizational factors, safety culture, probabilistic risk assess ment, U.S. Nuclear Regulatory Commission.

E-mail: thomas.wellock@nrc.gov. I. INTRODUCTION

Thomas Wellock is the historian at the U.S. Nuclear Regulatory Commission and the author of Safe Enough? A History of Nuclear For social scientists, fitting into the world of nuclear Power and Accident Risk (Berkeley: University of California Press, energy has been difficult. Psychologists quickly estab 2021). lished themselves in the human factors profession during This work was authored as part of the Contributors official duties World War II in aircraft design. By the 1960s, human as an Employee of the United States Government and is therefore a work of the United States Government. In accordance with 17 factors experts had a firm foothold in the defense and USC. 105, no copyright protection is available for such works commercial aircraft industries, aerospace, the military, under US Law. and consumer applications, but respect was harder to 1

2 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT find in nuclear power, and especially, in the nuclear navy, performance and trigger regulatory action. It was which had inspired the U.S. civilian enterprise. a generously funded and somewhat quixotic campaign.

In a 1969 episode notorious in the human factors com Perhaps no nuclear regulator was as supportive of research munity, Admiral Hyman Rickover, the director of the office on the nexus of organizational factors and reactor safety or as U.S. Naval Reactors, made plain his lack of respect for human conflicted in applying those findings to its regulations.

factors engineers. The father of the nuclear navy fired off The NRCs ambivalent quest to quantify organizational a memorandum of protest after receiving a report on the factors sprang from skepticism of the value of the social Navys plans for a comprehensive human factors program sciences in nuclear operations and an engrained belief that in research and engineering. He called the program the the agency should regulate power plants not people. Intrusive greatest quantity of nonsense I have ever seen . . .. It is replete oversight of a licensees business might destroy its sense of with obtuse jargon and sham-scientific expressions. The ownership for safety. Yet, TMI challenged these misgivings, program would require a vast new social science bureau and subsequent plant mishaps compelled the agency to cross cracy contributing absolutely nothing to the building of the line between regulation and management. The NRC-ships. Before an appreciative congressional committee, industry relationship became highly adversarial, and the Rickover elaborated with relish on his dim view of the social agency turned to social scientists to establish an objective sciences. I could just imagine . . . one of these specialists basis to judge whether a licensees management and organi advising a project engineer that . . . his fire control panel zational culture provided adequate operational safety. Experts should play soft background music to ease the tension during in organizational management and behavioral sciences joined combat. For engineers, he concluded, a human factors pro plant oversight review teams and received funding to develop gram was about as useful as teaching your grandmother how indicators of the organizational contribution to accident risk.

to suck an egg.1,2a Under the direction of Brookhaven National Laboratory, The human factors profession won a little more respect scores of scholars at national laboratories, institutes, and after the 1979 Three Mile Island (TMI) accident when the a dozen universities contributed to the projects goal of study U.S. Nuclear Regulatory Commission (NRC) launched ing in-depth licensee operational and safety cultures.

a research program into the influence of human and organiza The social science researchers who stepped into plant tional factors. Even so, the struggle for acceptance continues control rooms, however, became part of the NRCs adversar today in an industry dominated by engineers and nuclear navy ial relationship with industry. They were greeted with suspi veterans. Dr. Valerie Barnes, a psychologist, spent a career in cion by licensees and skepticism by regulators, especially the nuclear industry and at the NRC. Until her recent retire when their research findings came to the disappointing con ment, she was among a cadre of NRC social scientists that clusion that the risks posed by poor management attitudes and oversaw its human factors, organizational factors, and safety practices were not quantifiable. Their more qualitative culture programs. She recalled her engineering colleagues did insights and methods were useful but labor intensive and not understand that she was an industrial psychologist, not divisive. The NRC abandoned the work in the mid-1990s.

a therapist who saw patients. They dismissed her disciplinary Without an impartial, objective measure of organizational methods and insights into human behavior and culture as performance, the NRCs oversight program was mired in fluffy, unquantifiable, and of limited value in regulation controversy.

compared to the hard quantification bent of engineering dis In the late 1990s, the improved safety performance of its ciplines. Despite a career in the nuclear power industry and licensees and an industry-friendly climate in Congress com regulation, she concluded, We spoke completely different pelled the NRC to craft a less controversial oversight program languages.3 that was more performance-based and quantitative. In the This paper details an ambitious NRC program in the new program, the agency avoided direct evaluation of early 1990s to get social scientists and engineers to speak unquantifiable factors such as licensee safety culture. When the same language. The agency tried to translate the qualita in 2002 the vessel head on the Davis-Besse nuclear power tive results of social science research on organizational factors plant suffered severe erosion and came perilously close to as inputs into probabilistic risk assessments (PRAs). Using a loss-of-coolant accident, questions about the NRCs appro the latest social science methods and tools, the NRC aimed to priate role in policing licensee safety culture resurfaced. The develop quantitative indicators of organizational factors that, agency incorporated qualitative insights from safety culture when plugged into a PRA, would flag a licensees degrading research into its oversight program while still acknowledging the line between regulation and management.

a A colloquialism for the useless act of teaching someone to do As the historian of the NRC, I benefitted from the advice what they already know. and recollections of experts in PRA and human and 8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 3 organizational factors. As is befitting my historical training, issued a report calling for greater attention to human engi this is a work of history that applies the methods of the neering with improved control room design, operating pro profession to evaluate historical sources by scholars, technical cedures, and personnel training. Several years later, the NRC experts, and government officials to produce a reliable narra issued similar reports calling for control room design and tive and analysis of past events.4-6 The NRC and its prede procedures suited to accident conditions.15,16 cessor agency, the Atomic Energy Commission (AEC), have The emerging field of PRA also identified the risks long valued and supported an agency history program that posed by human error. In 1975, the NRC published the meets the standards of the historical profession.7-13 In this Reactor Safety Study, the most ambitious PRA attempted.

paper, I have excavated publicly available NRC and industry A major contribution was its recognition that maintenance sources, as well as contemporary news accounts and scholarly and operator error were significant contributors to accident literature on human and organizational factors. What emerges risk, perhaps even more than the major hardware failures is a story in which the notion and use of organizational factors postulated in the NRCs design-basis accidents.17 and safety culture in regulation evolved in a turbulent regula The Reactor Safety Study observations on human error tory environment of political conflict, an expanding concep prompted the industrys Electric Power Research Institute tion of human factors, and uncertain research findings. (EPRI) to contract with Lockheed Corporation for a major study of human factors in nuclear power plant control room design. Like the previous AEC report, Lockheed found scant II. ENGINEERS AS SOCIAL SCIENTISTS: HUMAN FACTORS evidence human factors had been considered, and engineers BEFORE TMI treated human factors as a black art. Even the most modern layouts violated basic human-factor standards. EPRI and the Before TMI, the role of human error in plant accidents NRC began modest programs to study and improve human received limited attention in control room design and proce factors, but major initiatives did not begin until after TMI dures. The AEC and NRC had to ensure nuclear power plants (Refs. 14 and 18).

provided adequate protection to the public, as required by Collectively, the human factors studies of the pre-the Atomic Energy Act of 1954. Regulators sought to achieve TMI era revealed the limited attention paid by design adequate protection in engineering terms of compliance to engineers and regulators to the human element of nuclear rules regarding design, operating procedures, and minimum plant operations. Even the scant attention paid to human qualifications for reactor operators. Nuclear power plant factors was more than the complete ignorance within the licensees had to operate plants according to technical speci industry of how organizational factors might contribute to fications that established administrative controls and safety an accident. Even if the industry had looked for it, there limits on hardware and plant conditions. On paper, the roles was hardly any social science research on the contribu for regulator and licensee were clear. Regulators focused on tions to disaster from cultural mechanisms, ignorance, hardware safety and monitored for violations of rules and and excessive organizational confidence.19-21 technical specifications. Licensees followed the rules and managed plant operations and personnel.

The safety importance of the man-machine interface III. TMI AND THE NRCS FIRST HUMAN FACTORS ERA the specialty of human factors expertswas less clear and (1979-1985) responsibility for it fell between the cracks. Neither regulators nor design firms employed human factors experts. Instead, Three Mile Island validated larger budgets for research architect-engineering firms borrowed from the functional on human factors. In the first few years after the accident, the control rooms of fossil fuel plants to create sprawling versions NRCs human factors research program consumed over at nuclear plants with reliable, oversized controls and indica $20 million. The pre-TMI view of error as a product of an tors spread across multiple panels and arranged for normal individual operator and his immediate environment carried operations. In off-normal events, panel layout was less than over into post-TMI studies of control room design, revisions optimal. Operators might dash about in haste from one panel to procedures, human error studies, and accident training.

to the next to synthesize the cascade of alarms and informa There was broad agreement that the accident might have tion into a diagnosis and select among dozens of identically been avoided with an effective lessons-learned program, arranged and shaped switches to stabilize the plant.14,15 more simulator training, and intelligent control room layouts By 1972, a series of mishaps made evident the risks from and procedures. Although organization and management a poor man-machine interface. Operator errors aggravated issues were identified as a concern, research funding was and prolonged several routine plant transients. The AEC initially limited.12 NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

4 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT Calls for a more substantial study of organizational attitudes, and morale and drew unmeasurable conclusions, factors came from outside assessments of the accident. The such as having a good feeling about plant management.

presidentially appointed Kemeny Commission and an NRC- Battelle argued for organization and management reviews sponsored study team under lawyer Mitchell Rogovin identi that could withstand a legal challenge with better evaluation fied management dysfunction as a contributor to the accident. criteria, training, data collection, and objective performance Kemeny called for higher organizational and management indicators.27,28 standards for licensees. Similarly, the Rogovin Report The NRC also contracted with Battelle to develop pressed the NRC to regulate organizational structure, a management assessment approach for plants already in human factors, and management.22,23 operation. It analyzed two scholarly traditions of organization Kemeny and Rogovin prodded the NRC into forbidden and management analysis. Management analysis reached territory, and agency staff tried to squirm away from direct back to the 1950s and studied the behavior, decisions, and management assessment. In late 1980, they produced broad, processes of individual and groups of managers. Battelle uncontroversial guidelines for acceptable utility structure and favored more recent work on organizational analysis, which safety functions. The dozens of NRC licensees ranged from studied elements of an organizations environmentits size, tiny municipal utility districts to Fortune 500 corporations. structure, governance, reward systems, and administrative They came with idiosyncratic management structures and controls. By steering clear of individual management beha styles. The agency conceded evaluation would be on vior and focusing on the organizational environment, Battelle a largely subjective basis and promised to show great flex expected it could develop a sophisticated integrated socio-ibility. Nevertheless, a draft and redraft of the guidelines technical, human machine perspective. It would avoid the elicited sharp criticism from utility executives that the guide subjective judgments of NRC licensing reviews and broaden lines were an over-reaction to TMI, unrealistic and over- the narrow perspective of engineers to include organizational zealous, and too prescriptive and unnecessarily restrictive. factors.29 In search of firmer ground, the staff launched research into There was a complication in using existing scholarship safety-related management attitudes and organizational for licensing oversight. Organizational research had studied variables [that] can be objectively assessed to relate man business performance and managerial efficiency, not safety.

agement behaviors to safety criteria through an appropriate Could the research measure safety performance? Battelle model.24-26 divided up the problem into three tasks: (1) develop objective For the first time, the NRC awarded experts in aspects of organizational factorsb most likely to influence safety, (2) organizational management substantial funds to investigate identify a set of safety performance indicators, and (3) find the link between utility organizational characteristics and correlations between the two sets. If successful, the NRC safety. In 1982, the Battelle Human Affairs Research Center would have indicators of organization and management that in Seattle, Washington, contracted for a series of reports on predicted safety performance. If they could not, they were not organization and management to be developed over several useful to the NRC or licensee.30,31 years. Battelle was a think tank with deep experience in Battelles researchers grouped organizational factors into nuclear power, and it had broad multidisciplinary expertise four categoriesutility environment, historical context, orga in engineering, social sciences, and business management. nizational governance, and organization designthat were One of Battelles first tasks was to report on how the most influential of utility behavior important to safety, such NRC assessed management competence when staff reviewed as a commitment to quality, compliance, efficiency, innova a utility application for a new operating license. In construct tion, and employee maintenance. Early analysis indicated ing observation teams to accompany NRC staff on site visits, organizational design was the most predictive of operational Battelle included human factors experts and a diversity of safety as indicated by intermediate outcomes that were scholars in business management and social sciences, such as presumed to contribute to safe operations, such as quality, social psychology, sociology, and political science. The NRC efficiency, innovation, and rules compliance.30,32 staff review used a formal standard review plan to assess training, the structure of plant management, regulations, and b Battelle did not define organizational factors, but current definitions staffing levels. Nevertheless, Battelles researchers observed capture their 1980s essence as the organizational structures, processes, that staff commonly rendered subjective judgments. They and behaviors that influence the actions of individuals at work. See S.

Peters and others, Organizational Factors in PRA: Twisting Knobs and lacked training in social science interview techniques or in Beyond, in the Proceedings of the 2019 International Topical Meeting how to perform a site assessment. A utilitys reputation on Probabilistic Safety Assessment and Analysis (PSA 2019), April tended to bias the assessments, and staff strayed into unstruc 28-May 3, 2019, Charleston, South Carolina, NRC ADAMS tured, subjective questions about management culture, ML19057A474.

8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 5 The first Battelle report published in 1983 offered an Issued a month later, the revised program eliminated upbeat message that its organizational approach could solve direct research and regulatory activity on organizational some of the NRCs touchy oversight issues with detached factors in favor of a more practical program capable of assessments. Rather than the intrusive site visits the NRC near-term accomplishment, as contrasted to the more used for licensing, it proposed a passive data collection academically oriented program previously described in model for operating plants. It would use existing, publicly the Plan. The Battelle studies would not be extended available qualitative and quantitative data on utility organiza beyond work already in production. In 1985, the NRC tional structure and plant performance from corporate reports went further and halted all human factors research.39,40 and documents already required by the NRC. Battelles pre Later scholars considered Battelles passive collection liminary conclusions found tantalizing indications that of data and focus on organizational structure shortcomings aspects of utility organization were linked to measured safety reflective of the pre-Chernobyl scholarship on organiza indicators.33 tional factors and accidents. It served as an intellectual As Battelle expanded its research, however, the correla stepping stone to research into organizational processes tions were harder to discern among a fleet of nuclear power and culture measured through hands-on assessments with plants in which no two were alike. It was challenging enough advanced psychometric surveys and interview tools.40 for the researchers to determine whether an organization with Research limitations aside, the cutbacks in human factors many layers of management was more prone to plant mishaps research reflected a changing regulatory environment during than one with very few, but it was especially so as they the Ronald Reagan administration of general skepticism accounted for variations in power output and reactor and toward regulation. There was pressure on the NRC to get steam plant designs. Nevertheless, researchers argued there back to a pre-TMI normal. In January 1981, the NRC was enough promise in the approach to warrant an extension controller promised that by 1983 the agency budget would of the NRC contract. The consequences of human error, it be just as though thered never been a TMI. Palladinos warned, were too great to ignore. There is comparatively deference to industry initiatives also accorded with the core little opportunity for instructive learning until after dysfunc belief that licensees had to take ownership and personal tional effects have occurred. With an assessment tool that responsibility for the safe operation of their plants. Thirty was predictive as well as explanatory of safety, the NRC years later, former Commissioner Kenneth Rogers echoed could optimize utility organization, learning, and resources; Palladinos sentiments. Licensees, he said, must never lose a more efficiently operated plant would be a safer plant.34,35 that sense of responsibility that it is their responsibility, not While the research was a mixture of potential and NRCs responsibility, to take the lead . . .. This is your plant, problems, Battelles approach was politically astute in this is your facility.41 proposing a model that recognized the realities of the In the mid-1980s, the NRC deferred to self-regulating NRCs adversarial relationship with licensees.36 Battelle initiatives under the leadership of the Institute for Nuclear reassured the NRC its methods would be minimally Power Operations (INPO). This was not the first time the obtrusive. Neither the NRC nor the industry wishes the NRC leaned on INPO to avoid friction with industry. The NRC to become involved in the day-to-day management organization had taken the lead on other operational issues of the nuclear plants, Battelle observed. This would and training to avoid more aggressive NRC regulation. It mean active involvement in supervision and decision- encouraged traits of navy management excellence.

making that would be unhealthy for both parties and the Caught between a tradition of management deference, post-public safety.37 TMI pressure for more active oversight, and budgetary con Battelles careful balancing of politics and science still straints, the NRC handed the lead to INPO with a warning.

ran afoul of the divide between regulation and management. As now envisioned, NRC will not develop management and At a December 1982 Commission briefing on human factors organization criteria for operating reactors unless the INPO programs, Chairman Nunzio Palladino expressed his discom effort proves to be unsatisfactory for our needs.39 fort with an aggressive human factors program. I get worried when we get our tentacles out so far where we seem to be big brothering every aspect of the operation . . .. I get a little IV. SAFETY CULTURE: CHERNOBYL, DAVIS BESSE, AND uneasy when we get into peripheral aspects because . . . if we PEACH BOTTOM (1985-1988) get into everything we tend to lose the initiative of the orga nization itself. After that, Palladino repeated the tentacles Dissatisfaction soon followed from a collection of events metaphor, and the staff promised to come back with a revised at home and abroad. The 1986 accident at the Soviet Unions program.38 Chernobyl nuclear power plant crystalized international NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

6 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT recognition of the importance of a safety culture in opera Peach Bottom shocked the NRC and raised the priority tions, and the International Atomic Energy Agency (IAEA) of licensee oversight. Thomas Murley, who became the became a consistent champion of developing safety culture director of the Office of Nuclear Reactor Regulation, con criteria and assessment tools.c While supportive of safety sidered it a watershed moment that forced the agency to culture discussions, the NRC also distanced its regulation find a way to assess plant safety culture and rid the indus and the industry from the Soviets by stressing distinctive try of what was called a fossil-fuel mentality. Despite features of the U.S. system, including safeguards against post-TMI reforms, human error and organizational factors degrading operator performance and the superior safety fea played a role in half of all plant events. Under the leader tures of light-water reactors.42 ship of nuclear navy veterans, INPO actively worked to Events made some wonder if the NRC was drawing overhaul Peach Bottom management. At the NRC, distinctions without a difference. Ten months before Chairman Lando Zech, a former nuclear navy admiral, Chernobyl, a rapid series of malfunctions and operator errors announced that the NRC needed to do all it could to create at the Davis-Besse Nuclear Power Station near Toledo, Ohio, a plant environment that maximized operator performance.

produced plant conditions uncomfortably close to TMI. The The NRC turned to its enforcement powers to compel NRC concluded lax management was a significant under management improvements. In 1978, just before TMI, lying cause of the event.43 the NRC issued just 14 monetary penalties for regulatory More disturbing were revelations at the Peach Bottom violations. By 1987, it issued 114. In 1989, the plant in Pennsylvania. In 1987, the NRC issued fines to Commission approved a policy statement on the conduct dozens of plant operators for inattentiveness (sleeping) of plant operations that included a definition of safety while on duty. Only a whistleblower tip alerted the NRC to culture as the personal dedication and accountability of the practice. Avoiding detection of plant staff, an NRC inspec all individuals to practices of plant safety and the promo tor turned off the lights in his plant office and hid under his tion of an environment of safety consciousness.49-54 desk until late into the evening. Emerging from his hiding spot, he entered the control room and caught a dozing opera V. NATIONAL ACADEMY OF SCIENCES REPORT (1988) tor. Further investigation revealed a culture that tolerated many operators dozing on shift, playing video games, reading In the search for answers to regulatory conflict, the magazines, and engaging in rubber band fights. Time maga agency looked again to the social sciences. After a 2-year zines headline, Wake Me if Its a Meltdown, was humorous hiatus in its funding of human factors research, the NRC tried commentary on a serious safety culture issue. The INPO again. It commissioned the National Academy of Sciences called Peach Bottoms management an embarrassment to (NASs) National Research Council to recommend a human the industry and the nation. The NRC ordered a shutdown of factors program. Chaired by human factors expert Neville units 2 and 3 that lasted over 2 years.44-48 Moray, the NAS panel was composed of diverse membership from the nuclear industry, engineering, traditional human factors experts, business management, and the social sciences.

c Safety culture was coined by the International Nuclear Safety The panel stepped outside its mandate from the NRC and Advisory Group of the IAEA. Initially, safety culture had a narrow articulated a research agenda that was a significant departure definition in the research literature as one factor within a broader organizational culture. In turn, organizational culture was one of many for the agency and the human factors profession. The NRCs organizational factors, such as resource allocation, training, and organi post-TMI human factors program, the panel wrote, was hin zational knowledge. Unfortunately, this logical hierarchy broke down as dered by the pursuit of purely technical solutions to human safety culture grew into an omnibus concept that subsumed many problems as might be expected of the NRCs engineering elements once considered organizational factors. For example, the culture. It observed the human factors profession suffered NRCs safety culture policy statement and related documents include several traits of a healthy safety culture previously seen as organiza from a lack of imagination, too. Human factors began as tional factors. See Identification and Assessment of Organizational a field of psychologists and engineers with the singular goal Factors Related to the Safety of NPPs, State of the Art Report, NEA/ of ensuring that controls and displays appropriately matched CSNI/R(99)21, Vol. 1, pp. 11-21, Nuclear Energy Agency, Committee the capabilities of humans. By the 1980s, it expanded into the on the Safety of Nuclear Installations (1999) and S. Peters and others, optimization of personnel selection, training, and the design Organizational Factors in PRA: Twisting Knobs and Beyond, in of habitable work environments, but the panel concluded the Proceedings of the 2019 International Topical Meeting on Probabilistic Safety Assessment and Analysis (PSA 2019), April 28- field needed to view nuclear power plants and other complex May 3, 2019, Charleston, South Carolina, NRC ADAMS technologies as sociotechnical systems affected by organi ML19057A474. zational factors and a technologys social context. Capturing 8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 7 this broader concept of human factors in research required needed social scientists to help them understand and multidisciplinary teams, including an array of social science quantify human and organizational behavior.65,66 disciplines.55-57 The panel called on the NRC to prioritize research into VI. BROOKHAVEN STUDY (1988-1995) organizational and management factors. Management can make or break a plant, Moray told the NRCs Advisory At the behest of Thomas Murley, the director of the Committee for Reactor Safeguards. Even more than the man- office of Nuclear Reactor Regulation, the NRC tried to machine interface, he said, it was essential that the NRC turn social science theory into regulatory reality. The idea identify what made for a positive organizational culture of was simple, if the NRC could quantify an organizations reliability and safety and develop appropriate regulatory contribution to accident risk, it would have an objective feedback mechanisms that would reduce accident risk.58,59 basis to shut down a poorly operated plant. In 1988, the By the time the NAS issued its report in 1988, much staff recommended the NAS report to the Commission had changed in organizational scholarship. The NAS com anticipating that in-depth research of licensee culture mittee included Todd LaPorte, a political scientist at the could be combined with its own inspection programs to University of California, Berkeley, who, among other produce measures that were predictive as well as descrip researchers at Berkeley, pioneered the study of high relia tive of degrading licensee performance. If successful, the bility organizations or HROs. HRO scholarship offered research would become a basis for integrating manage a practical alternative to earlier theoretical work by sociol ment factors into the probabilistic risk assessment PRA ogist Charles Perrow on Normal Accidents. Perrow process, and as a basis for developing indicators of orga argued that technologies such a nuclear power were too nization and management performance.54,67 complex and carried consequences too great to operate The NRCs interest in quantifying organizational factors safely. Technical complexity outstripped the capacity of and safety culture set it apart from the international nuclear organizations to manage them safely. The ambition of communitys qualitative approach. The IAEA produced HRO studies was to identify traits of organizations with numerous consensus documents on a safety culture definition successful records of operating in complex, high-risk envir and qualitative assessment guidance for IAEA safety culture onments where trial-and-error learning was not an option. reviews. IAEA technical committees viewed safety culture as LaPorte and HRO scholars typically studied organizations a search for tangible evidence of an essentially intangible with military or military-like discipline, such as aircraft concept. It encouraged review teams to draw qualitative carrier flight operations and air traffic controllers.60-64 judgments of attitudes, morale, motivation, and commit A new NRC human factors research program also ment to safety.68 stood to benefit from the maturation of social science In attempting to condense organizational complexities to survey tools that purported to quantify aspects of organi a number in a risk calculation, the NRC program seemed like zational culture. Through interview protocols and psycho engineering naiveté, but it also reflected the regulatory and metric inventories, such as the Organizational Culture industry trend in the United States toward risk quantification.

Inventory, researchers quantified an organizations cul The NRC Commission and some elements of the staff placed tural norms and expectations. Complementing traditional a significant bet on PRA to reform its cumbersome and qualitative cultural assessments, the inventories quanti controversial regulations. In the 1980s, the agency had pains fied how an organizations members thought about attri takingly assembled critical pieces of a more risk-based reg butes, such as managerial effectiveness and the quality of ulatory framework. It approved safety goals for nuclear interpersonal relations. Extending this work to NRC power plants that included numerical objectives of plant licensees might allow the agency to compare licensee risk. It completed an ambitious revision to the Reactor safety attributes through quantified indexes and to track Safety Study, and it required plant licensees to search for the rise and fall of their organizational risk profile. The plant design vulnerabilities that were typically performed combination of theory, methods, and tools heralded the with individual plant PRAs. Later in 1995, the NRC arrival of the social science PRA expert. Social scientists Commission added a policy statement on risk assessment might do more than simply enlighten engineers as to the more encouraging of the methodology than its cautionary different perceptions of risk held by experts and the statement issued in 1979 (Ref. 69).

public; they were ready to provide direct quantitative There was still a lot of work to do to integrate PRA into input into calculations of accident risk. As sociologist regulations. The quality of utility-produced PRAs varied, William Freudenberg argued, if nuclear engineers wanted and the NRC wanted high-grade products in regulatory to improve the accuracy of their PRA calculations, they applications. PRAs were voracious consumers of specific NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

8 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT equipment-failure-rate data and other quantitative inputs. their Nuclear Organization and Management Analysis Yet, the data on some accident scenarios were limited and Concept (NOMAC) model at its fossil fuel plant at the uncertainties in risk estimates were large. Experts had to Pittsburg, California. Satisfied with their scientific rigor, reduce these uncertainties for hard-to-quantify accident PG&E permitted the team to move on to the Diablo Canyon contributors, such as poor plant management.70,71 To be nuclear power plant. The NRC reported hopefully that useful to regulators, social scientists had to produce quanti NOMAC can be implemented in a reasonably nonintrusive fied results useful to PRA practitioners. manner, can be received favorably by utility personnel, and Across the engineering/social science divide, PRA that meaningful data can be extracted for use in exploring the experts were optimistic it could be done. George influences of organization and management factors on relia Apostolakis was a University of California, Los Angeles bility and risk.77,78 (UCLA), professor in nuclear engineering and PRA expert who went on to become an NRC commissioner. In 1989, he coauthored an editorial that captured the disciplines opti VII. RESEARCH IN AN ADVERSARIAL ENVIRONMENT mism that PRA could serve safety and resolve industry con flict. PRA had become a kind of lingua franca of risk-related So far, so good, but broadening the study to include the decision making . . .. We have a common language with which cooperation of other licensees posed a considerable chal to discuss a particular problem area, like nuclear risk. lenge in the conflict-ridden regulatory environment of the Without it, the nuclear industry would suffer chaos, confu 1980s and 1990s. As NRC executive director Victor Stello sion, controversy, fear, litigation, and paralysis. With living put it to a conference of regulators and industry, the United PRAs, utilities would know accident risk at every moment of States had the worlds most adversarial regulator-industry operation. Over the next few years, quantification of morale, relationship. We do not trust you, you do not trust us.49,79 esprit de corps, management attitude . . . should see consider Stellos was an accurate assessment. After TMI, the able progress. Apostolakis later recalled with a laugh, We agency created the Systematic Assessment of Licensee thought we could quantify everything.72,73 Performance (SALP) to combine quantitative and qualitative Brookhaven National Laboratory had a staff of psy assessments into a rating system of plant performance. After chologists, and the NRC contracted with it for a $5 million Davis-Besse and Peach Bottom, the agency added a capstone study that stretched over several years from the late 1980s senior management meeting to review the performance of to the early 1990s. Under psychologist Dr. Sonja Haber, each plant and a watch list of problem plants. Plants with Brookhaven worked with two other national laboratories, worrisome performance received special attention from a several institutes, and 12 universities to identify organi Diagnostic Evaluation Team (DET) that made a multiweek zational factors related to the safety of nuclear power assessment of operations. DETs consisted of about 15 staff, plants and to develop quantitative measures appropriate including behavioral scientists, that conducted a broad assess for PRA use. Apostolakis and his UCLA team were to ment in areas such as organizational culture and management integrate the Brookhaven results into a PRA model it beliefs, attitudes, practices . . . as well as key sociological designed to see if it worked.74,75 factors. NRC staff intended to feed DET data into the Brookhaven developed a structural model of a nuclear Brookhaven research. Initial DET assessments, staff reported, power plants organization drawn from research by Henry were quite useful and utilities appreciated DET insights.80,81 Mintzberg at McGill University in Montreal, Canada. The era of good feelings did not last. By the early 1990s, A nuclear plant organization, Brookhaven concluded, was the industry had turned against DETs and the SALP. DETs best described as a machine bureaucracy with highly for were often a prelude to placing a plant on the watch list, malized procedures and rules, specialized groups, extensive a black eye that got upper management fired, sent utility professionalism, and a special need for safety. Investigators stock prices tumbling, and required millions on maintenance identified over 20 promising organizational factors under five and operational improvements. This oversight regime was broad categories of control systems, communications, cul matched with blunt enforcement tools, such as fines and ture, decision making, and personnel systems. Brookhaven plant shutdowns, that were unique among the worlds regu concluded the best measurement methodologies included lators. Industry objected that the SALP and DETs were too research surveys, behavioral checklists, structured interview subjective, a claim substantiated to a degree by the NRCs protocols, and behavioral-anchored rating scales.76 inspection guidance documents, which called for judgments Wary but interested, the Pacific Gas and Electric on safety culture without any standards to measure it.82-86 Company (PG&E) allowed a team of researchers from the The NRC leadership began to look askance at DET University of California, Berkeley and Brookhaven to test out evaluations, too. In the late 1980s and early 1990s, navy 8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 9 veterans had taken over key leadership positions on the of the project, Haber concluded the teams produced good Commission and agency staff. The NRCs executive direc qualitative lessons. They analyzed 20 organizational fac tor, James Taylor, had worked under Rickover and was tors grouped into the four broad categories of culture, especially skeptical. Brian Haagensen, today an NRC communications, human resource management, and man inspector, worked as a contractor on DETs. He recalled agement attention to work practices related to safety.

that Taylor thought the DET evaluations were too aggres There were connections between safety and organiza sive, and the social scientists on the teams issued harsh tional traits such as effective communication, the ability grades without understanding management or nuclear tech of an organization to learn, management attention to nology. Taylor preferred Haagensens firm, in part, because operations and safety, and the external environment of it was led by nuclear navy veterans, including Haagensen.87 corporate and regulatory factors. They found some stable The Brookhaven teams that visited nuclear plants, then correlations between the factors, such as organizational worked for a regulator dubious of their capabilities and stu learning and safety performance, but they could not find died licensees suspicious of their motives. Rather than serve correlations with many others.89 as dispassionate scientific observers, they became entangled Applying those insights to PRA did not work. George in the research environment. In anonymous evaluations col Apostolakis concluded it was almost impossible to incorpo lected in 1991, they vented their frustrations. The nature of rate organizational factors into PRAs. Part of the problem, as the relationship between the NRC and the utilities permeates Haber recalled, was finding utilities willing to work with our role as contractors to the NRC, one researcher reported. Brookhaven to develop more data, but survey and interview While some utilities showed keen interest in their work, the results were a difficult fit for PRA anyway. It was one thing to anxiety of nuclear utilities at their presence was palpable. develop a 1 to 5 rating scale for various factors in a cultural There was little incentive for licensees to cooperate when inventory and quantify a good or fair utility organization.

research findings might become burdensome regulations. It was another to quantify organizational influence on equip Success is failure, one observed. The better the research ment failure rates or human error. Haber concluded that on the impact of organizational factors (success), the more continued efforts to correlate organizational dimensions likely the industry will put pressure on the NRC to cut the with performance indicators may have limited value as funding for future research (failure). The whole project, one a nexus to safety . . .. We consider Culture as a higher concluded, was hindered by the utilities lack of trust in the order factor which cannot be incorporated into [PRAs]. The NRC to use the results of our research sensibly.88 Brookhaven methodology was best suited for providing qua The scholars were even more exasperated with the litative information for inspections and diagnostic evalua NRC. Fearful of inflammatory findings, the NRC did not tions. The NRC staff concluded Brookhavens methodology allow the teams to study poor performing plants or inter was too resource intensive and had relatively low cost effec view upper management in corporate offices. While ele tiveness. It recommended discontinuing research until orga ments of the NRC staff supported the Brookhaven project, nizational factors could be integrated into PRAs (Refs. 89 some researchers detected from NRC management a gen and 90).

eral distrust of the social sciences and behavioral science data. Some of the social scientists also found they could not bridge the cultural divide with engineers on their own VIII. NEAR DEATH AND THE NEW REACTOR OVERSIGHT research teams. The gap between engineers/PRA-types PROCESS (1994-1999) and behavioral scientists does not seem to be closing very fast, one observed. Another wrote, Social scientists in an As the Brookhaven research progressed, industry dis engineering world will always have a tough time. satisfaction with the NRC oversight program grew. In 1994, The research teams bridled at the NRCs restrictions. the Nuclear Energy Institute, an industry trade organization, With a bit of irony, these experts in social and political contracted with the consulting firm Towers Perrin to review sciences wanted to rid their research environment of the the NRCs relationship with its licensees. It delivered its politics that defined it. They asked the NRC to calm assessment just two weeks before the momentous 1994 utilities and remove the fetters on their access to utilities elections where the Republican Party gained control of and corporate management. The future success of this Congress. The report claimed the NRC was an arrogant effort depends upon the cooperation of regulators, con regulator whose current regulatory approach represents tractors, and the nuclear industry. a serious threat to Americas nuclear energy generating While the teams asked for more support, the research capability. Even as plant operations improved, the NRCs itself had mixed results. In a 1995 summary presentation SALP program administered arbitrary, punitive judgments NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

10 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT and distracted plant management with minor issues. The questionnaires or surveys . . .. If the theoretical fragmentation report provided numerous examples of trivial NRC inspec of the field is not overcome, commentators will conclude that tion findings, such as leaving blank spaces on routine forms the work failed to realize its considerable promise. The term and poor housekeeping that missed dust bunnies behind safety culture, he lamented, might turn into hollow rheto a plant telephone. Special inspections such as the DETs ric that pays lip service to safety.96 were inflammatory, inflated, and not constrained to fact.91 As it entered the twenty-first century, the NRC did not As the industry expressed its dissatisfaction with the explicitly regulate safety culture in the ROP. Unable to quan SALP, poor performing plants made headlines. tify organizational factors, the ROP inferred them indirectly A whistleblower at the Millstone nuclear power station from other performance indicators, such as the number of made the cover of Time magazine. The NRC ordered the unplanned plant shutdowns. The INPO took the lead on shutdown of all three Millstone reactors. Millstone unit 1 inculcating safety culture among licensees. In 2004, George never produced electricity again, and the utility spent over Apostolakis presented a gloomier assessment of safety cul

$500 million on upgrades and reforms to get unit 3 back on ture quantification than he had in 1989: Defining indicators line 2 years later. In 1 year, the number of plants on the watch of a good or bad safety culture in a predictive way remains list jumped from 6 to 14 (Ref. 92). elusive. [PRAs] certainly do not include the influence of A turning point came in 1998. With the support of his culture on crew behavior and one can make a good argument subcommittee, Republican Senator Pete Domenici, that they will not do so for a very long time, if ever.97-100 a champion of nuclear power, forced the NRC to abandon The Brookhaven methodology went into exile. The its SALP process by threatening cuts to 700 staff. The NRCs Atomic Energy Control Board of Canada (AECB) was near death experience led to the creation of a new Reactor interested in the NRC-sponsored research. Haber left Oversight Process (ROP) that spelled out cornerstones of Brookhaven and with Dr. Michael Barriere adapted the safety keyed to more quantitative indicators of safety perfor NOMAC model to reflect advances in methodology and mance. NRC resident inspectors remained a bedrock presence research. Interested in the methodologys qualitative in the ROP, as they were in the SALP, but their findings and insights, the AECB simply required it be practical, gen evaluations of plant events were, as much as practicable, erate reliable results, and develop an accurate picture of quantified and categorized by their safety significance. Plant nuclear plant operations. While the Canadians applied the operations were evaluated by comparing performance indica model to numerous facilities, they found it was, as the tors against prescribed risk-informed thresholds. By accounts NRC had previously, a resource-intensive tool. Spain from industry, NRC veterans, and even nuclear power critics, investigated a similar model.101-103 the ROP was more objective than the SALP, less adversarial, A few countries, such as Switzerland, Finland, and still resulted in significant improvement in the safety Germany, Canada, and Spain formally established safety performance of the U.S. nuclear fleet. The ROP was part of culture regulations or required operators to perform safety a broader effort to risk-inform regulations by supplement culture self-assessments. Most other nations adopted pol ing traditional qualitative indicators of safety with more icy statements where safety culture appeared as a major quantitative measures. As the quantification of performance theme or stated key traits that constituted a good safety indicators grew in importance, however, some qualitative culture.102,104 assessments were left behind. That was the end of the DETs, recalled Brian Haagensen.87,93-95 The NRCs inability to translate social science research IX. DAVIS-BESSE AND A NEW APPROACH TO SAFETY into a practical tool aligned with similar frustrations among CULTURE (2002-PRESENT) academics who studied safety culture. In 1998, Nick Pidgeon, a psychologist and safety culture scholar at the University of As safety culture gained some currency abroad, the Wales, assessed the fracturing of safety culture research along Davis-Besse nuclear power plant suffered an event in disciplinary, theoretical, and practical lines. Some 10 years 2002 that forced the NRC to revisit its deference to the on from Chernobyl, the existing empirical attempts to study nuclear industry on the issue. During plant maintenance, safety culture and its relationship to organizational outcomes workers discovered substantial erosion in the 6.5-in.-thick have remained unsystematic, fragmented, and in particular reactor vessel head. Cracking in reactor control rod com underspecified in theoretical terms. Engineers wanted ponents led to the escape of corrosive borated water and a best solution, but social scientists had only managed to steam, which scoured a hole in the vessel the size of raise the thorny issue of whether culture can be measured a pineapple. Only a thin stainless steel liner on the inside at all using quantitative psychometric methodologies such as of the vessel head prevented a loss-of-coolant accident.105 8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 11 It was the most significant safety event in the United defense in depth, evolved from an almost exclusive focus on States since the TMI accident. First Energy Nuclear design safety to a multilayered system of plant defenses Operating Company (FENOC) had taken over operations including design, organizational, and human elements. And, from Toledo Edison Company and enjoyed a reputation as while the NRC did not develop a safety culture regulation, it a top performer. Its fall from grace was dramatic. It lost inserted a little qualitative judgment on safety culture into its hundreds of millions in revenue, including $33.5 million in ROP. In 2006, NRC psychologists, such as Julius Persensky, criminal and civil penalties. Some plant staff were convicted collaborated with other technical staff to create opportunities in federal court of concealing information from the NRC for inspectors to diagnose and act on safety culture weak about degrading plant conditions. Davis-Besse indicated nesses, a process to determine the need for an evaluation of that a plant could meet the good green column rating of a licensees safety culture, and guidance on evaluating and the ROP performance indicators while not revealing a plant conducting an independent assessment. To Valerie Barnes, culture that prioritized efficiency over safety. The public a recent addition to the NRC staff, integrating safety culture and Congress asked tough questions: Why had the licen into the ROP seemed misguided because NRC inspectors sees safety culture degraded? Where was plant manage engineerswould judge safety culture a job for social ment? Where was the INPO? Where were NRC inspectors? science experts. Nevertheless, she and other human factors Sixteen years after Chernobyl, why didnt the NRC have staff helped establish appropriate protocols in the ROP and a safety culture regulation? the language of the NRCs safety culture policy statement, As a condition of restarting the plant, the NRC which was approved by the Commission in March 2011 just required the licensee perform its own assessment of days before the accident at the Fukushima Daichi nuclear plant safety culture and contract for an independent safety power plant. The final product, she thought, captured the culture assessment. Sonja Haber led a team of consultants important elements of safety culture.109-111 that included Valerie Barnes and veterans of the nuclear navy. Using an updated version of the methodology and survey tools of the Brookhaven study, the team assessed X. CONCLUSION FENOC by widely agreed upon IAEA safety culture characteristics: (1) recognition of the value of safety, (2) The NRCs research and regulation of organizational integration of safety into all plant activities, (3) accoun factors and safety culture has been shaped by the inten table for safety, (4) learning-driven safety, and (5) clear sely adversarial relationship that developed between the leadership for safety. Despite improvements, the team NRC and its licensees in the post-TMI era. Conflict gave concluded, FENOC had made only partial progress social scientists a unique opportunity to influence regula toward those characteristics. FENOC spent several years tion, but it also erected substantial roadblocks. Mishaps in conducting additional safety culture assessments.106 the nuclear fleet and cases of unprofessional behavior The episode opened the NRC to attacks that it was forced the NRC outside of its carefully defined oversight a reluctant convert on safety culture. David Lochbaum of boundaries into the unfamiliar territory of assessing licen the Union of Concerned Scientists and Paul Gunter of the see management competence. Armed with new theories Nuclear Information and Resource Service penned an analy on organizational behavior, social scientists seemed sis entitled NRC has a brain, but no spine. They claimed the poised to demonstrate their practical relevance to the NRC worried more about the financial health of FENOC than engineers who dominated safety regulation. An adversar serving as an independent regulator. The NRC did not fare ial relationship, however, demanded assessment by unam much better on Capitol Hill. Ohio Senator George Voinovich biguous, objective data and methods, a task that did not pressed NRC Chairman Nils Diaz: Why do you disagree play to the qualitative strengths of social scientists.

with everyone that you should put in place a regulation to Applying their insights has required a delicate balancing monitor safety culture? Why do the Europeans do it? You are of quantitative and qualitative factors, even in the current going to be [doing safety culture assessments at] Davis-Besse era of limited regulatory conflict.

for the next 5 years . . .. Why [dont we] make that applicable The NRCs ambition to quantify organizational fac to all the facilities? Echoing Nunzio Palladinos reservations tors remains one of PRAs grand challenges. PRAs still 20 years earlier, Diazs replied, Because it will get into an do not explicitly model their contributions. Disagreement area that the Commission believes that we should not be, exists in how organizational factors are measured and which is managing the facility.107,108 translated into PRAs. There are not yet data demonstrat The sum of operational events since TMI broadened the ing a causal link between organizational factors and NRCs thinking on safety. Its fundamental safety concept, human error or equipment failure rates. If PRAs are to NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

12 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT provide the predictive capability the NRC seeks, these 3. V. BARNES (US NRC, ret.), Personal Communication challenges will need to be overcome.112,113 (Oct. 15, 2019).

The inclusion of safety culture in the NRCs ROP has 4. T. R. WELLOCK, Safe Enough? A History of Nuclear also produced mixed results. It is clear organizational factors Power and Accident Risk, University of California Press, have had important influence for pro-active design and opera Berkeley (2021).

tional improvements that optimize plant response to incidents, 5. T. R. WELLOCK, Engineering Uncertainty and and case studies show they play a role in plant events. Bureaucratic Crisis at the Atomic Energy Commission, Demonstrating their contribution to regulatory oversight has 1964-1973, Technol. Culture, 53, 4, 846 (Oct. 2012);

been more difficult. In 2014, NRC staff found the inclusion of https://doi.org/10.1353/tech.2012.0144.

safety culture in the ROP added little safety value and was 6. T. R. WELLOCK, A Figure of Merit: Quantifying the resource intensive. Another study conducted by NRC and Probability of a Nuclear Reactor Accident, Techno.

INPO human factors experts, however, found an empirical Culture, 58, 3, 678 (June 2017); https://doi.org/10.1353/

link between safety culture and safety performance. The tech.2017.0078.

authors recommended further research to confirm that the 7. R. G. HEWLETT and O. E. ANDERSON, The New relationships they found remain consistent over time.114,115 World, 1939-1946: Volume 1, the History of the United Almost 40 years after human factors experts joined States Atomic Energy Commission, Pennsylvania State the NRC, the goal of quantifying organizational factors University Press, University Park, Pennsylvania (1962).

predictive of safety remains a long-term ambition. How 8. R. G. HEWLETT and F. DUNCAN, Atomic Shield, have social scientists contributed to the NRCs under 1947-1952: Volume 2, The History of the United States standing and regulation of safety culture? As was evi Atomic Energy Commission, Pennsylvania State dent in this story and has been noted elsewhere, the University Press, University Park, Pennsylvania (1969).

social sciences brought to the search for safety an 9. G. T. MAZUZAN and J. SAMUEL WALKER, Controlling unprecedented level of interdisciplinary research and the Atom: The Beginnings of Nuclear Regulation, have promoted key safety culture traits discovered in 1946-1962, University of California Press, Berkeley (1984).

their research, such as an organizational learning cul 10. J. SAMUEL WALKER, Containing the Atom: Nuclear ture, and have established safety culture as a set of Regulation in a Changing Environment, 1963-1971, personal and organizational traits with a logical struc University of California Press, Berkeley (1992).

ture. Human factors specialists have worked with engi 11. J. SAMUEL WALKER, Permissible Dose: A History neers and the industry to articulate a common language of Radiation Protection in the Twentieth Century, for use in safety culture assessments. There is a similar University of California Press, Berkeley (2000).

effort to harmonize safety culture language across mul 12. J. SAMUEL WALKER, Three Mile Island: A Nuclear tiple international organizations and foreign languages. Crisis in Historical Perspective, University of California That common understanding of safety culture may be Press, Berkeley (2004).

their most important contribution. The NRCs engineers set 13. J. SAMUEL WALKER, The Road to Yucca Mountain: The out to turn the cultural concept, safety culture, into the quan Development of Radioactive Waste Policy in the United titative language that PRA specialists understood. Instead, States, University of California Press, Berkeley (2009).

human factors specialists taught the nuclear industry to think 14. J. L. SEMINARS, W. R. GONZALEZ, and and speak of safety culture in the qualitative language that S. O. PARSONS, Human Factors Review of Nuclear can be understood by social scientists and the public.116-122 Power Plant Control Room Design, EPRI NP-309, pp.

2-1 to 2-5, 20-1 to 20-5, 20-12 to 20-25, 21-2, Lockheed Missiles and Space Company (Nov. 1976).

References 15. N. O. FINLAYSON et al., Human Engineering of Nuclear Power Plant Control Rooms and Its Effects on Operator Performance, ATR-77, pp. 6-1 to 7-16,

1. Hearings Before the Joint Committee on Atomic Energy Aerospace Corporation (Feb. 1977).

Congress of the United States, 91st Congress, Second Session on Naval Nuclear Propulsion Program, March 19 16. Evaluation of Incidents of Primary Coolant Release from and 20, 1970, pp. 31, 107-108, U.S. Government Printing Operating Boiling Water Reactors, ML17151A684, U.S.

Office (1970). Atomic Energy Commission (Oct. 30, 1972).

2. J. STUSTER, The Human Factors and Ergonomics 17. Reactor Safety Study, NUREG-75/014, Main Report, Society: Stories from the First 50 Years, pp. 56-57, pp. 162-163, Appendix III, pp. III-59 to III-69, U.S.

Human Factors and Ergonomics Society (2006). Nuclear Regulatory Commission (Oct. 1975).

8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 13

18. Plan for Research to Improve the Safety of Light-Water NUREG/CR-3737, U.S. Nuclear Regulatory Commission Nuclear Power Plants, A Report to Congress of the United (Nov. 1984).

States of America, NUREG-0438, pp. 23-24, 42, U.S.

31. Organization and Safety in Nuclear Power Plants, Nuclear Regulatory Commission (Apr. 12, 1978).

NUREG.CR-5437, p. 9, U.S. Nuclear Regulatory

19. J. STUSTER, The Human Factors and Ergonomics Commission (May 1990).

Society: Stories from the First 50 Years, p. 45, Human Factors and Ergonomics Society (2006). 32. Organizational Analysis and Safety for Utilities with Nuclear Power Plants, NUREG/CR-3125, Vol. 1, pp.

20. B. A. TURNER, The Organizational and 4-6, 13, Vol. 2, pp 1-1 and 1-4, U.S. Nuclear Interorganizational Development of Disasters, Regulatory Commission (Aug. 1983).

Administrative Sci. Q., 21, 3, 378 (Sep. 1976); https://

doi.org/10.2307/2391850. 33. Organizational Analysis and Safety for Utilities with Nuclear Power Plants, NUREG/CR-3215, Vol. 2, p. 7,

21. Turner, Man-Made Disasters, Wykeham (1998). U.S. Nuclear Regulatory Commission (Aug. 1983).

22. J. KEMENY, Report of the Presidents Commission on 34. Organizational Analysis and Safety for Utilities with the Accident at Three Mile Island, pp. 63, U.S. Nuclear Power Plants, NUREG/CR-3215, Vol. 1, pp.

Government Printing Office (1979). 13 and 38, and Vol. 2, pp. 1-4 and 1-5, U.S. Nuclear

23. Three Mile Island: A Report to the Commissioners and Regulatory Commission (Aug. 1983).

to the Public, NUREG/CR-1250, Vol. 2, pp. 920, 1227, 35. An Initial Empirical Analysis of Nuclear Power Plant U.S. Nuclear Regulatory Commission (1979). Organization and Its Effect on Safety Performance,

24. Guidelines for Utility Management Structure and NUREG/CR-3737, pp. viii-ix, A-4, U.S. Nuclear Technical Resources, SECY-80-440, U.S. Nuclear Regulatory Commission (Nov. 1984).

Regulatory Commission (Sep. 19, 1980). 36. A Guide to Literature Relevant to the Organization and Administration of Nuclear Power Plants, NUREG/CR-

25. R. W. JURGENSEN, American Electric Power Service 3645, U.S. Nuclear Regulatory Commission (Dec. 1983).

Corp., to Harold Denton, U.S. Nuclear Regulatory Commission, 8012120504 (Dec. 8, 1980); S. H. 37. Organizational Analysis and Safety for Utilities with HOWELL, Atomic Industrial Forum, to Harold R. Nuclear Power Plants, NUREG/CR-3215, Vol. 2, pp.

Denton, U.S. Nuclear Regulatory Commission, 1, 1-6, U.S. Nuclear Regulatory Commission (Aug.

8012020363, (Nov. 20, 1980); E. E. UTLEY, Power 1983).

Supply and Engineering and Construction Co., to 38. Nuclear Regulatory Commission Meeting, Briefing on Harold Denton, U.S. Nuclear Regulatory Commission, Human Factors Program Plan, 8212230393, pp. 89, 8011250362 (Nov. 20, 1980); P. CRANE, Pacific Gas 94, U.S Nuclear Regulatory Commission (Dec. 14, and Electric Co., to Harold R. Denton, U.S. Nuclear 1982).

Regulatory Commission, 8012150149 (Dec. 8, 1980).

39. Revised Program Element: Management and Organization,

26. Critical Human Factors Issues in Nuclear Power Regulation for the Human Factors Program Plan, SECY-82-462A, U.S.

and a Recommended Comprehensive Human Factors Nuclear Regulatory Commission (Jan. 7, 1983).

Long-Range Plan, NUREG/CR-2833, Vol. 3, pp. 193,

40. Safety Culture: A Survey of the State-of-the-Art, 194-195, U.S. Nuclear Regulatory Commission (Aug.

U.S. Nuclear Regulatory Commission, NUREG-1756, 1982).

p. 27-32 (Jan. 2002).

27. M. V. NADEL, Analysis of Processes Used in Evaluating

41. Celebrating 25 Years of NRCs Principles of Good Utility Management and Organization for an NRC Regulation, ML16048A579, U.S. Nuclear Regulatory Operating License, BHARC-400/82/018, pp. iv-v, 1, 10, Commission (Jan. 19, 2016).

Battelle Human Affairs Research Center (June 1982).

42. Implications of the Accident at Chernobyl for Safety

28. Management and Organizational Assessments:

Regulation of Commercial Nuclear Power Plants in the A Review of Selected Organizations, NUREG/CR-United States, U.S. Nuclear Regulatory Commission, 3601, pp. 4, 28-29, U.S. Nuclear Regulatory NUREG-1251, Vol. 1, pp. 1-21 and 1-23 (Apr. 1989).

Commission (Feb. 1984).

43. Loss of Main and Auxiliary Feedwater Event at the

29. Organizational Analysis and Safety for Utilities with Davis-Besse Plant on June 9, 1985, U.S. Nuclear Nuclear Power Plants, NUREG/CR-3125, Vol. 1, p. 8, Regulatory Commission, NUREG-1154 (July 1985).

Vol. 2, p. 7, U.S. Nuclear Regulatory Commission (Aug.

1983). 44. M. WALD, The Peach Bottom Syndrome, New York Times, Sec. 3, p. 1 (Mar. 27, 1988).

30. An Initial Empirical Analysis of Nuclear Power Plant Organization and Its Effect on Safety Performance, 45. Wake Me if Its a Meltdown, Time (Apr. 13, 1987).

NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

14 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

46. V. STELLO, U.S. Nuclear Regulatory Commission, to J. 61. T. R. LaPORTE, K. ROBERTS, and G. I. ROCHLIN, C. Everett, III, Philadelphia Electric Co., Order Aircraft Carrier Operations at Sea: The Challenges of Suspending Power Operation and Order to Show Cause High Reliability Performance. Final Report, University (Effective Immediately)-Peach Bottom, ML021580019, of California, Berkeley (July 15, 1988).

ADAMS (Mar. 31, 1987). 62. T. R. LaPORTE and P. M. CONSOLINI, Working

47. U.S. Nuclear Regulatory Commission Annual Report in Practice but Not in Theory: Theoretical Challenges 1987, NUREG-1145, Vol. 4, pp. 10, 201-10, U.S. Nuclear of High-Reliability Organizations, J. Public Regulatory Commission (1988). Administration Res. Theory, 1, 1, 19 (Jan. 1991).

48. The Regulatory Craft: An Interview with Commissioner 63. K. E. WEICK, Organizational Culture as a Source of Stephen Burns, U.S. Nuclear Regulatory Commission High Reliability, California Manage. Rev., 29, 2, 112

( 2 0 1 9 ) ; h t t p s : / / w w w. y o u t u b e . c o m / w a t c h ? v = (Winter 1987); https://doi.org/10.2307/41165243.

uAbz9B78tNA&feature=youtu.be (current as of Mar. 3, 64. P. E. BIERLY III and J. C. SPENDER, Culture and 2020). High Reliability Organizations: The Case of the

49. T. MURLEY (US NRC, Ret.), Personal Communication Nuclear Submarine, J. Manage., 21, 4, 639 (1995);

(Oct. 29, 2019). https://doi.org/10.1177/014920639502100403.

50. An Overview of the Evolution of Human Reliability 65. R. A. COOKE and D. M. ROUSSEAU, Behavioral Analysis in the Context of Probabilistic Risk Norms and Expectations: A Quantitative Approach to Assessment, SAND2008-5085, p. 10, Sandia National the Assessment of Organizational Culture, Group Laboratories (2009). Organizational Stud., 13, 3, 245 (Sep. 1988); https://

doi.org/10.1177/105960118801300302.

51. J. REES, Hostages of Each Other: The Transformation of Nuclear Safety Since Three Mile Island, University of 66. W. R. FREUDENBURG, Perceived Risk, Real Risk:

Chicago Press (1996). Social Science and the Art of Probabilistic Risk

52. Nuclear Regulatory Commission Meeting, Briefing on Assessment, Science, 242, 4875, 44 (Oct. 7, 1988);

Human Factors Program and NRC Views of NAS https://doi.org/10.1126/science.3175635.

Recommendations, ML15132A550, U.S. Nuclear 67. Nuclear Regulatory Commission Meeting, Briefing Regulatory Commission (May 19, 1988). on Human Factors Program and NRC Views of NAS

53. Nuclear Regulatory Commission Policy Statement on Recommendations, ML15132A550, U.S. Nuclear the Conduct of Nuclear Power Plant Operations, Fed. Regulatory Commission (May 19, 1988).

Regist., 54, 3224 (Jan. 24, 1989). 68. ASCOT Guidelines, IAEA-TECDOC-743, pp. 12, 14,

54. NRCs Human Factors Programs and Initiatives, International Atomic Energy Agency (1994).

SECY-89-183, U.S. Nuclear Regulatory Commission 69. Use of Probabilistic Risk Assessment Methods in (June 16, 1989). Nuclear Regulatory Activities, Final Policy Statement,

55. N. P. MORAY and B. M. HUEY, Human Factors Fed. Regist., 60, 42622 (Aug. 16, 1995).

Research and Nuclear Safety, pp. 7, 13-18, 21, 80-88, 70. Nuclear-Power-Plant Severe-Accident Research Plant, National Academy Press (1988). NUREG-0900, Chap. 2 and 3, U.S. Nuclear Regulatory

56. N. MORAY, The Good, the Bad, and the Future: On the Commission (Jan. 1983).

Archaeology of Ergonomics, Hum. Factors, 50, 3, 415 71. WELLOCK, A Figure of Merit, p. 678.

(June 2008); https://doi.org/10.1518/001872008X288439. 72. G. E. APOSTOLAKIS, J. H. BICKEL, and

57. N. MORAY, The Human Factors of Complex S. KAPLAN, Probabilistic Risk Assessment in the Systems: A Personal View, Human Factors Issues in Nuclear Power Utility Industry, Reliab. Eng. Syst.

Complex Systems, pp. 11-40, G. R. J. HOCKEY, Saf., 24, 2, 91 (1989); https://doi.org/10.1016/0951-P. NICKEL, and K. A. BROOKHUIS, Eds., Shaker 8320(89)90086-0.

Publishing (2007). 73. G. E. APOSTOLAKIS (U.S. NRC, ret.), Personal

58. 335th General Meeting, 8803170286, pp. 22, 111, Communication (Oct. 28, 2019).

Advisory Committee on Reactor Safeguards, U.S. 74. S. B. HABER, J. N. OBRIEN, and T. RYAN, Model Nuclear Regulatory Commission (Mar. 10, 1988). Development for the Determination of the Influence of

59. N. P. MORAY and B. M. HUEY, Human Factors Management on Plant Risk, Proc. of 1988 IEEE Fourth Research and Nuclear Safety, pp. 7, 80-88, National Conf. on Human Factors and Power Plants, Monterey, Academy Press (1988). California, June 5-9, 1988, p. 349 (1988).

60. C. PERROW, Normal Accidents: Living with High-Risk 75. S. B. HABER, D. S. METLAY, and D. A. CROUCH, Technologies, Basic Books (1984). Influence of Organizational Factors on Safety, BNL-8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021

SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT

• WELLOCK 15 NUREG-44849; https://www.osti.gov/biblio/6663000 Vienna, Austria, April 24-28, 1995, p. 385, American (Aug. 6, 1990). Nuclear Society (1995).

76. Influence of Organizational Factors on Performance 90. Review of Organizational Factors Research, SECY Reliability, NUREG/CR-5538, U.S. Nuclear Regulatory 020, U.S. Nuclear Regulatory Commission (Feb. 1, Commission (Dec. 1991). 1993).

77. S. B. HABER et al., The Nuclear Organization and 91. TOWERS PERRIN, Nuclear Regulatory Review Study:

Management Analysis Concept Methodology: Four Years Final Report, Cover Memo, pp. 3, 10, 29, 56, and Exhibit Later, BNL-NUREG-47667, presented at the Conf. III-3 (Oct. 1994).

Human Factors and Power Plants: Power Generation

92. E. POOLEY, Nuclear Warriors, Time, p. 56 (Mar. 4, 1996).

The Next Decade and Beyond, Monterey, California, June 7-11, 1992; https://www.osti.gov/biblio/7046559-nuclear- 93. P. V. DOMENICI, A Brighter Tomorrow: Fulfilling the organization-management-analysis-concept-methodology- Promise of Nuclear Energy, Rowman & Littlefield, four-years-later (current as of Mar. 3, 2020). Lanham, Maryland (2004).

78. Organizational Factors Research Progress Report, SECY- 94. Request to Discontinue the Diagnostic Evaluation 90-349, U.S. Nuclear Regulatory Commission (Oct. 9, Program, COMSECY-00-0033, U.S. Nuclear Regulatory 1990). Commission (Oct. 4, 2000).

79. NRC Regulatory Information Conference, Washington, 95. ROP Framework, US Nuclear Regulatory Commission; DC, April 18-20, 1989, NUREG/CP-0102, p. 1, U.S. https://www.nrc.gov/reactors/operating/oversight/rop-Nuclear Regulatory Commission (Sep. 1989). description.html (current as of Apr. 20, 2020).

80. Diagnostic Evaluation Team Report for McGuire 96. N. PIDGEON, Safety Culture: Key Theoretical Issues, Nuclear Station, 8804130299, U.S. Nuclear Regulatory Work Stress, 12, 3, 202 (1998); https://doi.org/10.1080/

Commission (Mar. 8, 1988). 02678379808256862.

97. G. APOSTOLAKIS, How Useful Is Quantitative Risk

81. Nuclear Regulatory Commission, Briefing on Assessment?, Risk Anal., 24, 3, 517 (2004).

Effectiveness of the Diagnostic Evaluations, ML15139A402, U.S. Nuclear Regulatory Commission 98. Proposed Options for Assessing the Performance and (Nov. 23, 1988). Competency of Licensee Management, SECY-98-059, U.S. Nuclear Regulatory Commission (Mar. 26, 1998).

82. J. NELSON, NRC Scores Commonwealth Edison for Operating Problems at Zion, Nucleonics Week, p. 8 99. Staff Requirements: SECY-98-059, U.S. Nuclear (Sep. 13, 1990). Regulatory Commission (June 29, 1998).

83. R. ZUERCHER, NYPA Slow to Correct Fitzpatrick 100. Reactor Oversight Process, Handbook 8.13, Part I, p.

Deficiencies, NRC Diagnostic Report Says, p. 3, 4, U.S. Nuclear Regulatory Commission (Jun. 19, 2002).

Inside NRC (Dec. 30, 1991). 101. S. B. HABER and M. T. BARRIERE, Development of

84. D. STELLFOX, South Texas Project 12th Plant to Get a Regulatory Organizational and Management Review NRCs Diagnostic Evaluation, pp. 1, 11-13, Inside NRC Method, AECB Project No. 2.341.2, Atomic Energy (Apr. 5, 1993). Control Board (Jan. 20, 1998).

85. D. STELLFOX, Nuclear Plant Management Ills Prompt 102. Special Expert Group on Human and Organisational NYPA Chief Brons Departure, Nucleonics Week, p. 6 Factors (SEGHOF) State-of-the-Art Report on (June 10, 1993). Systematic Approaches to Safety Management, NEA/

CSNI/R(2006)1, pp. 65, Appendix 3.b, Nuclear Energy

86. S. R. RUBIN, U.S. Nuclear Regulatory Commission, Agency, Committee on the Safety of Nuclear to R. P. La Rhette, 9407190351, U.S. Nuclear Installations (Mar. 16, 2006).

Regulatory Commission (June 1, 1995).

103. N. DURBIN, Review of International Oversight of

87. B. HAAGENSEN, U.S. Nuclear Regulatory Commission, Safety Culture in Nuclear Facilities, Idaho National Personal Communication (Oct. 22, 2019). Laboratory, Battelle Energy Alliance (Mar. 2006).

88. T. G. RYAN, Organizational Factors Research Lessons 104. Management of Operational Safety in Nuclear Power Learned and Findings (1991), ML090710627, U.S. Plants, INSAG-13, International Nuclear Safety Advisory Nuclear Regulatory Commission (Sep. 1991). Group, International Atomic Energy Agency (1999).

89. J. J. KRAMER and S. B. HABER, Organizational 105. Degradation of the Davis-Besse Nuclear Power Station Performance Research at the U.S. Nuclear Regulatory Reactor Pressure Vessel Head Lessons Learned Report, CommissionPast Present and Future, Proc. of the Int. U.S. Nuclear Regulatory Commission; https://www.nrc.

Topl. Mtg. on Safety Culture in Nuclear Installations, gov/reactors/operating/ops-experience/vessel-NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021 8ANS

16 WELLOCK

• SOCIAL SCIENTISTS IN AN ADVERSARIAL ENVIRONMENT head-degradation/lessons-learned/lltf-report.html 114. Effectiveness Review of Cross-Cutting Issues, (accessed Sep. 30, 2002). ML14099A171, U.S. Nuclear Regulatory Commission (Apr. 23, 2014).

106. M. B. BEZILLA, U.S. Nuclear Regulatory Commission, Personal Communication to J. L. Caldwell, Resubmittal 115. S. L. MORROW, G. KENNETH KOVES, and of Redacted Organizational Safety Culture and Safety V. E. BARNES, Exploring the Relationship Between Conscious Work Environment Independent Assessment Safety Culture, and Safety Performance in U.S. Nuclear Report and Actions Plans for the Davis-Besse Nuclear Power Operations, Saf. Sci., 69, 37 (2014); https://doi.

Power Station, ML050660425, U.S. Nuclear Regulatory org/10.1016/j.ssci.2014.02.022.

Commission (Mar. 4, 2005). 116. G. GROTE, Social Science for Safety: Steps Towards Establishing a Culture of Interdisciplinary Appreciation, 107. P. GUNTER and D. LOCHBAUM, Anatomy of Proc. Human and Organizational Aspects of Assuring a Flawed Decision: NRC Has a Brain, but No Spine, Nuclear SafetyExploring 30 Years of Safety Culture, ML031110514, U.S. Nuclear Regulatory Commission Vienna, Austria, February 22-26, 2016, p. 101, (Nov. 15, 2002).

International Atomic Energy Agency (2019).

108. Hearing Before the Subcommittee on Clean Air, 117. NRC Inspection Manual: Guidance for Conducting an Climate Change, and Nuclear Safety of the Committee Independent NRC Safety Culture Assessment, IP on Environment and Public Works. 108th Congress, 2nd 95003.02, ML19066A376, U.S. Nuclear Regulatory Session. May 20, 2004, p. 25, U.S. Government Commission (Apr. 1, 2019).

Printing Office, Washington, District of Columbia (2006). 118. Safety Culture Common Language, NUREG-2165, 109. Historical Review and Observations of Defense in U.S. Nuclear Regulatory Commission (Mar. 2014).

Depth, NUREG/KM-0009, pp. 4-12 and 4-30, U.S. 119. Safety Culture in the ROP, ML18095A029, U.S.

Nuclear Regulatory Commission (Apr. 2016). Nuclear Regulatory Commission (Apr. 5, 2018).

110. Staff RequirementsSECY-11-0005Proposed Final 120. Traits of a Healthy Nuclear Safety Culture, INPO Safety Culture Policy Statement, ML110660547, U.S.12-012, Institute of Nuclear Power Operations (Dec.

Nuclear Regulatory Commission (Mar. 7, 2011). 2012).

111. Final Safety Culture Policy Statement, Fed. Regist., 121. Traits of a Healthy Nuclear Safety Culture, WANO PL 76, 114, 34773 (June 14, 2011). 2013-1, World Association of Nuclear Operators (May 112. S. PETERS et al., Organizational Factors in PRA: 2013).

Twisting Knobs and Beyond, Proc. 2019 Intl. Topl. Mtg. 122. S. B. HABER, From Safety Culture to a Culture for Probabilistic Safety Assessment and Analysis, Charleston, Safety: What Is It that We Still Have Not Learned?

South Carolina, April 28-May 3, 2019 (2019). Proc. On the Human and Organizational Aspects of 113. S. TINA GHOSH and G. E. APOSTOLAKIS, Assuring Nuclear SafetyExploring 30 Years of Safety Organizational Contributions to Nuclear Power Plant Culture, Vienna, Austria, February 22-26, 2016, p. 219, Safety, Nucl. Eng. Technol., 37, 3, 207 (June 2005). International Atomic Energy Agency (2019).

8ANS NUCLEAR TECHNOLOGY

• VOLUME 00

• XXXX 2021