ML22133A229
| ML22133A229 | |
| Person / Time | |
|---|---|
| Site: | Nuclear Energy Institute |
| Issue date: | 08/30/2021 |
| From: | Weinisch K, Young D KLD Engineering, PC, Nuclear Energy Institute |
| To: | Office of Nuclear Security and Incident Response |
| Hoffman R | |
| References | |
| EPFAQ 2021-001 | |
| Download: ML22133A229 (6) | |
Text
Emergency Preparedness Frequently Asked Question (EPFAQ)
Page 1 of 5 8-9-21EPFAQ Number: 2021-001 Date Accepted for Review:
Originator: David Young / Kevin Weinisch Organization: NEI / KLD Engineering Relevant Guidance: NUREG/CR-7002, Criteria for Development of Evacuation Time Estimate Studies Applicable Section(s): Section 4.3 Status: Out for Public Comment QUESTION OR COMMENT:
Background
Section 4.3 of Revision 1 of NUREG/CR-7002 (ML21013A504) dated February 2021 states:
Preparation activities, including the time to receive the notification and time to prepare to evacuate, are developed as elements of the trip generation time, sometimes referred to as mobilization time Each population group has different considerations for trip generations times. Surveys of residents within the emergency planning zone (EPZ) are commonly used to collect some of the data needed to develop the time distributions. When a licensee relies on surveys, the licensee should describe the survey method (e.g., telephone, interview, online poll, etc.), number of participants, and statistical relevance. Trip generation times base[d] on a sample survey of residents are subject to statistical uncertainty. Additionally, temporary conditions within the EPZ (e.g., frequent natural hazards, public health emergencies) may lead to skewed survey results. To minimize the uncertainly in the trip generation time, updates to the evacuation time estimate (ETE) may use new and previous sample data for the various population groups and situations that influence the trip generation time. Trip generation times and evacuation logistics should be developed based on assumed normal activities and operations. The data used to develop trip generation times should be summarized in the ETE study. The licensee should describe the methods used to minimize uncertainty in the development of trip generation times.
Trip generation time becomes an increasingly important component of the ETE as the evacuation demand decreases (NRC, 2020). The ETEs for the transit dependent residents, special facility residents, and schools are each developed separately from those for the general public. The logistics of the trip generation times for these groups can be complex and may affect the ETE more than the actual travel time out of the EPZ.08/30/2021 Emergency Preparedness Frequently Asked Question (EPFAQ)
Traditionally, licensees have employed random sample telephone surveys within the EPZ to gather demographic data and mobilization times in support of ETE studies. However, the percentage of U.S. adults living in a home with an operational landline telephone has declined from 90% in 2004 to less than 40% in 2020, according to data gathered by the Centers of Disease Control and Prevention (CDC).1 Conducting telephone surveys has become increasingly difficult and prohibitively expensive2 given the rapid decline in landline telephone use and the frequency of spam and telemarketing phone calls to cell phones. Many licensees are now relying on online based surveys to gather the data needed for trip generation distributions. Licensees are working closely with Offsite Response Organizations (OROs) to advertise the survey website and survey purpose to people living and working in the EPZ.
Several licensees are having difficulty realizing a statistically relevant sample size despite a good faith effort from the licensee and/or OROs. This is especially true for low population sites, which have fewer households to complete the survey. The margin of error is relatively unchanged by number of households based on the nature of the statistical equations. As shown in Table 1 and Table 2, an EPZ with only 5,000 households requires approximately the same number of completed surveys as an EPZ with 30,000 households to achieve a similar margin of error. This is especially problematic, as the guidance points out, as low population sites have low evacuation demand and ETE closely parallel trip generation/mobilization times due to limited traffic congestion during evacuation.
Table 1. Sample Size and Margin of Error for a Hypothetical EPZ with 30,000 Households
Sample Size Margin of Error 100 9.78%
200 6.91%
300 5.63%
400 4.87%
500 4.35%
Table 2. Sample Size and Margin of Error for a Hypothetical EPZ with 5,000 Households
Sample Size Margin of Error 100 9.70%
200 6.79%
300 5.49%
400 4.70%
1 https://www.statista.com/chart/2072/landline-phones-in-the-united-states/
2 One licensee reported recently spending $32,000 on a telephone survey for non-ETE purposes which achieved a sample size of 400 households (approximately 4.9% margin of error). An ETE survey with this sample size cost approximately $10,000 in 2010.
Page 2 of 5 8-9-21 Emergency Preparedness Frequently Asked Question (EPFAQ)
500 4.16%
Question 1
What does the NRC staff consider to be an acceptable margin of error and/or confidence interval for a survey conducted in support of an ETE study?
Question 2
What steps should a licensee take if the recommended margin of error and confidence level cannot be achieved despite good faith efforts by the licensee and the OROs?
PROPOSED SOLUTION:
Proposed Solution to Question 1
Acceptable confidence intervals and margins of error are as follows:
Confidence Interval Margin of Error 90% Less than or equal to 5%
95% Less than or equal to 8%
Proposed Solution to Question 2
One or more of the following alternatives can be used:
Expand the survey sample area by accepting survey responses that are from beyond the study area but within a certain distance from the plant (i.e., including survey responses from within 30 miles of the plant, etc.).
Use the telephone survey results from the ETE study conducted for the site after the 2010 Census, assuming the sample size and margin of error were acceptable.
Perform a statistical analysis of telephone survey results over the last 20 years for more than 50 nuclear power plant ETE studies and estimate mobilization times for suburban and rural sites, then apply to the site depending on whether the site is suburban or rural.
Apply the survey results from a similar plant (similar population, geographic area, etc.) that did achieve an acceptable margin of error.
Perform a sensitivity study on mobilization time to account for the margin of error from the survey. For example, if the hypothetical EPZ with 30,000 household gets 200 completed surveys corresponding to an approximate 7% margin of error, vary the mobilization time to account for the margin of error (mobilization times are comprised of several different time distributions, so it is not simply a 7% error) and determine the change in ETE.
Page 3 of 5 8-9-21 Emergency Preparedness Frequently Asked Question (EPFAQ)
NRC RESPONSE:
Question 1
The guidance in NUREG/CR-7002, Revision 1, states that when surveys are used to collect data for trip generation times (TGTs), the licensee should describe the survey method, number of participants, and statistical relevance. The guidance does not prescribe a specific margin of error or confidence interval to be met. The NRC recognizes that survey data will always contain some level of uncertainty (e.g., aleatory uncertainty due to probabilistic variability, which is irreducible). Such statistical errors are unavoidable, and a large margin of error in survey data, on its own, would not indicate that results of the ETE study would be unacceptable. This occurs primarily because, for many sites, variation within the ETE scenarios, such as time of day and day of week, should account for a larger expected variation in the ETE than the statistical error in a sample survey of mobilization times. The guidance in NUREG/CR-7002, Revision 1, states that licensees should describe the statistical relevance of the underlying survey data so that the uncertainty is understood and considered in the development process and interpretation of results.
Question 2
The guidance in NUREG/CR-7002, Revision 1, recognizes the uncertainty and limitations introduced by survey methods used to develop TGTs and provides licensees with guidance on ways to minimize uncertainty in the TGT. Section 4.3 describes one acceptable method to minimize uncertainty by including the survey data from prior site-specific ETE studies. The guidance also states that licensees should describe the methods used to minimize uncertainty in the development of TGTs.
The NRC staff finds that methods to minimize uncertainty based on supplementing site-specific survey data with additional site-specific survey data are acceptable. However, the survey area should generally not extend beyond the extent of the analysis area for the traffic simulation model.
Because ETEs are site-specific, the NRC staff finds that the use of non-site-specific data as a basis for developing site-specific TGTs is not acceptable. However, the use of non-site-specific data to minimize uncertainty in the site-specific survey data or to assess the uncertainty in the site-specific survey data and its potential impact on the ETE, is acceptable.
The NRC staff finds that sensitivity studies are acceptable for inclusion in ETE studies as a method to describe the impact of the level of uncertainty in the underlying survey data. Similar such sensitivity studies are used to examine the variability in other parameters important to ETE studies as described in NUREG/CR-7002, Revision 1.
Page 4 of 5 8-9-21 Emergency Preparedness Frequently Asked Question (EPFAQ)
RECOMMENDED FUTURE ACTION(S):
INFORMATION ONLY, MAINTAIN EPFAQ
UPDATE GUIDANCE DURING NEXT REVISION
Page 5 of 5 8-9-21
ML22133A229 OFFICE NSIR/DPR/RLB NSIR/DPR/RLB NSIR/DPR/POB OGC/GCRPS/RMR
/NLO NAME RHoffman RH JQuichocho TSmith TS HBenowitz HB MNorris for MN DATE May 17, 2022 May 17, 2022 May 24, 2022 May 24, 2022 OFFICE NSIR/DPR NSIR/DPR/RLB NAME RJohnson RJ RHoffman RH DATE May 27, 2022 May 31, 2022