ML20247H041
| ML20247H041 | |
| Person / Time | |
|---|---|
| Site: | Seabrook  |
| Issue date: | 04/25/1989 |
| From: | Dillman D, Moriearty S MASSACHUSETTS, COMMONWEALTH OF, WASHINGTON STATE UNIV., PULLMAN, WA |
| To: | |
| References | |
| OL-I-MAG-079A, NUDOCS 8905310142 | |
| Download: ML20247H041 (31) | |
Text
, -
~ ~ ~ ~
' * ^
8.
W sk/
. UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION DCCKrir',
Uwc ATOMIC SAFETY AND LICENSING BOARD
'89 My 23 pg;,9 Before the Administrative Judges:
OFFlu n; e Ivan W.
Smith, Chairman 00CXr t i, SR!$cq (W.
Dr. Richard F. Cole Kenneth A. McCollom L
)
In the Matter of
')
Docket Nos. 50-443-OL
)
50-444-OL i
PUBLIC SERVICE COMPANY
)
(Off-Site EP)
L OF NEW HAMPSHIRE, EI AL.
)
)
(Seabrook Station, Units 1 and 2)
)
April 24, 1989
)
CORRECTED TESTIMONY OF DR. DON A. DILLMAN AND SHARON MORIEARTY ON BEHALF OF THE ATTORNEY GENERAL OF THE COMMONWEALTH OF MASSACHUSETTS REGARDING JI CONTENTION 48 NUCLEAR REGULATORY COMMISSION Dor.':ct !b. AWM9 -01. O!!icial Exh Jh "49 3 -C '
in the c:U.et ci d 19 h-! -
Y st:0 __._._
m M D E E RcLJ RD ______
j Ay -.
V
. R M CTED SO US -
\\
wa 7-W r Ibacp__.DATE.
m a.
R6p0Titf w.w 8905310142 890425
{DR ADOCK 05000443 E
PDR a
i l
~ - - - -
_________________&___*)_
4
SUMMARY
OF TESTIMONY The following testimony concerns the inadequacy of the survey which was conducted to identify the special needs residents.
Dr.
Dillman, an expert on mail and telephone surveys, like the one at issue, analyses and critiques the content, form and manner in which the survey was conducted.
In short, the survey components he finds lacking are the sources from which the initial mailing list wa
.rawn;-
the terminology and language held in the cover letter and questionnaires; the method for " verifying" the survey results; and the resulting tally of targetd individuals.
Dr. Di11 man's testimony also sets out the extent to which the common errors of non-coverage, measurement and response rate, all of which must be considered when conducting surveys of this type, are evidenced in the sub;oct survey, Bo.th Dr. Dillman and Ms. Moriearty believe that the open-ended nature of the survey questions will not be conducive to accuracy.
Ms. Moriearty approaches the survey from the perspective of a demographer with practical experience in assessing needs for the disability community.
Her testimony includes a brief description of the kind of information this type of survey should have focused on.
She discusses the importance.of addressing functional needs in order to determino appropriate assistance.
She also gives a comprehensive discourse on the 1985 United States Census,arear s'rvey which focused on function limitations.
It is from this documtat chat Ms. Moriearty extrapolates her estimation of what a representative number of disabled individuals needing assistance would be in the EpZ.
l
q :.,
- a+
ne.
E 4J I
~
Bothierparts believe'.thatithe,results-of the survey ~.are'not o
Representative of theinumber of special needs people,who might.need.
assistance during,an: emergency.
Dr. Dillman shows this by using.the-
'l
' survey. results Eas~ a sainple representation.
gr TESTIMONY
?"
Q..-pl' ease, each state your'name and briefly describe'your professional-qualifications.
A.
(Ms. Moriearty)
My name is'Sharon Moriearty.
I.am the t
Deputy Director of the Massachusetts Office c f Handicapped Affairs' (MOHA).
'I'h9ve acted in this' capacity since April, 1986..As deputy director of MOHA,'I am responsible for
. compiling relevant research and demographic information.
regarding-people.with disabilities.
In this capacity, IJ am familiar with a variety of survey methodologies and
,v
[
classification schemes used to characterize disability in
't the general population.
These include surveys sponsored by the' federal government', such as census, health interview and housing surveys, as well as self-identification-
. instruments used by private' polling. concerns, and classification methods used by the medical profession and state and federal regulatory agencies.
In addition, through my supervision of the Consumer Services Program, MOHA'provides services and assistance anually to approximately five. thousand individuals with varying levels I
h' and types of disability.
A copy of my resume is attached.
Additional information on my professional qualifications and my fourteen years experience in the disability field is contained in the testimony I filed regarding JI 49. ;
i
____ __ _ _ m
m
[
A:
(Dr.1Dillman)f.My'name is Don A. Dillman and I have a ph.D.
J,'
a
!? mc "
l in-Sociology from' Iowa State University.. I_am presently j
Director ofithe Social and Economic Sciences Research Center e
(SESRC)'and-Professor inr the Departments of' Sociology-and.
Rural Sociology at Washington State' University.
I am an authority on the design and implementation of mail and telephoneLaurveys.
My book Mail and Telechone Surveys:
The Total Desian~ Method (1978) was the first book ever
. ublished'which provided detailed procedures for obtaining p
n high quality and quantity of response to mail and' telephone surveys, and it still regarded as a leading text in the field.. The research center I direct conducts approximately
- 40 survey-related projects each year utilizing mail, telephone, and/or face-to-face. interviews.
I1am also the author of more than 90 publications, most of which concern the implementation or interpretation of survey information, i.ncluding the following which are of the most relevance to the' evaluation of'the proponent's surveys:
" Increasing Mail Questionnaire Response for Large Samples I
of,the General Public" (1972).
"Towards the Assessment of Public Values" (1974).
"The Contribution of Personalization to' Mail Questionnaire Response as an Element of a Previously Tested Method" (1974).
" Increasing Mail. Questionnaire Response:
A Four State Comparison" (1974).
" Decreasing Refusal Rates for Telephone Interviews" (1976).
"Our New Tools Need Not Be Used in the Same Old Way" (1977).
"Research Ethics:
Emerging Concerns for the Increased Use of Mail and Telephone Survey Methods"
~ -
eg.:
(1977).
" Mail andlSelf-Administered Surveys" (1983).
"The importance:of. Adhering to Details of the Total Design Method for Mail'Snrveys" (1984).
"You have Been Randomly Selected.
. Survey Methods for the Information Age" 3:
(1985).
" Elements of Success (in Needs Assessment Surveys)" (1987).
" Administrative Issues in Mixed Mode p
Surveys" (1988).
I have twice served as a guest professor at the German Center for Survey Methods and Analysis in Mannheim, West Germany.
I have lectured there on survey issues as well as in-the Netherlands and Great Britain, and at numerous universities.in the United States.
In 1988, I served as instructor for a course on mail and telephone survey
' techniques in the University of Michigan's Survey Research Center's Summer Institute.
I have conducted numerous technical seminars on the conduct of surveys for such groups as.the U.S. General Accounting Office, and Inspector General's Office of the U.S. Department of Health and Human Services, the staff of the Washington State Legislature, the Ohio State Department of Mental Health, the 1890 Land Grant Colleges and Universities, the Marshall University College of Education, and the American Statistical Association.
My curriculum vita is attached.
i Q:
Dr. Dillman, what' experience have you had in conducting surveys and gathering statistics?
i 1.
i
'A:
(Dr. Dillman)
The social and Economic Sciences Research Center (SESRC), which I direct, is one of the few university survey research centers in the United Staces with an established capability for designing and implementing mail, telephone, and face-to-face interview i
surveys.
The Center conducts approximately 40 survey projects per year.
I have designed, implemented, or otherwise worked on surveys utilizing each of these survey methods since 1964.
During these 25 years I have designed, implemented, or provided consultation on well over 1,000 surveys for myself and/or other people.
The Center I direct has a research program in which we conduct several experiments each year on means of improving mail and telephone surveys.with respect to both the quality and quantity of response.
Q:
You each have expertise on a different aspect of the subject survey.
Accordingly, I may address certain questions to one, certain question to the other.
Alright?
A:
(Both)
That is fine.
Q:
Are you familiar with the manner in which the Applicants have gathered the information regarding the special needs residents?
)
A:
(Dr. Dillman)
Yes, I have reviewed some of the materials l
provided by the proponent.
In addition, I have read the Applicants' answers to Interrogatories which bear on this issue.
Copies of these items are attached to the Appendix to this testimony.
Briefly, it is my understanding that the mail survey primarily consisted of a form letter and one-page questionnaire card which was folded and sealed and - _ _ _ _ - _ _ _.
y
-m was then sent to residents of the selected communities whose l,
names and addresses.were taken:from utility listings or voter registrations.
Respondents filled out this questionnaire and returned it'.
A second questionnaire was apparently sent to households which had not responded.
I have reviewed the questionnaire and the cover letter.
It is my understanding L
that an. attempt was made to survey.by telephone those o
respondents who indicated that an " impaired" individual resided in their household.
I have reviewed the
" verification" questionnaire which was apparently used for this follow-up, but it does not include a very detailed description of the methods for contacting individuals nor does itfinclu'e information about how many people were d
actually interviewed.- Apparently, the main purpose of these surveys was'not to garner a sample of the resident special needs population but to identify those individuals living in
~
the'EpZ, by name and address, who were " impaired" such that they would need special assistance in the event of evacuation.
i A:
(Ms. Moriearty)
I am familiar with the contents of the questionnaires which were sent to households in the survey area in an attempt to identify and verify special needs residents in the EpZ.
I am also familiar with section 3.6(c) of the SpMC which states that additional sources of information used "to identify handicapped individuals" are mailings of special needs posters to a variety of organizations in and around the EpZ.
There was also a
" response card," geared to residents having a disability or special needs, which was to be filled out and returned. - - - - _ _ _ _ - _ _ - - _.
i#
s
-1 L
The method-for eliciting the desired information is a tiries of close-ended questions which do not provide 1
E
(
adequate.information about individual needs.
The questions
.I g
seem-to. assume that transportation-related needs are the only needs a person might have and the questions' isolate j
very extreme medical. conditions which are less likely to be found'in a community setting. :In my opinion, this will-q L
translate into an incomplete and inaccurate set of answers-In my opinion, the questionnaire seems dasigned to be very exclusive.
b Q.
Before we go any further regarding this survey, can you both provide any comments you had with respect to the special needs poster?
A.-
(Dr. Dillman) The most this poster does, indeed what it is-intended' to do, is encourage people to call in order to.be surveyed.
Presumably, the same type of survey would ensue for those who called in as for those who responded by mail.
As such, the same types of problems with respect to the subject survey would apply.
I would like to add that it was not indicated in the documents I reviewed how many-
';argeted individuals, if any, were able to be surveyed as a result of the poster.
As such, its' worth in terms of augmenting the identification process, can't be ascertained.
(Ms. Moriearty) In terms of the poster being a means of l
[
augmenting the information already derived from the mail-in I
l survey, it should be noted that people with disabilities are often very reluctant to self-identify for reasons l /
1 l
1
g.
y 4f a
j
[
related to' confidentiality.
In-addition, without the appropriate prompting and questioning geared towards elicitingfinformation on financial needs and limitations, some disabled; individuals do not even recognise themselves s
as such.
I discuss-this again at a later point.
People with certain kinds of. visual impairment would have to have posters read to them.
Q.
.Thank you, Ms. Moriearty, how would you determine whether a
.special needs assessment survey was in fact representative of the resident special needs population?
A:
A complete survey should document both intrinsic and extrinsic factors which affect an individual's ability to function.
In addition to information on the nature of the disability and its functional manifestations, such a survey would include information on en.ironmental factors, such as social and economic resources and physical accessibility of the environment, the level of technological support available to the person with a disability, such as professional assistance or adaptive-equipment, is also critical.
The individual significance of these factors, as well as their interrelationships, varies with each individual.
When discussing conditions that entail emergency response, a fourth factor must be added -- that of unusual environmental risk.
Situations inducing high stress, such as a nuclear disaster might produce, can exacerbate the conditions of disability in people with vulnerable physiologies, rendering a person less functional than an in situ survey might represent.
8-
.m l
A l
)
~
Q:
Dr. Dillman, how do you view the accuracy of surveys
)
conducted to identify the special needs residents?
)
1 A:
(Dr. Dillman)
Very inadequate.
In my judgment _a' survey.
i conducted in the manner described in the materials provided to me cannot possibly be relied upon to identify the names and addresses of all or even a majority.of the individuals who. live in the specified geographic area and are
" impaired" to the degree that they would require assistance during an evacuation.
The inadequacy of this survey stems from a number of specific deficiencies which when considered together compound the overall inadequacy of the survey for the purpose of identifying all " impaired" individuals living in the study area..
In. order to produce' accurate results in a survey of this type, three types of potential survey errors must be addressed in order to' be assured that results as accurate as possible'are obtained.
They include:
noncovergae error
~
(making sure every household in the population has the opportunity to be surveyed); measurement error (being sure that the desired information is accurately obtained); and I
resoonse rates (obtaining responses from all or nearly all of the households in the population).
A fourth category of survey error - samolina error -- does not apply to this particular survey inasmuch as the objective was to survey all households rather than a sample of them.
Q:
Dr. Dillman, can you describe how the aforementioned errors are evidenced in the SpMC survey? 1 1
i J
l A:
(Dr. Dillman) Certainly.
I'll go through the potential survey errors one at a time.
d NONCOVERAGE ERROR)
The list of households was obtained from utility billing records or voter registration lists.
I do not have specific knowledge of these lists for these communities.
However, in most communities some utility bills are sent to owners, or a third party, rather than to the occupant.
This is especially true for rental units.
Oftentimes the l
utility bills are included in the rent.
The reasons for this situation varies, but sometimes it happens that landlords want to be absolutely sure that utility bills get paid and that such things as heat do not get turned off in the winter when damage might result to property.
In addition, an effect or correspondent circumstance accompanying the existence of an impairment.is a lower income potential.
Thus, it seems likely that " impaired" people would more likely than the average person to live in rental housing and would therefore be more likely to remain unidentified through use of utility billing records.
l l
A:
(Ms. Moriearty)
Mr. Dillman is correct in his supposition that people with disabilities are much more likely to be in lower income brackets.
Indeed, people with disabilities are twice as likely to live on poverty incomes as other !
i l
.4:
Americans.
As individuals who are twice as likely to be homeless, many do not'have a permanent address. _ persons with' lower incomes also move more frequently.
A: _
(Dr. Dillman)
S'imilar problems attach'with reliance on voter registration lists.
Only a portion of the households in any. community contain registered voters.
Typically, registered _ voters are inore likely to have higher incomes and greater education.
It is possible that certain types of." impaired" people are less likely to be registered voters and therefore not be included on or identified by the voter registration list.
In both the above cases I believe a bias would exist against households in which " impaired" people live.
A:
(Ms.nMoriearty)
Again, Mr. Dillman's conclusions are
(
correct.
Indeed,.many polling places and voter registration facilities are not even accessible to people with certain disabilities.
A:
(Dr. Dillman)
Another noncoverage problem concerns addresses.
It is apparent from the results of the special needs survey that a substantial number of the households in each of the communities had " bad" addresses.
The percentage of mail-outs classified as having bad addresses range from 7.1 percent to 24 percent.
This suggests that something is wrong with the mailing list.
In the case of the voter registration list, it is possible that frequent updatings are not,made.
In the case of the utility list it is less clear where the problem of " bad" addresses might lie.
In any event, it is clear that a problem exists. - - _ _ - - _ = _
m-9' I..
Q:
What uis the overall effect of the noncoverage problem?
A:
(Dr. Dillman)' 'The overall.effect, I believe, is that an unknown'but significant number of households-did not have'a chance to be surveyed and there is reason to believe that
'these households are more likely than others to contain
" impaired" residents.
Q:
Can you expound a bit on the " Measurement Error" portion of your analysis?
A:
_(Dr. Dillman).In many respects the measurement error problem restates the problem which Ms. Moriearty flagged in I
her earlier comment on open-ended questions.
By addressing the measurement error p'roblem, you are ensuring that the information which you are seeking to obtain is, in fact, the information which you elicit.
Contrary to this purpose, the questions asked in the questionnaire are vague and use terms'that are not defined.
Moreover, the reliance on open-ended questions (as opposed to close-ended questions which give answer categories to choose from) presents' additional and special problems.
Q:
What are those problems?
A:
(Dr. Dillman) The first problem is with the use of the word evacuation.
This word can refer to being evacuated out of a building or evacuated out of a geographic region.
By mentioning several disasters such as a tornado, hurricane, or earthquake, the likelihood is increased that the respondents will think of the word evacuation as meaning help in getting'outside of the bu.ilding in which they live.
Certain " impairments" such as not having upper body :___-____
g y
'y;
~
L.
L*
]
e
- s
' strength 1or not'having useLof.a-. limb could'. prevent getting oneself out"oftthe debhis of-'a building in the wake of,a d
natural disaster, yet that' person might still;be-able to drive Lhisicar... Conversely,. a person might be able to get-in:or out' of.any buildingLbut'might have a type of f
Limpairment which would prevent her from driving a car.
Nowhere'inlthe cover letter or' questionnaire is it clarified as to what kind of evacuation is being asked A
about.
pl The second problem is with.the word " impaired".. The letter w
p refers to physically impaired.
The first questions asks "is anyone in your. household impaired to'the-degree-they.
would require assistance during an evacuation?"- Certain kin's of " mental" impairments might not be a problem for d
evacuating a building, but could be a serious problem for
' leaving a geographic area.
This question needs to have included some specification as to what kind of impairment b
is atLissue.
Also, " impaired" is a word not familiar to or understood by many people.
No' definition'or context is
.provided.
The combination of vagueness with respect to the word impairment in conjunction with similar vagueness on evacuation are fatal flaws in the questionnaire.
For these reasons alone the results of the survey cannot be trusted.
A':
(Ms. Moriearty)
This is a crucial point Mr. Dillman is making.
In my experience, traditional surveys of i
disability show enormous error for the reasons Mr. Dillman has mentioned.
For planners like myself, who are involved L l
.my
- (.ff Y
s i
"hg Pi cr
',f swith[thefday-to-day practical realit'ies of' delivering services to this population, traditional methods of y
lcountingLand! classifying disability, fin particular the use.
"ofe.medi'ca11 taxonomies, can;bel misleading and incomplete.
~
IFirst,l diagnostic. classifications, which are based on-medical etiologies and therapies, tend to yield significant underestimations of the target' population, because their application depends on cultural parameters'as well as treatmentEtaxonomies.
For' instance, in a 1978 U.S.
. government health interview survey, a diagnostically. based
' tool!which yields"an: estimate of disability wh'ich is higher-x than"t'he 1980 census but. lower ~than the 1986 census report on disability, 273 Massachusetts citizens-in' wheelchairs 4:
identified themselvesLas."without a limitation" and 4640 identified-themselves as "without a major limitation."
Errors of both over and under self-identification on
~
diagnostically-based surveys can be magnified'to the extent l:
that;a disability is-mediated by compensatory environments or: technology, to the entent the disability has not been diagnosed, has not been successfully treated, or to the extent that the diagnosis is not understood by the layperson.
Second, these classifications tell us little about-the actual functional limitations a person might p,
have.
For instance, a neurological disorder can have 1
greatly varying functional manifestations which cannot be l-L-
generalized across a common diagnostic category.
A person L
with cerebral palsy may use a wheelchair or be fully
! L
\\
.~
Er ambulatory, or may be severely speech-impaired or fully articulate.
t
. A person with diabetes may be severely n.
t
' mobility-impaired or experience minimal disruption of major life activities.
The primary functional disability of a I
person with a respiratory disorder or a heart condition could be mobility impairment or less disruptive dependence j
on a' pharmaceutical regimen.
Third, diagnostic classification schemes omit critical information regarding the individual's environment while the severity of a disability is also a function of environmental and technological support.
A person who is deaf and fully equipped with appropriate communication technology may not consider herself' " disabled."
If, however, this support were suddenly removed, she would be vulnerable.
Environmental needs also include social and economic dimensions.
A poor person who.ip disabled is less able to-compensate for that disability and navigate in'a community setting than a wealthy person with a disability.
The advantages of the most recent U.S. Census survey (1985) over methods which are not geared specifically to practical needs are that the census survey is a more intensive, open-ended inquiry and uses functional criteria which illuminate the practical ramifications of disability in day-to-day life.
I have attached a copy'of this survey in the Appendix.
The questions inquire as to practical areas of daily activity using terminology and concepts which are I
more universally. understood within a lay community, such as !
i h
c'
[.
"can you climb a flight of stairs without assistance?"
In addition to its focus on a practical issue, this kind of question has the ability to weed out,, or at least minimize, environmental and technological compensations which the interviewee may take for granted.
The survey explores several different dimensions of daily activity in this way, including activities of seeing, hearing, speaking, walking, carrying, lifting, climbing, personal care, and general interior and community mobility.
It distinguishes the severity of limitation in each of these categories and includes a more detailed examination of those who need actual personal assistance as well as important social and economic information on the total population identified as disabled.
I believe its application to a situation like emergency planning for a general populati.on is unusually apt, both because of its practical focus and because it also enables a planner to estimate the nature and extent of needs within this population in the event that social, environmental, or technological supports are suddenly disrupted or. disabled.
A:
(Dr. Dillman)
A more general problem with the questionnaire which goes back to Ms. Mcriearty's points on open-ended questions and diagnostic classification is that it places the burden upon the respondent to figure out what kind of impairments, disabilities, or handicaps should be reported.
Apparently, it's also up to the respondent to l
figure out and report what kind of " assistance" he would need.
The " verification" process seems to have been only i f 1
_______-___________a
m.
m
)
'l I
used to disqualify people from the category of " disabled j
s requiring assistance"..It does not appear that effort-was devoted to verifying the "not requiring assistance" status-1offthe remainingLrespondents.
In other words,.the
. verification process was on'ly used to pare down the number of those.who were already identified as impaired rather than endeavoring to identify those in the impaired population who claimed they didn't need assistance, a claim which seems to-have been made in the context of a lack of informedness'as to what type of assistance would be required relative to the particular impairment and the particular protective action.
A substantial proportion of respondents mentioned "other".
The telephone verification form goes into more detail-than
~
'the original questionnaire and in that respect addresses "other" isspes that are not likely to have been raised at the initial survey level.
Of course, the obvious point to make at this juncture is that this verification procedure only " verifies", in an imcomplete manner, an already j
incomplete. survey result.
Still another problem with the survey questionnaire is that the request for telephone number does not indicate whether the desired telephone number is for the " impaired" person (who may hold down a job) or for the person who filled out l
the questionnaire.. Simply put, the answers to this questionnaire cannot be relied upon for the intended l
purpose of the questionnaire. L_ - -- -
N J.-
_ e
.,:e i
Q:
Dr. Dillman,'can you' discuss the;" response rate" factor?
E "A :
(Dr.' Dillman)
Cartainly.
The reported response rate of 29 i
u.
percent is11ess..than half of what one would expect from a
- wel'l-done mail survey. -There are~ several f actors which L
probably contributed to the' lower response rate.
'First,tthe' vagueness of the questions is.very likely to' I
'have' confused'a number of people.
As a-result, people who Lmight have been1otherwise willing to fill:out th'e 4
questionnaire perhaps felt that the results would not be useful and therefore~that it was.a waste of time to complete.the questionnaire.
~
Second, the cover letter was in itself very confusing.
It came from New Hampshire according to the letterhead, but concerned evacuations in Massachusetts.
The letter informs the readerithat the' survey seeks to identify "both.
physicallyiimpair'pd and non-impaired residents."
What is a
" physically non-impaired resident"?. Is that personLthe target'of-the survey?
The. choice of words here is extremely poor.
The letter tells the reader that the information will be used by " local'and state emergency management agencies and public Service of New Ha'mpshire in th event of an emergency to provide assistance to our
. physically impaired neighbors."
Again, the language is poorly chosen'and the sentence poorly constructed.
Why is i
the reader asked for information to help "our" neighbors?
If the primary point of the survey is to get accurate 1,
1 1
r$se a;
g
-information as to the various disabilit'ies'and impairments.
l t
of members of the-household, the letter which focuses on "ohysically impaired" and " neighbor", undermines the purpose of the survey itself.
A third problem with the letter is that it was a form
.rather'.than>" personalized" letter.
In my experience, this I
fact alone can be expected to lower the overall response rate.
e..,
Fourth, the respondents were. asked to disclose information which many would consider:quite personal and confidential.
.There was,nothing included in the letter about how this
-information would.be handled and who would have access to
.it.
In fact, the letter tells people right away that the information is'to be disclosed-to governmental authorities.
For some or all of these reasons, potential respondents needing assistance.may have decided not to return the information.
This problem may have been increased by the
' fold-over. card not being sealed on the ends (if I understand the structure correctly) so that not even a-l sealed envelope was available to protect the confidentiality of the personal information.
A:
.(Ms. Moriearty)
Due to the experience of social discrimination which remains their. number one barrier to assimilation,. disabled people are very reluctant to reveal personal information regarding a disability.,
s
ye i
I tr
(
[- 5. 3
..g f,:
-(Dr. Dillman')..It.should also be noted that the question on l.
the survey which reads, "should we have to verify any of
.[
this information, when'would be.the best. time to call?"
couldlhave a. chilling effect onl response insofar.as some
. readers could construe it as a check on their~ honesty.
Q:
.Dr. Dillman,_can you. sum up your' discussion on these three points?
A:
.(Dr. Dillman)
In sum,'the implementation of this survey-demonstrates inadequacy on all three of the major sources of' survey error applicable to.this-study:
noncoverage,
= measurement error, and response rate.. Methods of overcoming, to some degree,~each'ofLthese problems should have'been used in order to compile a survey with the stated objective:of-identifying those residents of the area with.
certain " impairments" who would need assistance in an emergency.
Q:
'Dr.
Dillman, What is your opinion on the follow up I
. verification that was conducted?
1 i
A:
(Dr. Dillman) The "special needs survey verification form" i
was apparently used for the purpose of additionally
' determining whether the 653 respondents initially identified would actually need assistance.in the eventoof
)
an evacuation.
j I have reviewed the form used in May, 1988, to conduct the survey.
I am also aware of the fact that the result of the
" verification" was the elimination of 301 people who had initially identified themselves as needing assistance.
I l I
\\
w p
have; earlier made comment regarding other aspects of what I-perceive to:have been the effect of this " verification".
Now, I will discuss the implementation of the verification process as is indicated by the " verification" form.
In short, the contents.of the-form suggest that the verification process'could appear to be questioning the respondent's honesty in responding to the earlier questionnaire.
Thus, the respondent may have become anxious when " verifying" his or,her original information.
Second, a fundamental element in conducting telephone surveys is to make sure that each interviewer asks each question in the same way.
The questions presented on this verification form do not appear as questions but rather as short phrases.
Thus, it is necessarily left to each interviewer to actually phrase the questions.
Interviewers are very.likely to'do this in a variety of ways and as a result get different results.
For example, on question 4 of the verification form there is a." description of impairments".
One interviewer.might simply ask the respondent to describe his " impairments".
Another might read through the entire list and ask which ones, if any, apply.
Reading through the entire list is likely to result in a more acc6 rate record.
The same analysis is true concerning life-sustaining equipment in question 5.
A small, but perhaps significant difference between the verification form and the initial survey questionnaire, is l I
Ny L
.. g thatleye impairments in question 4 are described in parentheses;as " blind".
How limiting.
It' appears that p
- such. things as an inability to drive at night would not.be
' discussed or accounted for.
1 The pattern of. vagueness contained in'the original mail questionnaire repeats itself here.
For example, "Does the impaired have.any other problem we need'to know about?"
It' is'not clear what kind of. problem "we" need to know'about.
Q:
'In' sum,.what is your. opinion on the " verification process"?
A:
(Mr.'Dil'Iman) I do not feel that the information obtained f
on the~special needs survey verification form meets-the standards of professional and accurate. surveys, and I
.further believe that these data should not be trusted on
. making decisions.about respondent impairments.
The only value I can see as having possibly beenLgained through this effort would be if the original goal and purpose of the
- survey had been to compile a samole survey.
However, if
.the goal is not merely to' ascertain the. number of individuals within the study area who fall within a certain category, but rather, to identify specific personal information like names, addresses, and the types and degree of impairments necessitating assistance during an emergency, then a sample survey application doesn't really help.
Discussion of this principle at this point however, is 1
useful in illustrating the incompleteness of the existing data. 11, l
g
~
- Q '. :Mr.~Dillman,fI'want',to pick up on:that point, regarding.the identification ofia " sample" of the target population.. I recognizeothat'you have made7a number of comments.
~
describing the; inadequacy:of the' original mail'surveyfand*
telephone follow-up.-
However' suppose that.the results-n iwere.to be:acceptedfasLvalid for those who. responded.-LWhat.
Lis theLmostfthat could.be claimed'with_ respect to how completely theicompared population living in the study area 37 fc has been identified?
- p LA' Well> first, I will1 return to my conclusion Jhat'the surveys were'so'poorly done that-I do'notuthink the-results~
can b'e used'to reach any conclusions about'the number of.
households with' impaired individuals living in the study-area.
However,'if we were to accept the introductory-survey material as unbiased, the response rates as
. acceptable, and the survey itself as valid, then the most' that could be'said is that the survey has only identified a~
fraction'of1the impaired individuals who live there.. If it is.: accepted that the number of impaired individuals is 653
.orl502-(depending upon which figure' supplied.by'the-proponent of the survey you choose), and we divide that by
.the total number.of positive and negative responses received (5,546), we would then multiply the resulting; percentage 1(11.77 or 9.05)' times the total number of households'in the area to establish the total population of impaired individuals who live there.
This results in an estimate of the total population of impaired individuals-in i
the EpZ area to be more than 1,900 (1,901 to 2,472) individuals.
Under this formula, you would come out with the higher figure, 2,472, if you used the higher reported result - 653'. L J
h
w
4 Q:
Ms. Moriearty, in your opinion, out of a population of approximately.52,500, how many of that number would be considered within the population targeted by the subject survey?
A:
(Ms. Moriearty)
Well, my analysis. entails first an estimate of how many individuals would be considered
~ disabled, and then, of those, how many would be within the population to be targeted, in other words, residents with disabilities such that they could not act independently in an emergency.
To begin with, out of a population of 52,500, I would. estimate that approximately 9,545 would_be I
disabled.
I have_ attached population reports you gave me to look at in the corresponding appendix.
Q:. Thank you.
How did you reach the number referred to above?
A:
(Ms. Moriearty)
The estimate is extrapolated from the'1985 U.S. Census survey I discussed earlier, which is entitled
" Disability, Functional Limitation, and Health Insurtace I:
L Coverage."
It surveys disability in the general population.
As the first major government survey employing functional criteria to define disability, the report is considered by MOHA to be more accurate than previous Census
{
or medical surveys which rely more heavily on diagnostic
(
classifications or diagnostic prompts.
The estimate is broken down as follows:
8768 individuals would be residents who, according to the survey trends regarding' prevalence in the general population, could be identified as "having a functional _
eT+c,
+
._ e
' limitation"1at the presentitime..777 individuals represent nf'
-that' portion of.the target zone population which, al,though currently not identified-as having a functional limitation, y,
.have sought medical help more than twenty times in the year previous to the survey.
To infer from the federal
~
government's health interview surveys, which employ m
diagnostic categories, the most common disablingLconditions in the general population arise from causes which are.
life-style related such as heart disease, substance abuse, and hypertension.
These conditions are'usually highly stress-sensitive.
The varying functional limitations which result from disabling conditions of this nature cannot be adequately. addressed by mere knowledge of the "name" of the condition.
This estimate captures a portion of the general population which would most likely be vulnerable to frank disability.in an emergency.
However, althouth.this survey is designed to elicit more practical information about disability than others, it still relies ~ heavily on self-perception and self-identification.
As I mentioned earlier, a small percentage of individuals who are apparently disabled to a surveyor do not identify themselves as disabled.
This phenomenon is corroborated by other government surveys.
1 1
Therefore, the estimate is conservative.
Q:
Can you describe how that number would include resident special needs individuals apart from those living in institutions or facilities.
I.
~k.
L
'A:.
(Ms;'Moriearty)
The survey' cited above is a survey of the no'n-institutional population.- A very small percentage of persons _with disability (less than 5%) are institutionalized on a long term basis.
The 5% estimate includes elderly-disabled individuals.
The rate of long term institutionalization among non-elderly disabled persons is much lower, perhaps' approaching 1%.
As a point of interest, in Massachusetts, about 1-2% of non-elderly f
disabled persons live in government-sponsored community housing programs.
In many respects, some of these individuals are already connected to trained and knowledgeable. staff that could offer assistance in an emergency.
Q:
In your opinion, does a figure of 352, 502, or 653 (all of which are "results" which have.been reached by this survey) out of aftotal 52,500 represent an accurate estimate of special needs residents who could not act independently in an emergency?
A:
No.
Based on'the' aforementioned U.S. Government survey estimates, which present a profile of actual functional capabilities, any of those figures present a substantial underestimation of the special needs population.
Of the 9,545 individuals projected above to have special needs, approximately E
2.235 can be identified, based on this survey, as needing personal assistance with a major life activity under " normal" conditions.
The remainder of the estimated special needs population would need varying intensities of personal assistance in an emergency to the extent that key environmental and technological supports are not available or are disrupted.
p !
l
A
p
.s-f g
g v.,
- Q.
'It is'interesti'ng'that you?both' arrive at figures that?are relativelyfclose.in range,1especially.given the fact that=
- you are each coming'from;a different perspective. 'Indeed, Lthe~ higher
- of Dr.JDi11 man'sfestimates which uses~the
. reported 502. survey 1 result,;and Ms. Moriearty's extrapolation from the Census: Bureau come quite:close, 2,472 and 2235, respectively.
A.- ~(Ms.'Morieart'y)Yes, it'is interesting isn't.it?!.In sum-l it seems that whether you look at the results from Dr.
~
'Dillman's perspective or mine, it is~ clear the Applicants' survey:results are inaccurate.-
Q.
Dr. Dillman, what are your final thoughts?
A.-
The overall' survey effort was poorly done and under no circumstances should the results be used-to claim that all i
.or.even a majority of the " impaired" residents of the study-area have been identified.
This conclusion results from-several problems, such as: 1) some households did not have a chance to; participate because of the bias in the mailing j
list; 2) the use of the vague and undefined words (e.g.,
q evacuation and impairment) resulted in responses that could l
not be clearly interpreted; 3).an inexcusably low' response 1
rate was'obtained because of the lack of personalized procedures, the creation of' respondent confusion, I
uncertainty and mistrust by the content of the cover letter and questionnaire; and 4) the use of a follow-up telephone survey that was vague and not written so that interviewers could consistently administer it.
In sum, these results cannot be.taken seriously as having produced a list of all residents in the study area who have " impairments" that would require them to need help to evacuate the area.,
E
. b,. -
Q.
Thank you both.
I think we have exhausted this subject.
Shall we sign off now?
A.
(Both)
Yes.
f.
9 6
l
L:o D
4 UNITED STATES OF AMERJCA NUCLEAR REGULATORY COMMI6SION' q
q ATOMIC. SAFETY AND LICENSING BOARD j
Before the Administrative Judges:-
Ivan W.
Smith, Chairman Dr. Richard F. Cole i
Kenneth A. McCollom l-
)
In the Matter of
)
Docket Nos. 50-443-OL
)
50-444-OL PUBLIC. SERVICE COMPANY
-)
(Off-Site EP)
OF.NEW HAMPSHIRE, EI AL.
)'
)
(Seabrook' Station, Units 1.and 2). )
April 24, 1989
)
ERRATA SHEET TO CORRECTED TESTIMONY.OF DR. DON A. DILLMAN AND SHARON MORIEARTY ON BEHALF OF THE ATTORNEY GENERAL OF THE COMMONWEALTH
~
OF MASSACHUSETTS REGARDING JI CONTENTION 48 Page 9, line 28, which read:
type, three-of potential survey errors must be is now on page 10, line 28, and should read:
type, three types of potential survey errors must be Page 17, line'53,-which read:
figure out and report what king of " assistance" he would should read:
figure out and report what kind of " assistance" he we'11d
---L.,
l__-_-
i
j.s.- e i
y Page125,: lines 41"and 43,' which-read:
7 C
the higher figure, 2,472,.if:you used the. lower reported' result -
502.
u Lis now'page 24, lines-51-and 53, and should, read:
the. higher. figure, 2,472, if you used the higher reported result -:
p.
=653.
1Page 26,'line 55 snd page'27 linesil-11,.which read:
~Disorderslof the skin, which'can. affect. general physiology;as well
~as tactile sensation'might also fall in this category as would, n
conceivably, certain kinds of organic brain dysfunction.
.However,
' individuals: with-sensory impairment ucould have functional manifestations which also place them'into categories of mental and mobility impairment as well.
is now page 26, lines 1-31,-and should read:
777. individuals represent that, portion of the target ~ zone Lpopulation which, although currently not identified:as having a functional limitation,'have sought medical help.more than twenty times inlthe year ~ previous tofthe survey.
To infer from the
. federal: government's health interview surveys,.which employ diagnostic; categories, the most common disabling conditions in the
- general population:arise.from causes which are life-style'related such'as-heart disease, substance: abuse, and hypertension. !These-conditions?are usually highly stress-sensitive.L The varying functional: limitations'which result from disabling conditions of this nature'cannot be' adequately. addressed by mere knowledge of
['
the "name" of the.conditio.4.
This estimate captures a portion of-the general population. which would most _ liftely. be vulnerable to 1
-frank disability in an'emergancy.
COMMONWEALTH OF MASSACHUSETTS JAMES M. SHANNON ATTORNEY GENERAL
< Q i/L(L Go (b
t Pamela Talbot
[
Assistant Attorney General Nuclear Safety Unit Department of the Attorney General One Ashburton. Place Boston, MA 02108-1698 (617) 727-2200 DATED:
April 24, 1989 }