ML20032C229
| ML20032C229 | |
| Person / Time | |
|---|---|
| Site: | Crane  |
| Issue date: | 11/03/1981 |
| From: | Trunk E METROPOLITAN EDISON CO., PENNSYLVANIA STATE UNIV., UNIVERSITY PARK, PA |
| To: | |
| Shared Package | |
| ML20032C214 | List: |
| References | |
| ISSUANCES-SP, NUDOCS 8111090522 | |
| Download: ML20032C229 (14) | |
Text
-
LIC 11/3/81 UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING BOARD In the Matter of
)
)
METROPOLITAN EDISON COMPANY
)
Docket No. 50-289 SP
)
(Restart)
(Three Mile Island Nuclear
)
Station, Unit No. 1)
)
LICENSEE'S TESTIMONY OF EDWARD V. TRUNK
^
8111090522 811102 PDR ADOCK 0500028 i
TABLE OF CONTENTS Page I.
Nature of Testimony..............................
1 II.
Background and Qualifications l'
III.
Scope of Task and E,?' Sod of Analysis 3
IV.
Report Dated September 2, 1981 and Follow-Up Letter Dated September 7, 1981...................
5 V.
Report Dated September 21, 1981..................
8 VI.
Report Dated October 1, 1981.....................
9 VII.
Final Report Dated October 14, 1981.............. 10 VIII.
Conclusions.....................................
11 i
6 y
,,..r y-.,-.
,r_,c-,
,.m.
BY WITNESS TRUNK I.
Nature of Testimony My name is Edward V. Trunk and I am an Assistant Professor of Engineering at the Pennsylvania State University
("PSU") in Middletown, Pennsylvania.
I am here today to testify on behalf of Metropolitan Edison Company (" Licensee")
regarding my independent analysis of several sets of Licensee-l administered exams to ascertain, by comparison if the individ-1 uals' answers on the exams, whether there were any indications of cheating.
This testimony will describe the scope of my i
investigation, the manner in which it was performed, the results obtained and the follow-up work done.
This testimony will also illustrate, at least in part, that the indications of cheating by TMI-1 operator license candidates on the 1980 Kelly exam, including Category T, the Category T and non-Category T make-ups, the 1979 mock exam and two sessions of the 1981 ATTS mock exam were extremely limited.
II.
Background and Qualifications I received a Bachelor of Mechanical Engineering degree from the City College of New York in 1956, a Masters of i
i Mechanical Engineering degree from the Rose-Hulman Institute of Technology in 1963, and completed doctoral studies at Washington University, St. Louis in 1969.
I currently am a member of the American Society of Mechanical Engineers
)
('ASME"), hold various national positions and was past chairman T
k
E of the ASME Susquehanna Section.
I am also a member of the American Society for Engineering Education.
Prior to joining the faculty at PSU in 1969, I worked from 1956 to 1962 as a mechanical engineer-for Allis-Chalmers Company.
I then taught mechanical engineering at Rose-Hulman Institute and Washington University for five years between completion of my masters and doctoral studies.
I have served at various times as a consultant to IBM, Anaconda Aluminum Company, AMP cnd Quaker Oats Company in the areas of niechanical and industrial engineering.
My responsibilities have included job analysis and training.
I was also a consultant to the President's Commission on the Accident at Three Mile Island in 1979, and assisted in inter -
pretation of technical documents.
Since coming to PSU in 1969, I have lived in Middletown and have followed the construction and operation of the Three Mile Island Nuclear Station.
I have also taken many tours of the facility with my thermodynamics class, I am therefore familiar with the facility and its history, partico-larly its most recent history.
I was assisted in my evaluations of the TMI exams by a colleague, Donald L.
Miller, an Assistant Professor of Engineering at PSU.
Mr. Miller received a Bachelor of Science degree in Mechanical Engineering from PSU in 1965 and a Masters of Engineering degree from PSU in 1969.
He currently is a member of the ASME and vice-chairman of the Susquehanna Section, a member of the American Society for Engineering Education and a registered professional engineer in Pennsylvania.
He joined the PSU faculty in 1969 and also taught part-time at Elizabethtown College during the 1974-75 and 1976-77 school years.
III. Scope of Task and Method of Analysis John F. Wilson, counsel to Licensee, telephoned me on Saturday, August 29, 1981 and asked me, on behalf of Licensee r if I would compare several examination papers of TMI-l opera-tors to determine whether there were indications of cheating.
Mr. Wilson informed me that two unnamed licensed operators had been found to have cheated on a mock NRC exam and an NRC-administered exam in April, 1981 and had resigned.
He thus suggested that I review every word of every exam supplied to me to ascertain if there were any additional indications of cheating by any individuals.
I accepted the job, believing that I was qualified to perform this work because I am neutral and can make independent, unbiased judgments, I am generally knowledgeable about the subject matter of the TMI exams, I am very familiar with the TMI facility and from my teaching work, I know how to spot potential problems when grading or reviewing exams.
I began my analysis with Mr. Miller two days later on Monday, August 31, 1981.
The two of us developed a method of reviewing the exams which proved to be very effective.
For
Instance, our first set of exams contained thirty-two individual Kelly Category T exam papers.
We arranged all of the exams in a row on a long table and examined one question at a time from all the exams.
Mr. Miller and I moved along the table in opposite directions so that we would each review the exams in a different order.
As we reviewed each exam, we noted peculiarities in each answer and called them out to each other.
Use of unique words, misspellings, wrong answers, the sequence of items listed, use of letters or numbers to identify items, vertical or horizontal layout of answers, and the length of the answers were among the numerous characteristics we noted.
If one or both of us recognized a similarity, we pulled the two papers from the row and placed them side by side.
In that way, questionable exams could be studied more closely.
After completing all the responses to a question, a second review was made by scanning the responses rapidly and checking them off to indicate completion.
This method optimized our retention of mechanical details so that we might spot similarities otherwise forgotten.
When a questionable item was found, the item was noted on a separate work sheet.
We found that with large sets of exams it was necessary that both of us review together.
Several of the make-up exams only contained two or three papers, however, and we were able individually to review these small sets.
In addition to reviewing the exams themselves, we reviewed some training materials and answer keys.
The answer
. e
~ _ _. -
~ __-_ _. _. -.
t keys provided guidance in what was sought and provided the accepted form of a response.
Review of these documents also helped us, in some cases to see explanations for otherwise unexplained parallelisms.
Even though we had a limited amount of time in which to complete our analysis, we nevertheless had ample time to perform our work carefully and thoroughly.
We read every word of every an der we reviewed.
We therefore have a high degree of confid.nce in the soundness of our conclusions.
IV.
Report Dated September 2, 1981 and Follow-Up i
Letter Dated September 7, 1981 Our first report to Mr. Wilson was dated September 2, 2
1981, and was hand delivered the same day.
We had reviewed thirty-two RO, sixteen SRO and all the Category T exams i
administered by Kelly in April, 1980, thirty-three first-round l
Category T make-ups (5 separate exams) taken in November and December, 1980, sev'en second-round Category T make-ups taken on March 27, 1981, and four third-round Category T make-ups taken l
on June 25, 1981.
We concluded that all the exams administered l
by Kelly (including the Category T exams of April,1980) were l
l "significantly independent in manner and substance to preclude
~any possibility that cheating or any cooperative effort had taken pla e."
By in large, almost all of the exams and make-ups contained unusually varied answers, indicating generally an independent thought process rather than a memorized or copied response.
t i
s
,.~-.v.-
_,,_,w
...n,
,,n-.-
~,.m,,
nm.._,,.,,
_,,,,,e
,.w.,_.,,,,,,4en.n,mm.-,.
v - m.
e~,.,.
,.1,.a,nn,*,.
We did find, however, that the three rounds of Category T make-up exams contained several unusual similarities in the respcnses to certain questions.
Because similar responses are likely to be found on tests like these (that is, tests requiring short answers, numerical or logic sequences, or accepted terminologv) we followed up all similarities with cross-checks of cther responses and the answer key.
G and H, for example, were found to have eleven sets of parallel answers, some of which appeared in all three rounds of make-ups.
We suggested in our Report that the evidence supported a possible collaboration between the two individuals.
W and GG were found to have two sets of very similar answers on the first-round Category T make-up given on December 19, 1980.
MM's answers to the two questions were very close to those of W and GG but did not have exact wording.
We noted possible cooperation among these individuals.
Finally, we noted similarities between S's and Y's answers on the December 19, 1980 first-round Category T make-ups.
These similarities resembled two responses given by G and H on the prior November 26 exam.
We concluded that there was insufficient evidence to support collaboration in this case.
On September 3, one day after delivery of our first Report, Mr. Miller and I had a meeting with Mr. Wilson and his colleague Richard D.
Lloyd in the TMI-2 Administration Building to discuss our findings.
We were given some training materials
and some new information, such as the fact that the second-round make-up exam was a take-home exam.
With this new knowledge, we reviewed again the parallelisms of G and H as compared to their responses on earlier tests and the responses of other examinees.
The extremely small likelihood of cooperation on a take-home test, and the comparison of the responses with training notes and with other responses eliminated several of the stronger parallels we had found.
I sent John Wilson a follow-up letter dated September 7, 1981, in which I suggested that the strength of the evidence against G and H had been greatly diminished.
Although I then believed that the evidence was inconclusive, I suggested that the answer key for the second-round Category T make-up exam given on March 27, 1981, be compared to the training materials.
I also suggested that the relationship between G and B be investigated to determine whether they might have been good friends and might have studied together often.
I understand that Messrs. Wilson and Lloyd I
interviewed G and R on September 11, 1981, only a few day:s after receiving my letter.
This interview and conclusions drawn from it are explained in greater detail in Licensee's Testimony of John F. Wilson (" Wilson Testimony").
l l
1 l l
L
V.
Report Dated September 21, 1981 During the week of September 14, 1981, I got a e
telephone call from John Wilson asking me, on behalf of Licensee, to analyze two sessions of the 1981 ATTS mock exams.
I agreed and again performed the work with Mr. Miller.
Our second Report to John Wilson was dated September 21, 1981.
We had reviewed twenty ATTS mock RO exams taken on April 1, 1981, and eight ATTS mock SRO exams taken on April 2, 1981.
These were the only ATTS exams which I understand the NRC had not previously reviewed in its original investigation of cheating on the April, 1981 NRC exam.
We concluded that there was no visible evidence that any cheating had taken place on these examinations.
As I noted previously with regard to the Kelly exams and Category T make-ups, individual responses on these ATTS exams varied greatly.
The only curious parallelisms we fcund involved two of BB's answers on the SRO mock exam which were identical to the answer key.
We presumed either that BB had cheated by using the answer key in responding, or that the answer key was l
l l
made up after the exam was taken and utilized BB's answer.
I understand that on September 22, 1981, the day after our second Rep 6rt was issued, Messrs. Lloyd and Wilson l
l interviewed BB and Nelson Brown, the grader for the section of l
l the ATTS exam containing BB's two answers.
The results of the interview are explained in the Wilson Testimony. 1
VI.
Report Dated October 1, 1981 On September 30, 1981, Mr. Miller and I again met with Messrs. Wilson and Lloyd to discuss our prior findings.
John Wilson provided us with additional training materials and requested that we write a follow-up to our September 2, 1981 Report utilizing these newly received materials.
We agreed to perform this work and our third Report issued October 1, 1981.
Our review of the training materials and better understanding of the testing environments a3iowed us to remove from suspicion several sets of similar answers noted in our first Report.
In this third Report, we recognized, for example, that certain answers of G and H and S and Y were virtually identical to the handout provided in training.
We found it probable, therefore, that these individuals had memorized the teaching materials.
The second set of similar answers from S and Y also were virtually identical to the language on the transparency used in preparation for the exam.
Again, we found memorization the probable answer.
The second-round make-up was a take-home exam.
This format is unlikely to encourage individuals to copy from their colleagues.
G and H had several sets of similar answers on this exam, and because both of them failed the exam, we believed it unreasonable to assume that they had cheated at that time.
The same question regarding generation of hydrogen gas appeared on both the first-and second-round Category T -
make-ups, and G and H made the same mistakes in both answers.
The second mistake was understandable because both G and H easily could have misunderstood the instructor's corrections i
which were written on the first exam.
The first mistake, however, was still unexplained.
G and H submitted another similar answer on the second-round make-up that was correct but was very differe '-
from the instructor's answer key.
Although B's answer had some additional information that G's answer lacked, we were still uncertain about the explanation for these answers.
Since we had written our first September 2, 1981 Report, we had learned that the third-round Category T make-up exam had been strictly proctored and that only four operators had taken it.
This led us to eliminate the two short answer questions we previously had noted in our first Report.
Our only other comment concerned the similarities among the answers of MM, W and GG.
Upon reexamination of those answers, we found that MM's answers, although somewhat like the others, were not similar enough to suggest cheating.
In sum, we were left with three questionable items:
7 the two sets of G and H answers described above, and the answers of W and GG which were identical.
Licensee's response to these open items is described in the Wilson Testimony.
VII.
Final Report Dated October 14, 1981 l
During the week of October 5,1981, John Wilson telephoned me once again and asked me, on behalf of Licensee,
l to review the Kelly non-Category T make-up exams, various review tests and quizzes and a 1979 mock NRC exam.
I agreed to perform the work and, with Mr. Miller's aid, completed it and submitted a fourth Report dated October 14, 1981.
This Report included an analysis of the Kelly non-Category T make-up exams from the 1980 Cycles 7-3 (weeks 1 through 6), 7-4 (weeks 2 through 6) and 7-5 (weeks 1 through 6).
We also reviewed additional Kelly non-Category T make-ups on particular topics including three heat transfer quizzes from February and March, 1981, and eight review tests.
Finally, we reviewed Licensee's March, 1979 mock NRC exam nation.
In total, we reviewed 169 separate exams for this final Report.
We concluded that except for two incidents, the exams presented absolutely no visible evidence of cheating.
The two similar sets of responses were discovered on the Cycle 7-5 exams and involved responses by G and H again.
One set of answers consisted of very closely worded short responses while the second set consisted of a long paragraph with only minor variations.
We concluded that G and B appeared to have cheated on the exam, particularly in view of the similarities in the long answers.
Messrs Wilson and Lloyd fol'swed up on this finding, as explained in the Nilson Testimony.
VIII.
Conclusions After having reviewed hundreds of exams written by TMI-l operator license candidates, Mr. Miller and I were very )
e impressed with the wide variety in the form of responses we saw.
We believe this variety indicates that operators at TMI-1 generally have an adequate understanding of the information presented to them during training and are therefore able to respond to questions with original thought rather than with a memorized answer.
After comparing tests and performing a follow-up investigation, we were lef t with a few response similarities that could not be dismissed as trivial, nor could they be explained as being one-or two-word answers, logic sequences, key words, random parallelisms, or derived from training materials.
However, these few remaining items did not consti-tute a strong enough case for us to conclude positively that cheating had occurred because they were too few in number and, for the most part, too short.
Certainly, we saw absolutely nothing that could be classified as rampant cheating--even on the take-home tests.
Finally, we feel confident about the soundness of our t
l conclusions.
We approached this entire review with great care and seriousness of purpose and spent much time reviewing each and every answer.
We therefore believe that in the exams we l
reviewed, we noted all the questionable items which merited l
further scrutiny.
l t
l l
i
[ l f
t
-.