ML20128C658

Brief Supporting Ucs 850513 Appeal of ASLB 850503 Partial Initial Decision Approving Licensee Training Program.Aslb Erred in Authorizing Restart Before Licensee Responded to Finding of Program Inadequacy.W/Certificate of Svc

ML20128C658
Person / Time
Site:	Three Mile Island
Issue date:	07/01/1985
From:	UNION OF CONCERNED SCIENTISTS
To:
References
CON-#385-654 ALAB-772, SP, NUDOCS 8507030646
Download: ML20128C658 (69)

Text

- - - _ - . _.-

brS I ,a July 1, 1985 - UCS DOCKETED USNRC UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING APPEAL BOARD 25 JL -2 No:49 0FIlCE OF SEcq w .

. . . .. .... ....... .... 00CXEIt EPVW

)

METROPOLITAN EDISON COMPANY ) Docket No. 50289 SP

) (Restart Remand on (Three Mile Island Nuclear ) Management - Training)

Station, Unit No. 1 )

.................................) ,

I l

UNION OF CONCERNED SCIENTISTS BRIEF ON APPEAL '

OF THE PARTI AL INITIAL DECISION ON LICENSED OPERATOR TRAINING l

l I"$E98BS$19" In ALAB-772, 19 N. R.C. 1193 (1984), the Appeal Board i

overturned the Licensing Board's earlier Partial Initial Decision on the adequacy of licensed operator training at Three Mile Island Unit No. 1. Pursuant to that remand, the Licensing Board held further hearings on the issues raised by the Appeal Board.

As a result of Licensee's decision to address the remand by defending its entire training program, the hearings addressed the general issue of the adequacy of the licensed operator training program. Partial Initial Decision, Slip op. (hereafter "PID") at 11.

On May 3, 1985, the Licensing Board issued a Partial Initial Decision in which it approved Licensee's training program subject to the requirement that Licensee " implement a plan for the 8507030646 850701 PDR ADOCK 05000299 G PDR D}

3

a 1

' l o

evaluation, af ter training, of the performance of its trained reactor operators and senior reactor operators in the job setting, under both normal and abnormal operation." PID a t 216.

On Ma y 13, 1985, the Union of Concerned Scientists ("UCS") filed a notice of appeal pursuant to 10 C .P. R. S 2.762(a). For the reasons discussed in this brief, UCS urges the Appeal Board to reverse the Licensing Board's decision.

As discussed more fully below, the PID should be reversed for the following reasons:

1. Having found Licensee's program to be inadequate in the area of post-training evaluation of operator performance, the Board erred for two reasons in authorizing immediate restart. First, the Board erred in authorizing restart before Licensee had even begun to respond to the Board's finding that Licensee's program is inadequate. Second, the Board erred in ignoring UCS' r ig h t to litigate the adequacy of whatever Licensee might propose in response to the Board's finding of inadequacy.

2. The Board erred in finding current TMI-1 operator attitudes to be acceptable on the basis of the testimony of Michael Ross and the Reconstituted OARP Committee, and on the ground that the attitudes will improve once the operators are allowed to restart TMI-1.

3. The Board erred in approving Licensee's program of operator examinations. In particular, the Board erred in approving Licensee's oral examinations when those examinations are administered by individuals who are not trained or qualified to administer oral examinations.

4. The Board erred in accepting and relying upon the conclusions reached by the Reconstituted OARP Review Committee. In particular, the Board erred in ignoring the fact that the Committee filed a formal filing with the Commission in which it prejudged the program and committed itself to approval of the program before it had undertaken any significant review of the program. When the Committee reached that prejudgment, it had not undertaken an adequate review under any standard.

)

's 4 EO9EEE9EEE OU9 9EE961EIl96 9E IEE EE99EE91EE The issues in this appeal stem ultimately f rom the Commission's order of August, 9,1979, which directed that TMI-l must remain shut down pending a hearing on a number of issues, including the managemeat capability and technical resources of the Licensee. In that decision, the Commission directed that two relevant " s ho r t- te rm" actions be taken as necessary but not sufficient conditions of restart of TMI-1. First, all TMI-l operators were to be retrained and reexamined. Second, the Licensee was ordered to demonstrate the adequacy of its managerial capability and resources, including "the technical capability and training of operations staff . . . ." CLI-79-8, 10 N.R.C. 141, 143-46, 149 (1979).

Af ter an extensive hearing on management issues, includ ing the substantive adequacy of the TMI-1 licensed operator training program, the Licensing Board issued a decision in August 1981 generally favorable to Licensee. LBP-81-32, 14 N.R.C. 381 (1981). Because of the contemporaneous discovery of cheating on NRC licensed operator examinations, however, the Board retained jurisdiction of the case to consider the impact of this new information on its findings and conclusions on Licensee's management competence. Id. at 403. The Board subsequently reopened the management proceeding and appointed a Special Master to hear evidence on the impact of the cheating incidents at

i

'8 a l TMI-1. During the course of the further hearings, much evidence was taken demonstrating that the problems with GPU's training program went well beyond instances of cheating. The Special Master concluded on the basis of the evidence that the training 1

program was poorly administered, weak in content, and ineffective in its method of instruction. LBP-92-34B, 15 N. R.C . 918, 1020.

He concluded that the GPU training program did not meet the requirements of the Commission's 1979 order. Id. The Licensing Board then found that there had been a breakdown in the integrity of Licensee's training and testing program at TMI-1. LBP-82-56, 16 N.R.C. 281, 300. The Board imposed several requirements, primarily future audits, directed at obtaining future assurance of the adequacy of the training program. Id. at 365, 384. The Board also concluded, however, that the identified weaknesses in the program did not undermine the Board's earlier decision favoring restart. Id. at 301.

In ALAB-772, the Appeal Board reviewed the entire record in the TMI-l restart proceeding on the ability of GPU Nuclear Corporation's management to operate TMI-l safely. ALAB-772, 19 N.R.C. at 1201. In response to issues argued on appeal primarily by UCS and TMIA, the Appeal Board endorsed the Licensing Board's characterization of the question that had to be answered following the cheating incidents at TMI-1, viz. , "is the instruction adequate to prepare the operators to operate the plant safely?" It disagreed with the Board, however, "on its affirmative answer to that question." ALAB-772, 19 N.R.C. at l

> , 1232-33. The Appeal Board believed that the record in the reopened proceeding had perhaps raised more questions than it had satisfactorily answered. "For example, does the training program actually enhance the operators' knowledge or simply encourage memorization for test-taking purposes? Are the licensee and N.R.C. examinations an ef fective way to measure an operator's ability to run the plant? Do the format and content of the examinations encourage cheating?" Id. at 1233.

The Appeal Board ruled that the " principal difficulty" with the Lice . sing Board's decision was its "f ailure to consider, as prcmised and in a meaningful way, its earlier finding that licensee's training program was ' comprehensive and acceptable. '"

Id. Instead, the Board " relied on the post-cheating testimony of only licensee and the Staff." Id. Of particular additional significance was the large degree of reliance by the Licensing Board on the testimony of a panel of GPU consultants known as the OARP Review Commi ttee, who reviewed licensee's accelerated operator retraining program in 1980-81 before the cheating became known. As the Appeal Board stated, "[T]he Board essentially pr esumed that the earlier, favorable expert testimony by the outside consultants would not have been altered by the cheating revelations." Id.

In addition to these general rulings, the Appeal Board raised a number of specific issues and concerns. These are detailed in the Board's new Partial Initial Decision at 4-5. The issues of particular concern in this appeal involve the attitudes of

,, licensed operators, the adequacy of Licensee's examinations, and

'the adequacy of the OARP Review Committee's review of the training program.

As the Licensing Board has explained, the scope of the proceeding extended to the adequacy of Licensee's entire licensed operator training program. P ID a t 11. Licensee presented the testimony of the OARP Committee, which had lost one of its original members, but gained another. The new member was Mr .

Frank Kelly, who had previously been a consultant to GPU on examination-related issues. Tr. 31,964. Licensee also presented the testimony of several of its own personnel. The Staff presented a panel of witnesses. UCS presented Dr. James J.

Regan, an industrial psychologist with 31 years of experience in the area of training and education, including service from 1973 to 1982 as the founding Technical Director of the Navy Personnel Research and Development Center. Regan, ff. Tr. 33,532, at 1-4.

In the Partial Initial Decision, the Licensing Board ruled in favor of Licensee on most issues. In one crucial area, however, ,

the Board ruled that UCS had prevailed. The Board ruled that Licensee' licensed operator training program was inadequate in that it failed to provide for formal evaluation of operator performance on the job after training. PID at 143-156. Despite this ruling, the Board authorized Licensee to restart TMI-1. The Board characterized the need to evaluate operator performance on the job as a long-term requirement within the meaning of i

'o ,, CLI-79-8. PID at 155. It then held that Licensee could restart TMI-l without fulfilling this requirement. The Board required only that Licensee file a proposal for evaluating on the job performance, which would be subject to the comments of the parties. PID at 216-17. It allowed restart regardless of the adequacy of the proposal.

UCS appeals four specific aspects of this decision. The facts relevant to each issue are discussed in the appropriate section rather than here.

OggUgggT I. The Board Illegally Authorized Restart Af ter Ruling That UCS Had Prevailed On The Issue Of Post-Training Evaluation Of 9ESEatgg,Pggfgggaggg,gg,Thg,g,gb.,,,,,,,,,,,,,,,,,,,,,,,,,,,,

The Board ruled that Licensee's licensed operator training program is inadequate because Licensee does not use periodic, formal on-the-job performance evaluations for training revision or for any other purpose after initial on-the-job training. In the Board's words, "UCS has prevailed on this issue." PID at 144.

In analyzing this issue, the Board explained that the need for post-training evaluation of job performance is one of five concepts that "are essential to complex developmental endeavors." PID at 145. Indeed, the need for such an evaluation is so fundamental as to be " intuitive." Id. More important, it is one of the five general principles governing training that the Commission has adopted in a recent policy statement. See discussion, PID at 156-166. Thus, this is not a minor point.

Ra the r , it is comparable to the need for " evaluation of trainee

mastery of [ learning] objectives during training," PID a t 14 4, n.

23, another of the five basic principles adopted by the Commission. That issue was the subject of extensive litigation in this hearing with respect to both the response to cheating and the adequacy of Licensee examinations. This, then, is a major issue, fundamental to any determination as to the adeguacy of Licensee's operator training program.

Despite its recognition of the significance of this issue and its ruling that Licensee does not meet this fundamental requirement, the Board ruled that Licensee need not delay restart o f TMI-1. The Board justified this action on the ground that the requirement to develop and implement a plan for evaluation of post-training job performance "would be considered a long-term requirement within the meaning of the notice of hearing." PID at 155. The Board then retained jurisdiction as this issue and ordered that Licensee submit its proposal for an evaluation plan within thirty days, and that the NRC Staff, UCS, and other parties provide their comments within forty-five days. PID at 217. On June 24, 1985, the Board accepted Licensee's proposed evaluation plan as demonstrating reasonable progress toward satisfactory completion of the long-term action of implementing an adequate evaluation program. LBP-85-21.

The Board has committed three fundamental errors in approving restart after holding Licensee's training program to be inadequate in this area. First, the Board incorrectly treated this issue as a long-term action rather than a short-term action under the notice of hearing, CLI-79-8. This has a significant i _.

a _9_

impact on the rights of the parties and the extent of the remedial actions that Licensee must take prior to restart.

l Second, the Board violated the rights of the parties by finding j Licensee's proposed evaluation plan to be adequate without further hearings on the record. Third, the Board erred in finding that submission of a proposed plan constitutes reasonable progress toward satisf actory implementation of a post-training evaluation program.

A. The Adequacy Of Post-Training Evaluation Is A "Short-Term Action" That UCS Is Entitled To Litigate P { i g{,To,, R,e s 3a { gy,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,,

In establishing this hearing, the Commission ordered that, The subjects to be considered at the hearing shall include:

(1) Whether the "short term actions" recommended by the Director of Nuclear Reactor Regulation (set forth in Section II of this Order) are necessary and sufficient to provide reasonable assurance that the Three Mile Island Unit 1 facility can be operated without endangering the health and safety of the public, and should be required before resumption of operation should be permitted.

CLI-79-8, 10 N.R.C. at 148. Among the "short-term actions" recommended by the Director of Nuclear Reactor operation was a requirement to (e) Augment the retraining of all Reactor Operators and Senior Reactor Operators assigned to the control room includ ing training in the areas of natural circulation and small break loss of coolant accidents including revised procedures and the TMI-2 f accident. All operators will also receive training  :

at the B&W simulator on the TMI-2 accident and the  ;

licensee will conduct a 100 percent reexamination of l all operators in these areas. NRC will administer  !

complete examinations to all licensed personnel in accordance with 10 CFR 5 5. 20-23. 9.

CLI-79-8, 10 N.R.C. at 144. ,

F 7 - y

, The issue on which UCS prevailed is an aspect of this short-term action item, which was specifically identified by the Commission as an issue that must be resolved prior to restart and as an issue that is subject to litigation by the parties in the adjudicatory hearing established by CLI-79-8. The issue of the adequacy of reactor operator training and reexamination includes the adequacy of the performance evaluations necessary to determine whether trained personnel are performing adequately on the job.

Under CLI-79-8, there is no question that UCS is entitled to litigate the adequacy of Licensee's compliance with the short-term actions identified by the Commission. There is also no question that the short-term actions must be completed prior to restart. The Board has violated both of these principles.

The Board seeks to justify its action entirely on the ground that formal evaluation of operator performance in the job setting "is almost by its very nature a function best performed after restart." Licensing Board Response to CLI-85-2, LBP-85-10, April 11, 1985, at 8. The Board itself admits, however, that

" operators have impo r tan t responsibilities during shutdown."

Id. The Board has cited nothing in the record to support the proposition that performance evaluation cannot be adequately performed prior to restart, althoug h on its face that proposition is incorrect in light of the extensive testing and other activities that Licensee must perform prior to criticality.

Since Licensee bears the burden of proof but has presented no

_11 evidence on the issue, the Board was required to rule Licensee has not prevailed and that performance evaluations must be done prior to restart.

The Board has utterly ignored UCS' r ig h t to litigate both the adequacy of any proposed evaluation plan and the adequacy of implementation of that plan. The Board has also utterly ignored the fact that UCS is entitled to litigate these matters prior to restart. For these reasons alone, restart must be halted, and this issue must be remanded to the Licensing Board for further hearings.

B. The Board Illegally Denied UCS The Right To Litigate The ed ssg agy _gf ,Th g,gv alg ag igg,gla g . ,,,,,,,,,,,,,,,,,,,,,,,,

The preceding discussion establishes that UCS is entitled to litigate the adequacy of the evaluation plan prior to restart if the development and implementation of the plan is a "short-term action." The same would be true even if this were considered to be a "long-term action," as the Board has characterized it.

According to CLI-79-8, The subjects to be considered at the hearing shall include:

(2) Whether the "long-term actions" recommended by the Director of Nuclear Reactor Regulation (set forth in Section II o f this Order) are necessary and sufficient to provide reasonable assurance that the facility can be operated for the long term without endangering the health and safety of the public, and should be required of the licensee as soon as practicable.

CLI-79-8, 10 N.R.C. at 148. Further, the Commission ordered that TMI-l remain in a cold shutdown condition pending "satisf actory completion of the required short-term actions and reasonable,

grogress,toward , satisfactory,comgletion of . . . required long-term actions. . . ." CLI-79-8, 10 N.R.C. at 146 (emphasis supplied) .1 Under CLI-79-8, therefore, the parties have a r ig h t to litigate the suf ficiency of long-term actions and the adequacy of progress toward completion of those actions.

The Board's decision denies UCS the right to litigate either of those. issues. Having prevailed in its argument that the Licensee's training program is inadequate for failure to evaluate on-the-job performance after training, UCS is now entitled to litigate the suf ficiency of any plan for performing such evaluations. UCS is also entitled to litigate whether the mere preparation of a plan, as required by the Board, constitutes reasonable progress, or whether some extent of implementation is required before such a finding can be made.

In blatant disregard of UCS's rights under the Commission's 1979 order, the Board has simply ruled by fiat that the mere presentation of a plan is enoug h, and it has accepted Licensee's paper arguments in support of its plan, while providing UCS with no opportunity litigate adequacy of the plan in any meaningful 1

Strictly speaking, the Commission referred here to a particular group of long-term actions identified in another section of the decision. Training-related actions are not included in the reference, o f course, because the Commission considered them to be more important sho r t- te rm items, rather than long-term items. If they are now to be considered to have been transferred into long-term items, they must be subject to the requirements that the Commission applied to the long-term items that the Commission considered to be important enough to be subject to the hearing requirements and the need to demonstrate l reasonable progress. i l

)

- way. The fact that UCS was allowed to file written comments on the plan cannot substitute for UCS' right under the Commission's decision and the Atomic Energy Act to a full adjudicatory hearing on the issues.

The Board's decision approving the plan illustrates the importance of upholding UCS' r ig ht to litigate the issue. First, the Board agreed with two of UCS' most significant criticisms of the plan, and it required that Licensee take those actions necessary to respond to the criticisms. In doing so, however, the Board made a number of f actual assumptions about how evaluations can and should be performed. More important, the i Board essentially left the matter to be resolved by the Licensee in its own discretion based upon what the Board viewed as an implicit assurance from Licensee that evaluations will be ,

effective. LBP-85-21, Slip. op. at 10-11. This is an astonishing decision after the Licensee had already rejected UCS' comments that gave rise to the Board's concern.

If, as it was entitled to do under CLI-79-8, UCS had been allowed to litigate these issues, it would have shown not only why the Board should accept its criticisms, but what Licensee should be required to do. The Board would not have been left with so little information that it had to leave the remedy up to the offender.

i a

i

.> ,. C. The Board Erred In Ruling That Submission Of A Proposed Plan Would Constitute Reasonable Progress On A Long-Term Ac tion Item.

Although the Board never directly addresses the point, it's decision can most favorably be viewed as a determination that submission of an evaluation plan would constitute reasonable progress toward completion of what the Board views as a long-term action item. The other possible interpretation, which is more accurate given the language of the decision, is that the Board simply ignored the reasonable progress requirement in allowing restart even before the plan had been drafted or submitted.

As s um ing the former interpretation of the decision, the Board, in effect, decided that the submission of a plan would be enough regardless of what was in the plan. Otherwise, the Board could not have authorized restart immed iately . It would have had to delay restart at least until it had had an opportunity to review the plan under its continuing jurisdiction.

But the mere submission of a plan cannot constitute reasonable progress toward completion of anything. The plan may be utterly inadequate, as UCS believes is true here.

The fact that the Board has now approved the plan does not moot this issue. To the contrary, given the serious weaknesses in the plan, UCS believes that this approval simply confirms that the Board was seeking only some plan - any plan - to justify restart. Stated otherwise, the Board applied the wrong standard to its review of this plan. The fact that it purported to be an adequate plan did not make it one.

i II. The Board's Favorable Decision On Operator Attitudes Is ga gg[gg g,3gg,(([gga l[g,Qgggg@g,Qggg,Rggga gg,[ggg[ f ;,.,,,

Pursuant to ALAB-772, one of the major issues litigated at length in the remanded hearing was the attitudes of licensed reactor operators. See PID a t 49-60. The Board held, in essence, that operator attitudes are now acceptable, and that, "If unwanted operator attitudes still linger at TMI-1, they will be relieved substantially when the operators are permitted to perform the duties they were trained for." PID a t 6 0.

The Board's decision is baseless. It depends upon the testimony of Michael Ross, the TMI-l Manager of Operations, whom the Board itself characterized as " unrealistically def ensive," .

I and on the testimony of the OARP Committee, which initially reached its conclusions as to attitudes without interviewing a I

single operator. It ef fectively ignores the principle, which no one seriously disputed, that the significant question with respect to attitudes is not what they are at a given point, but what the trends are over time. Both Dr. Regan and the NRC Staf f testified that systematic, structured interviews are necessary to make these determinations, but there is no evidence in the record i

of any current surveys of operator attitudes or, indeed, any i

surveys since the so-called RHR Report revealed significant l negative attitudes in late 1982.  !

In addressing the issue of operator attitudes, the Board relied primarily upon the testimony of Michael Ross, the F2 nager of Plant Operations. PID a t 55-57. This reliance was unfounded.

3

Mr. Ross is the person responsible for the operation of TMI-1. Doubtless, he, more than anyone else, is anxious to bring the reactor on line. He is also responsible for the attitudes of the reactor operators, who work for him. Any indication of negative attitudes on the part of the operators would be a negative reflection on FT . Ro ss . Thus, as the Board itself said, he "is hardly a disinterested observed." PID at 57. Yet, as the decision reflects, the Board relied very heavily on his opinions.

The Board's reliance upon Mr. Ro ss is undercut not only by Mr. Ro s s ' general interest and bias, but also by his specific f ailure to see attitude problems where trained professionals had specifically identified them. In 1982, Licensee commissioned a psyc holog is ts' survey of operator attitudes. That survey is now known as the "RHR Report." PID a t 50. As the Board acknowledged, the RHR Report contained many indications of serious problems of employee attitudes at TMI-1. PID a t 51-53.

Although the report has weaknesses identified by the NRC Staff, PID at 53, it is the only independent systematic survey evidence in the record concerning operator attitudes.

UCS does not contend that the RHR Report is determinative of current operator attitudes. Ra the r , it is a significant data point from which to determine trends in attitudes. See PID at

54. The problem here is that it is the only significant data point, and the Board relies primarily upon Mr. Ross to support its decision that attitudes are now acceptable. See UCS Proposed Finding 291. Even the Staf f agrees with UCS on that point. Id.

. _17_

The issue, then, is whether Mr . Ro s s ' testimony on this point is reliable. Mr. Ro s s' reactions to the findings in the RHR report demonstrate his testimony on attitudes is not reliable.

When he first read the RHR Report finding that a majority of TMI trainees and a substantial minority of other personnel thought management was more concerned about generating electricity than about safety, he was not concerned about the finding. He complained first that he had some dif ficulty understanding what the Report was saying and then argued that he was aware of what attitudes actually were, so he was not concerned about the findings. But it is not dif ficult to understand what the report was saying on this point. While the writing is not perfect, it clearly indicates a perception on the part of TMI trainees and many others that management is more concerned with generation than with safety. Moreover, Mr . Ross could have checked with RHR or reviewed the raw data if he was unable to understand the report, but he failed to do either of these. Ross, Tr.

32,565-567.

On questions of the adequacy and relevance of training, Mr .

Ross once again had no concerns about the RHR f indings. To the contrary, his rather creative reading of the report emphasized the fact that training was improving, so the program was doing well. He even managed in his deposition to interpret the language at the bottom of page 28 of UCS Training Exhibit 6 as meaning that forty-nine out of fifty of those who revealed these negative attitudes could have been from Oyster Creek, althoug h l

.' that is simply not a fair reading of the language. When confronted with this unsupportable interpretation, he retreated into his complaint that he has always had trouble reading the RHR Report. Ross, Tr. 32,577-589.2 Mr. Ro s s ' reactions to the RHR report were not reasonable. He consistently read unfavorable language in a f avorable light, and he failed to seek explanations for patently unfavorable statements. Instead, he simply decided he could not understand the report in those areas. Incredibly, he concluded that the RHR Report "was very good and very positive about TMI." Ross, R. 32,573. Whatever the validity of the report, it certainly was not that.

Not surprisingly , Mr . Ross testified that operator attitudes were " absolutely good." His opinion is based upon operators' performance in training and his general interactions with them.

2 Mr. Ross also testified that the RHR conclusion with respect to compliance with procedures was of no concern to him when he first read it. In essence, he had no concern about the matter both because he thoug ht he knew what operator attitudes were on the subject and because he read the RHR report as indicating that a majority of operators agreed that the procedures were complied with. Although RHR reported that only a slight majority had this favorable attitude, Mr. Ross made no ef fort to determine whether this meant, as it easily could have, that a substantial minority believed that procedures were not being complied with. Ross, Tr.

32,567-572. Had Mr. Ro ss inves tig a ted the matter, he would have discovered that a large majority of operators at TMI believed that procedures were being followed, so that this particular issue did not appear to be a problem. UCS Training Exhibit 7.

The problem, therefore, is not with this particular RHR finding, but with the fact that Mr . Ross seems incapable of believing that anything can reflect badly on TMI, even when the language of a report clearly seems to do so. He simply interprets the language in a favorable, if distorted, way, and he does not make any effort to determine whether or not his optimistic interpretation is correct.

. . ,. He has not done any particular interviews for the purpose of ascertaining operator attitudes. In light of Mr . Ro ss ' blind optimism about attitudes at TMI-1, even at the time of the RHR report, his highly subjective judgments today are unreliable. He has made no systematic ef fort to develop the necessary information, and it is clear that anything but the most structured survey would be so colored by his perceptions and his obvious self-interest in reporting favorably about the attitudes of the men who work under his supervision that it is more likely to reflect his favorable views rather than whatever views operators may actually have.

A similar problem exists with respect to the testimony of the OARP Committee, on which the Board also relied in its discussion of operator attitudes. We discuss the Committee's actions in greater detail below. It suffices here to note that the Committee addressed the issue of operator attitudes in the Special Report that it issued on June 28, 1984. S.R. at 72-73.3 Every bit of information on which the Committee based that report came from Licensee management. S.R. at 3, Staff, ff.

Tr. 33,148 at 7, 15, 16. The information known to the Committee did not include the RHR Report. Tr. 31,851, 32,038, Uhrig, Tr.

31,816. Nor did it include any interviews with the operators themselves. Kelly, Tr. 31,818, 31,836-838.

3 The Special Report was received into evidence as attachment 1 to the Committee's direct testimony, which appears following Tr. 31,749. We will cite the Special Re por t as"S.R.".

i

' Yet the Committee responded to the issue of operator attitudes.

And it did so in a report that it knew the Commission might well rely upon as a major basis for approving the restart of TMI-1.

Uhrig, Tr. 31,790-91.

Having taken a stand favorable to the training program, the Committee could hardly back down. There was a virtual imperative for the Committee to support its previous conclusions, which it proceeded to do. But it did so through unstructured interviews that did not meet the criteria that both UCS and the Staf f agreed were necessary to determine operator attitudes. More important, in doing these interviews, the Committee informed the operators that they were preparing testimony for the restart proceeding.

Kelly, Tr. 31,847. The testimony of Committee member Dr. Gardner is uncontradicted in establishing that this sort of introduction may bias the interview. Tr. 33,290. It is hardly surprising that these operators who strongly support the restart of TMI-l exhibited positive attitudes when they knew that to do so would enhance the likelihood of restart.

Ultima tely , neither Mr . Ross nor the OARP Committee are reliable witnesses on the issue of operator attitudes. It is a sad commentary, indeed, on this Licensing Board and this agency that the NRC Staf f itself agreed with UCS that structured surveys are necessary to address operator attitudes, yet the Board relied upon a witness whom it acknowledged to be significantly biased, and upon the OARP Committee, which tainted its own testimony with premature conclusions, signaled operator interviewees that they should exhibit positive attitudes, and f ailed to undertake a systematic and reliable examination of operator attitudes.

i

Finally, the Board relies on the proposition that attitudes will improve once the reactor is restarted. There is no basis for this conclusion, and it cannot support restart. Licensee must demonstrate adequacy of its program before restart. Hopeful suppositions are not enough.

The Board's decision on this issue should be reversed. If the NRC is to have any credibility, it must base its decisions in this, of all cases, on objective evidence. That evidence is readily obtainable through the use of carefully designed surveys. The case should be remanded to the Licensing Board with directions to order the implementation of such a survey, to be followed to periodic surveys to provide the trend data that all parties agree are necessary. Restart may be authorized only if these surveys reveal acceptable attitudes on the part of the reactor operators.

III. The Board Er red In Relying Upon The Testimony Of The Be E9n s 513used,g ASP ,Cggm{ 33 g g . ,,,,,,,,,,,,,,,,,,,,,,,

In ALAB-772, the Appeal Board reversed the Licensing Board's previous training decision largely because the Licensing Board had relied upon the testimony of the OARP Committee, which had not had the opportunity to take the cheating incidents into account when it testified. Thus, the Appeal Board specifically soug h t the testimony of the Committee on various issues raised by the cheating incidents.

Licensee presented, and the Licensing Board accepted testimony from the Reconstituted OARP Review Committee. Licensee argued that the Committee's testimony constituted an

.' independently sufficient ground to approve Licensee's training program. PID at 209. In a Solomonic response, the Licensing Board "(could] not find that [the Committee's] findings standing alone would have satisfied all of the concerns expressed in ALAB-772 or would have established independently the adequacy of the licensed-operator training program," but it held that, " Th e Review Committee and its members responded appropriately to the questions put to them in ALAB-772." PID a t 213. In addition, the Board held that, " Licensee could not have prevailed on the ALAB-772 issues without the Review Committee because the Appeal Board expressly called for the opinions of its members." I d,.

The Board's rulings are in error. First, they are on their face inherently inconsistent. If the Committee's testimony did not satisfy the concerns raised in ALAB-772, it could not have been adequate to constitute an appropriate response to that decision, and thus it could not have been adequate to support a f avorable decision on the training program.

More important, for reasons that relate directly to the credibility and legitimacy of the Commission's decisions, the Board should have given no weight whatsoever to the Committee's te s t imony . Ra the r , for the reasons discussed below, the Board should have held that the Committee's testimony was of no worth to the proceeding and should have ordered Licensee to obtain the services of an independent group of similar experts to address the issues raised in ALAB-772.

The fundamental flaw in the Committee's testimony is the fact that the Committee prejudged the adequacy of Licensee's training

program before it had enough information to make a reasoned judgment. As discussed in detail in UCS' Proposed Findings78-113, the Committee was not brought back together until late May 1984. UCS' Proposed Findings78-113 are attached as Exhibit 1 to this brief. Based solely upon descriptions of the program by management personnel and other information provided by management, the Committee issued a "Special Report," in which it stated without qualification that, "the GPU Nuclear training program produces qualified operators and is adequate to support the restart of TMI-1." S.R. at 83. The Committee did not interview anyone other than management personnel in the course of preparing this report. Nor did the Committee undertake any independent review of the training program while preparing this report. See UCS Findings78-113.

In preparing the Special Report, however, the Committee knew that Licensee would rely upon the report in arguing for approval of restart in an impending meeting of the Commission scheduled for June 27, 1984. Indeed, the Committee stated that it had prepared the Special Report "specifically for the [ then) impending Nuclear Regulatory Commission meeting" at which the Commission had indicated that it would " formally consider the issue of restarting TMI-1." S . R. a t 3, Uh r ig , Tr . 31,79 0. Dr.

Uhrig and the other members of the Committee were unquestionably aware that the Commission might well rely upon their Special Report in the June 27 meeting in deciding whether TMI-l should be restarted. The very purpose of the report was to persuade the Commission to vote to restart TMI-1.

1

.. ,. This scenario raises very serious questions about the credibility and legitimacy of the Special Report, the Committee's l tertimony, and the Licensing Board's decision. The Committee

! knew that its findings on the adequacy of the training program at TMI-l could have a direct ef fect on the public health and safety. It necessarily follows that the Committee was aware of 1

the need to undertake the investigation necessary to. assure that the conclusions in the Special Report were accurate and well founded.

l l There is no significant dispute that the Committee did not do l what was necessary to support the conclusions stated in the l

l Special Report. Certainly the Doard makes no finding that the l

conclusions in the report were adequately supported. PID a t 173. See Exhibit 1, UCS' Proposed Findings99-113. Rather, the l Board's response is to emphasize the Committee's passing comments earlier in the Special Report, that the report was a " quick response." PID a t 171. The Daard erra in excusing the Committee's actions on this basis. When the Committee wrote the Special Report, it knew that the Commission would rely upon what it said as a considered judgment on the basis of which the Commission could confidently support a restart decision. The Committee could not expect the Commission to take the " quick i

! response" comment as a qualification of the Committee's ultimate conclusions. Since the Committee itself expressed no doubt that

! it had done everything necessary to support its conclusions, it must have expected the Commission to accept those conclusions as they were stated, without qualification.

. The Committee's willingness to approve the training program based solely on information provided by Licensee management f atally taints the Committee's testimony. In this, of all cases, the Commission must demand objective information. It must assure that its decision is credible and is perceived as legitimate by the public. Having committed itself to broad favorable conclusions on the basis of inadequate information, the Committee lost any credibility it might have had. The Committee's testimony now appears as a predetermined conclusion whose purpose is to assure restart of THI-1, rather than to review the training program with a critical and objective eye.

The fact that the Committee later did additional work does not alleviate this problem. It serves only to demonstrate still further that the Committee did not undertake an adequate investigation in the first place.

In addition to the residents of the Three Mile Island area who will be exposed to radiation released by inadequately trained personnel, the victim of the Licensing Board's decision is legitimacy and the future of the Nuclear Regulatory Commission and nuclear power in the United States. As the Commission continues to accept baseless, predetermined " expert" tes t imony rather than insisting upon objective information, its credibility and legitimacy will decline to the point where the public will not accept its actions.

UCS urges the Appeal Board to hold that the testimony of the Reconstituted OARP Committee was not entitled to any weight in

i l * * . i thie proceeding, and to remand this case to the Licensing Board.

l The Appeal Board should prohibit restart until an independent i l

group of experts has done what the Committee was supposed to do ,

and has legitimately addressed the concerns raised by the Appeal j l

Board in ALAB-772.

IV. The Licensing Board Erred In Approving Licensee's I Examinations In Light Of Licensee's Dependence Upon En ts !!* bis.9Ea l.Ex am lea s190 s .. ..... ... .............

The Licensing Board accepted as an "important point" UCS' >

argument that the use of a variety of tests or examinations ,

serves the purpose of each type of test validating the other.

PID at 120. Thus, the various written and oral examinations and simulator tests must themselves be well developed and P

administered in order to serve as cross-checks on the adequacy of the other types of examinations. Indeed, the Licensing Board l emphasized that where "there is no job performance validation of training and testing, any failure of the tests to cross-validate l each other would assume major significance." Id.

llaving accepted this undisputed principle, the Board then erred in accepting Licensea's oral examinations as "a final check I

on the written and simulator tests," and as "well structured and logically executed." pID a t 132-133. The Board rejected the need for specific training of those who administer oral  ;

examinations and for objectivity in the administration of the

\

examinations. In essence, the Board approved reliance upon the  ;

l unchecked and untrained judgment of those who supervise licensed l

l reactor operators.

l

, l

.- The record contradicts the Board's conclusion. Dr. Reg an specifically testified to the serious weaknesses in oral examinations. In particular, they tend to lack standardization 1

and specificity, thereby allowing extraneous influences such as personal interaction to influence the outcome. Oral examiners must be trained in the art, and there must be clear criteria for evaluating operator performance. Regan, f f . Tr . 3 3,532 at 15-16.

These points are essentially undisputed on the record. The Board relics, however, upon th'e testimony of 72. Ross to the usefulness of oral examinations as a means of exploring what an operator knows. PID a t 128-131. This reliance is misplaced in light of the f act that Mr. Ross himself is not trained in the administration of oral examinations. Ross, Tr. 32,601-602.

Thus, while Mr . Ross may have an interesting conversation with an operator that appears to reflect the operator's knowledge, Mr .

Ross is not trained to administer the oral examination in such a way as to assure that he is actually testing an operator's knowledge, rather than reading his own knowledge reflected in the conversation. The Board conveniently ignored this point in relying upon Mr. Ro s s .

Oral examinations are the second major type of examination relied upon by Licensee to test trainee knowledge and to validate other types of examinations. Licensee gives a comprehensive oral examination to each trainee at the end of each replacement program. This consists of an examination on f undamentals given by a Board of two to three people and a one-on-one examination that involves a plant walk-through and covers plant systems

.. operating procedures. Licensee gives a similar oral examination in each annual requalification cycle, except that the fundamentals are incorporated into the one-on-one walk-through, and there is no Board examination. Leonard, Tr . 3 2,532.

The oral Board examination given under the replacement training program is administered by representatives of training and operations. Tr. 32,535. The one-on-one oral examination may be administered by licensed operators or licensed or certified instructors. The mix of training personnel and operations personnel who administer the one-on-one examination may be anything from all of one to all of the other, so that operations personnel may adminster many or even all of these examinations.

Leonard, Tr. 32,538. In addition, Mr . Ross typically does another one-on-one walk-through examination, although this is not required by the applicable procedures. Ro s s , Tr. 32,535.

The situation is different in the requalification program.

There, Mr . Ross and other supervisory members of the operations department administer the oral examination. Ro s s , Tr . 3 2, 5 3 9.

There was no testimony as to whether training department personnel who administer oral examinations have any special I qualifications or training to do so. The record establishes, however, that none of the operations personnel who administer these examinations, includ ing Mr . Ro s s , has any special training to do so. Rather, Licensee relies upon their subject matter expertise in the areas in question. See e.g., Ro s s , Tr .

32,601-602, 32,615.

l, a

r

r <

Not only do the oral examiners have no particular training

~

for this complex task, but both 'the examinations themselves and -

the grading are highly subjective. As shown by UCS Training Exhibits 20 and 30, the oral examination check lists provide some rudimentary guidance as to the . subjects that are to be covered in oral examinations. They do not, however identify what aspects of particular subjects are to be discussed on the examination, nor do they assure that examiners will ask the same questions of each candidate. It is lef t to the judgment of the individual L

adminstering the examination how to grade the issues that he is to cover. Leonard Tr. 32,611.-

Rema r kably , there are no standards by which to determine whether an individuals' grade on an oral examination constitutes passing or failing. There are three categories of grades on particular subject areas: satisfactory, marginal, and unsatisfactory. There is no requirement, however, that a candidate achieve any particular number or percentage of satisfactories in order to pass, nor are there any criteria

-requiring particular performance within the categories covered on the oral examination. Indeed, 60% right and 40% wrong could be a i

pass. At one point in his deposition, as he later admitted in ,

his testimony, Mr . Ross indicated that a majority, which he undoubtedly understands to be anything greater than 50%, might be sufficient to pass the examination. Ross, Tr. 32,603-618. Thus, whether an individual passes or fails is totally subjective and within the discretion of the examiner.

I

The significance of the percentage figures to which Mr . Ro ss testified is not that-the Licensee will pass-all trainees who obtain a bare majority on the oral examination. Ra the r , it demonstrates the utter subjectivity of the process and the decision. As- explained by Mr . Ro s s , the oral examiner makes a judgment whether the individual knows 80% of the material being tested. Ro s s , Tr . 3 2, 618. Thus, under the current system, a candidate could well achieve 60% correct answers, yet be found to have an 80% knowledge on the basis of the examiner's opinion of the situation. In his deposition, Mr . Ross characterized this as an " arbitrary judgment" on the part of the examiner. On redirect e xamina tion, he sought to change that characterization to a

" subjective" judgment. Ro s s , Tr . 33,067. His initial characterization was more accurate. The very fact that the judgment is so subjective, particularly when there is virtually no standardization of the examination itself, renders the ultimate grading decision an extremely arbitrary judgment.

This problem is compounded by the fact that there is no review of the oral examinations. The supervisor of licensed operator training checks the documentation, but that is nothing more than a bare checklist. Tr. 32,540-546, UCS training Exhibit

30. Al thoug h Mr . Ross presented conflicting testimony on the subject, it appears that no one reviews the questions or answers with either the examiner or the examinee. Ro s s Tr . 32,602-608.

Dr. Regan was extremely critical of the use of oral examinations unless they meet exacting requirements, includ ing particularly standardization of subjects and questions in order o

to minimize the subjective nature of the effort. Re g an , ff. Tr.

33,532 at 15-16. These oral examinations serve a crucial role of validiating the other examinations used by Licensee. They are not narrow inquiries into specific identified subjects. Ra the r ,

as Mr. Ross emphasized, they permit in-depth discussion and probing in to the candidate's understanding of the f acility.

Moreover, most of the oral examinations, particularly in the requalification program, are administered by operations personnel with no training in administering oral examination. And those very people are also supervisors of the candidates, althoug h they may not be supervising the particular shift at the time they give the exam.

The oral examinations thus suf fer f rom precisely the problems that Dr. Regan clearly identified. The examiner may be influenced by the comparison, favorable or unfavorable, between the performance of one candidate and the performance of the previous candidate. An examiner who is not well trained in oral examinations, or at least in interview technique, will tend to do too much talking. The examiner may be influenced by the " halo e f f ec t" in which he may misjudge the overall performance because f

i an individual does very well in one particular area. And, s ig ni f ican t here, there is the problem of co-workers

administering the examinations, which may result in the intrusion l

l of extraneous personal considerations, as when a congenial i

personality results in a higher grade than justified by a candidate's knowledge. Re g a n , ff. Tr. 33,532 at 15-16. In lig h t of these considerations, the lack of standardization of the B

,. questioning, and the arbitrary nature of the grading. The Licensee's oral examinations are of no use in validating other examinations or in determining or predicting operator performance with any reasonable precision. The Licensing Board's decision to the contrary was incorrect.

In sum , the oral examinations administered by Licensee are so deficient as to be. useless either. as independent examinations of operator knowledge or as checks against the accuracy of other examination methods. The Board dismisses this problem by minimizing the significance of oral examinations, arguing that they are not supposed to be predictors of operator performance on their own, and that their real use is to validate the other examinations.

There is no support for this conclusion. In order to validate the other tests, the oral examination must be adequate in its own right. There is no testimony or other evidence to the contrary.

The crucial point, once again, is whether this Commission should permit the public health and safety to depend upon extremely subjective judgments, in this case the judgments of supervisory operations personnel whose daily personal relationships with the operators necessarily color their administration of Licensee's formless oral examinations. The Commission must demand more than this.

UCS urges the Appeal Board to reverse the Licensing Board on r

this issue. The Appeal Board should hold that Licensee's

examination process is inadequate because it depends upon unreliable oral examinations.

gggg}gsigg For these reasons, UCS urges the Appeal Board to reverse the Licensing Board's favorable ruling on the adequacy of the licensed operator training program at TMI-1.

Respectf ully submitted, 6 - W ' f k pl William S.,Sordan, III HARMON, WEISS & JORDAN 2001 S Street, N.W.

Suite 430 Washington, D.C. 20009 (202) 328-3500 Dated: July 1, 1985 I

i 1

• ,0 EXHIBIT 1 UNION OF CONCERNED SCIENTISTS PROPOSED FINDINGS OF FACT AND CONCLUSIONS OF LAW ON THE ISSUE OF LICENSED OPERATOR TRAINING AT TMI-l FINDINGS 78 - 113 1

i f

78. The Committee's methodology in preparing the Special Re po r t is most clearly established by a review of the Committee's actions f rom May 30, 1984, to June 28, 1984, the day the Special Re po r t was completed. Committee members were contacted either by the company or by Dr. Uhrig within approximately two weeks before Ma y 3 0, 1984. Tr. 31,788-790. When Dr . .Long contacted Dr .

Uhrig, he explained that the purpose of the meeting was to answer some of the issues raised in ALAB-772 and that this needed to be

(

- e done very-quickly so t, hat the report would be available for a 1

meeting of the Commission that was then scheduled for June 27, 1984. As a result of that discussion, Dr. Uhrig understood that the Commission might vote on whether to restart TMI-l in the same meeting in which the Commission considered the Committee's report. Uh r ig , Tr . 31,790-91. Al thoug h it is not clear how other Committee members were informed of the purpose of the Special Report and its involvement in the Commission's meeting, they testified that their understanding of the matter was the same as Dr. Uhrig's. Ici .

79. Four of the Committee members first met as a Reconstituted Committee on May 30, 1984, at Three Mile Island.

The meeting lasted from May 30 to June 1, 1984. Dr. Kimel had a conflict and was not present. In addition, Dr . Uhrig wa s not present on June 1, 1984. Tr. 31,789, 31.791-92. Thus , Dr . Uhrig spent two days at the meeting, while the other three participants spent three days each, for a total of 11 days spent by Committee members during the first meeting.

80. The first time that the Committee received any informa tion about the current training program a t TMI-l wa s a t 1

the meeting on :b y 30. "2 Tr. 31,792-793.

1 This does not, of course, includ e information obtained by the Committee in the course of its original worx several years earlier. It also does not include information that Dr . Ga rdne r may have obtained in the course of giving a seminar on test Dr.

construction some two yea rs ea rlie r . Gardner, Tr . 31,792.

Gardner did not testify that he received any significant in torma tion dur ing that seminar, and the Eoard concludes that the first time that any member of the Committee received and began to review information signiticant to this proceeding wa s :a y 30, 1984.

81. During the meeting of May 30 - June 1, 1984, at Three Mile Island, the Committee received various briefings from Licensee management personnel, toured the f acilities at Three Mile Island, and had individual discussions with various management personnel whom the Committee identified in the Special Report. Uh r ig , Tr . 31,7 9 3.

82. As far as we can tell, the Commi ttee spent approxima tely half of its time from Pay 30 to June 1 in meetings with GPU management. In addition, the Committee appears to have spent between one-third and one-half of the remaining time during that period in discussions with management. Tr. 31,794-796.13 Thus, the Committee spent between two-thirds and three-quarters of its time f rom P.ay 30 to June 1 with Licensee management personnel. The Committee appears to have spent the some additional time reviewing documents, particularly ALAB-772, which Committee members saw for the first time on Ma y 3 0, 1984.

According to Dr. Gardner, however, the purpose of this meeting was to determine what the Committee would focus on and what approach it would use in attempting to respond to ALAB-772.

Gardner, Tr. 31,797. Finally, the Committee assigned tasks for preparation of the Special F.eport, including which members of the Committee were to draft which sections of the report. Tr.

31,793-798.

13 Dr. Christensen appears to have estimated that the Committee spent slightly less time in briefings with Licensee management personnel. Tr. 31,796. Altnough the Eoard has no doubt or Cr. Christensen's integrity, he consistently revealed a poor mencry for specific facts and dates. See, e.c., Tr.

31,522-624. Thus, the Board finds .ha t in ec tima te consistent witn figures given by Dr. Uhrig and ::r . Ke lly is more likely to be correct.

83. When the Committee members lef t Three Mile Island, they appear to have taken a number of documents with them for review.

Dr. Uhrig, in particular, took home approximately five inches of material. Uh r ig . Tr . 31,796. Af ter he lef t Three Mile Island on Fay 31, Dr. Uhrig spent some 12 to 15 hours1.736111e-4 days <br />0.00417 hours <br />2.480159e-5 weeks <br />5.7075e-6 months <br /> reading these materials before the Committee reconvened on June 6. Dr.

Christensen could not recall how much time he spent on this activity, or whether he read all of the material that he took with him. Mr. Kelly estimated that he spent about 10 hours1.157407e-4 days <br />0.00278 hours <br />1.653439e-5 weeks <br />3.805e-6 months <br /> reading TMI-l materials during this period, and Dr. Gardner estimated that he spent two or three days in that activity. Tr.

31,800-801.

84. Converting these estimates to a reference point of days, Committee members spent some 5 to 6 days on their TMI-l work during this period, plus an unknown amount of time put in by Dr.

Christensen. The Committee must have spent all of this time reviewing information that it had received during its first meeting because no member of the Committee received any further inf ormation before the Committee's second meeting began on June 6, 1984. Tr. 31,799-800.

85. The Committee reconvened on June 6, 1984, in Farsippany, New Jersey. This time , the entire Committee was present, includ ing Dr. Kimel. Chrig, Tr. 31,801. Dr. Kimel first received information concerning his work with the Committee when he arrived on June 6. Kimel, Tr . 31,800. All members of the Committee were present for the entire time, except that cr. Kimel

f visited Three Mile Island on June 7, 1984, the second day of the

( meeting. Tr. 31,802. Thus, the Committee spent a total of 15 days on its TMI-1 work during the meeting of June 6-8, 1984.

86. On June 6, 1984, the Cormittee spent most of its time throughout the day discussing ALAB-772, including who was going to prepare material dealing with particular issues, what the general approach should be, and what the approach should be for responding to particular issues. Uh r ig , Tr . 31,803.

87. On June 7, 1984, while Dr. Kimel visited Three Mile Island, the remaining members of the Committee held a series of meeting at which they discussed how they were progressing. They appear to have spent time individually preparing the report, including writing their portions of it, and meeting periodically, which they did three or four times during the day. According to Dr . Uh r ig , the day was basically spent "in our own wr iting --

reading and discussions," the " iterative process" by which the Committee developed and wrote the report. Uhrig, Tr. 31,804.

88. While at Three Mile Island, Dr. Kimel talked with Mr.

Newton and Mr. Gif ford, a GPUN Vice-President who is responsible for communications at TMI. He also talked with Mr . Ir iza rry and Mr. Eoltz, who demonstrated the Ea sic Principles Training Simulator. Kimel, Tr. 31,803-804. He mu s t have been there approximately half a day since he left Newark Airport f or TMI a t 8:00 a.m. and started back f rom TMI a bout 1: 0 0 p .m . He rode back from Three Mile Island with Mr. Newton, with whom he discussed the remanded issues. Upon his arrival in Pa rsippany,

l' the Committee. The purpose of those conversations and of the remainder of his work during this period was to discuss and edit the draft, which Dr. Coe had sent to the other Committee members. As far as we can tell, Dr . Uh rig spent approxima tely .

two days on Committee work between the time he left Parsippany on June 9 and when he returned on Wednesday, June 13, 1984. Uhrig, Tr. 31,810-811, 31,807.

93. While he was in Parsippany on June 13 and 14, 1984, Dr.

Uhrig was involved in continuous editing and drafting of the Special Report based upon his telephone conversations and some material that Committee members had mailed in. Uhrig, Tr.

31,807. By the end of June 14, 1984, the Committee had pr9 duced a semi-final document, after which final editing was done over the telephone with other Committee members. Id,.

94. Af ter Dr. Uhrig lef t Parsippany on June 14, 1984, he spent some 20-25 hours on the Special Report before the document left his office on June 28, 1984. This transla tes into 2 1/2 to 3 days. Uhrig, Tr. 31,811.

95. Dr. Kimel estimated that he spent approximately three days working on the Special Report between the time he left Farsippany on June 8, 1984, and June 28. Kime l, Tr . 31,812. Dr.

Ch ristensen estimated that he spent 75-60 percent of his working hours on the Special Report between June 8 and June 28, plus additional hours on weekends, and probably work hours that he did not record. Christensen, Tr. 31,312-813. Re f erring to a ca le nda r for the pericd, this translates into somewhere between

(~ -s2-10 and perhaps 15 days. Mr. Kelly spent about two days on the project during this period, Tr. 31,813-814, and Dr. Gardner spent 10 to 15 days on the Special Report between June 8 and June 28.

Ga rdne r , Tr . 31,814.

96. Based upon the Committee's testimony, we are able to determine that the Committee spent between 63.5 and 75 days on the Special Report. We cannot determine precisely how many of those days the Committee spent before June 12, the day the Committee originally expected to complete the report, Uhrig, Tr.

31,807-808, and June 28, the day the report was finally completed. At best, we can estimate that Dr. Ga rdner must have spent a- minimum of 9 to 11 days on Committee work af ter June 12.

$ (He spent 10 to 15 days after June 8, and there were only four days between June 8 and June 12.) Dr. Christensen must also have spent a minimum of 9 to 11 days after June 12. ft . Kelly could have spent all of his time before June 12. Dr. Uhrig spent 2 1/2 to 3 days after June 14. Dr. Kimel could have spent all of his time before June 12. Thus, subtracting the time that was or must have been spent af ter June 12, we conclude that the Committee could have spent approximately 45 to 50 days on the special Re po r t before June 12, the date the report wa s originally due.

97. The time spent on this report is significant because, as we have previously emphasized, the Committee knew that the Commission might rely upon this report in deciding whether to allow the restart of T1I-1. The figures in the preceeding paragraph demonstrate one disturbing thing. The Committee was l

l t

. - - . . - - - . , . . ~ , ,.-,,.- ~,- - - - - - , - - - - - - - - - . , . - . . - , - - , , - - - - , . . - - . - - - , . - , - - . - , - . . ,

I presumably prepared as of June 8, 1984, to file the report on the original deadline of June 12. Only on June 8 did the Committee know that more time was available for its work. Committee members then found it appropriate, and presumably necessary, to spend some 18.5 to 25 additional days on the report, which they could not have spent if the deadline had not been relaxed. That is between 29% and 33% of the total time spent by the Committee.

Although hardly conclusive, it is disturbing that the Committee was prepared to issue the Special Report, with its unqualified conclusions and potentially significant influence on the Commission's decision, without doing a third of the work that it eventually felt necessary to do on the report.

98. Our concern is only increased when we consider what the Committee actually did in preparing the Special Report. As the Committee's testimony reveals, and as the Staff has testified, S.R. at 3, and Staff, f f . Tr . 3 3,14 8 a t 7, 15, 16, the Committee obtained information only f rom Licensee management personnel, and it spent its time examining documents rather than undertaking the investigation necessary to verify .anything that it learned. In fact, a review of the time spent by the Committee reveals that it began drafting the report on June 7, 1964, after the Committee members had spent only some 27 days on the project. At tha t point, Dr. Kimel had received relevant information only the previous morning, yet he participated in the drafting. The Committee spent a large part of its remaining time drafting and editing the report, so that the time that the Committee actually spent in any sort of investigation must have been minimal. We

l

• believe it would be charitable to conclude that the Committee

' spent half of its time obtaining and analyzing information, as opposed to discussing the issues, deciding how to proceed, and massaging the drafts. Although we do not consider ourselves to be bound by Dr. Regan's estimate on this point, we find it both instructive and disturbing that if we are correct in our estimate, the Committee spent between 10% and 13% of the time that Dr. Regan estimated would be necessary to do an adequate study. Re g a n , f f. Tr . 3 3,532 a t 22. We do not believe that Dr.

Regan's estimate could be in error by a factor of 10.

11. The Committee's methodology in preparing the Special Report wa s inadequate.

99. Our review of appropriate actions that the Committee failed to take in preparing the Special Report further heightens our concerns:

1. The Committee did not review any actual training in the licensed operator training program, whether in the classroom or at the simulator.  !!o member of the Committee even visited the B&W simulator.

Uh r ig , Tr . 31,816, 31,823, Ke lly , Tr . 31,818,-919 Ch r istensen , Tr . 31,822-24, 31,905-906.14 14 Christensen exhibited considerable confusion about when Cr.

he first monitorcu a training class. Tr. 21,822-24. He ultima tely confirmed, however, that he did not attend any classes until August 1984. Tr. 31,905-906. Dr. Gardner testified that he did sit on one class, but it wa s not part of the licensed operator training program. Tr. 31,82C-821.

I

= .

2. The Committee did not interview any trainees or operators who were not part of Licensee management. Uhrig, Tr. 31,816, Kelly, Tr. 31,818, 31,836-838.15

3. The Committee did not undertake any evaluations of licensed operator training instructors. Uhrig, Tr.

31,825. .

4. The Committee did not review any particular job-task analyses. Uh r ig , Tr . 31,825.16

5. The Committee did not review operating procedures, emergency procedures, or ATOG procedures to determine if the training program was consistent with those procedures. Uh r ig , Tr . 31,823-824. In fact, neither Dr. Uhrig, Mr . Kelly, nor Dr.

Christensen reviewed these procedures at all.

Uh r ig , Tr . 31,816, Kelly, TT. 31,618, Ch r i s tense n ,

Tr. 31,822. We have no evidence in the record as 15 Mr. Kelly did discuss operator attitudes with Mr . Boltz, a simulator instructor , but the conversation wa s apparently limited to the proposition that a lack of turnover among operators indicated that morale was high. We have no evidence that Mr .

Kelly addressed any other issues wi th Mr . Eo l t z , includ ing attitudes toward the training program, reactions to the cheating incidents, or anything of that sort. Tr. 31,638.

16 Dr. Kimel asserted that he did peruse the Plant Cperations Ma n ua l, whicn " looked like task analysis" to him. He apent about half an hour in this perusal. Tr. 31,825. The Flant Operations Ma nua l, which is based on learning objectives and thus would at least be relevant to job-task analyses, is 9 volumes and several feet thick. Kimel, Tr. 31,826. While we were impressed witn Cr.

Kimel's backg round and qualifica tions, we cannot imagine that he undertook a serious review of either ;ob-task inalyses or the Plant Cpe ra tions '!a nua l ttself in that ha lf nour . (

{

to whether Dr. Kimel or Dr. Gardner reviewed these procedures at all before issuance of the Special Re po r t , b u t it is clear that without nuclear expertise, Dr. Gardner did not have the competence to undertake such a review, Gardner, Tr. 31,822, and Dr. Kimel did not have time to do an extensive one if he did any. Since Licensee must carry the burden of proof and has not presented any proof on the point, we conclude that no member of the Committee reviewed the procedures in question before issuance of the Special Report.

6. The Committee did not observe the administration of any examinations of any sort. Uh r ig , Tr.

31,824.17 17 The Special Report asserts that Dr. Ga rdner observed the administration of two oral examinations during the CARP Committee's original work. Sr. at 50. We took this to mean that Dr. Ga rdner had seen oral examinations administered by the Licensee's personnel, which would make the testimony relevant to this proceeding and could support the Committee's favorable conclusions. Indeed, that is the only reasonable reading of the Special Report. We are quite disturbed to learn that the oral examinations discussed in the Special Report were not even administered by Licensee personnel, but by Mr . Kelly and one of his employees. Gardner, Kelly, Tr. 31,820. If the Committee had read its Special Report with care, it would have known that we and the Commission would interpret this aspect of the Special Re po r t as supporting the Licensee's training program, when it did no such thing. We doubt that the Committee intended to be deceptive, but the report is deceptive on this point. Th is sort of carelessness seriously undermines the Committee's presentation and testimony, particularly because the Committee relied so heavily on its subjective Judgment and its expertise, as opposed ,

to data obtained through the careful and standardited procedures proposed by Dr. Regan and the !!RC Staff.

i i

l l

b

l *

7. Only Fk . Kelly appears to have reviewed any written examinations, two reactor operator and two senior reactor operator examinations in the 1982/1983 requalification cycle. Tr. 31,814-815.18 Mr.

Kelly first testified that he took tne examinations with him for extensive review, but not for input into the Special Report. Id . When asked to expand on that answer, Mr. Kelly simply stated that he did review those examinations prior to the Special Re po r t . Kelly, Tr. 33,275-276. We cannot determine on the basis of this testimony whether Mr. Kelly undertook any review of examination construction, balance of questions, potentially excessive memorization, technical accuracy, or any of the other matters at issue in this proceeding.

8. Dr. Uhrig, the Chairman of the Committee, did not spend any significant time reviewing training materials, and he spent none reviewing curricular materials. Uh r ig , Tr . 31,815. He also did nothing to check the accuracy and consistency of the curricula materials or examinations against the 18 The record is unclear as to whether Cr. Ga rdner reviewed any written examinations before issuance of the Special Report.

At one point he testified that he did not,Tr. Tr 31,619.

. 31,315,It butmakes at no another point he testified that he did.

difference, however, because he also testified that at the time the Special Eeport wa s issued he had reached no judgment as to whether the examinations adequately covered the ma terial, or whether they required the proper balance of mental activities.

.Ld.-

actual design of the plant. Uhrig, Tr. 31,817.

! There is also nothing in the record to suggest that any of the other members of the Committee undertook such an ef fort prior to issuance of the Special Re po r t . We are constrained to conclude that the Committee undertook no such review if it presented no testimony to that ef fect af ter the question was raised. Mr. Kelly's testimony that he did not review any training program descriptions prior to issuance of the Special Report, Ke lly , Tr .

31,818,19 is consistent with our , conclusion on this point.

9. Prior to issuance of the Special Report, no member of the Committee did anything to evaluate the consistency of the examinations and answer keys with the actual current T!!I-l design. Uhrig, Tr.

31,826.

10. Prior to issuance of the Special Report, the Committee did not review the responses by TMI-l operators to the RHR Report, which had assessed employee attitudes with apparent negative results in 1982. Chrig Tr. 31,826. Since the Committee obtained all of its information with respect to 19 Mr. Kelly la ter testified tha t he reviewed the scope and content of the requalification program in connection with his review of examinations. Tr. 33,283. Since he reviewed examinations both before and after issuance of the special report, tnis is consistent with his sta tement that he did not review program descriptions ;rior to .2suance of the spect31 repcrt. Thus, we conclude that he must have examined tne program descriptions after issuance of the spect31 report.

( -59 operator attitudes from management parsonnsi and did not even attempt to interview the operators themselves, the Committee should have insisted upon reviewing any documentary evidence related to operator attitudes. Since they did not even learn of the existence of the RHR report in any detail until October, Tr. 31,851, 32,038, they clearly did not assure that they had seen all of the relevant evidence.20 100. In light of these deficiencies, the Committee's work in preparing the Special Report was grossly inadequate to reach an independent judgment on the issues remanded in ALAB-772 or to support the unqualified conclusions presented in the Special Re po r t . A comparison to the methodologies discussed by Dr. Regan and the NRC Staf f reveals that the Committee did virtually none of what it needed to do.

101. A summary of Dr. Regan's specific recommendations appears in Findings 50-56. These actions, or some substitute for them, are necessary elements of an evaluation of a training

'O Licensee sought on cross-examination to minimize the significance of tne Committee's failure to review the RER report. In essence, Licensee's point is that later information about attitudes in MUREG-0680, Supplement 4, may estaclish that the RHR report findings were incorrect, or at least that the situation was not as bad as it might have seemed from reading the RHR r eport itself. Tr. 33,235-245 Th is e f f o r t is irrelevant to the issue here. Cur concern is whether the Committee undertook an independent and sufficient effort to evaluate the TMI-1 training program. If tne Committee failed to review information such as the RHR report and make its own independent Judgment ot the significance ot the intormation, we question wnetner the Connt ttee's actions we re sufficiently free of Licensee ave been influence. The argument that the Committee so cia nce concerned about the ?.Hn r epor t does not alleviate tais concern.

This ;s another examplo of the need for care and ccJectivitf in a review of tnis sort.

\

{ . .

program. In the context of TMI-1, review of job-task analysac and learning objectives is probably most important because those are the primary means by which the Licensee seeks to assure that its operators are adequately trained to operate the plant safely. The Committee did not review these matters for the Special Report. Nor did it attempt to evaluate the quality of the instructors, observe the use of the simulator, review examinations to any significant extent, or interview trainees to determine their attitudes toward the training and the jobs. Most important, the Committee did nothing that we can identify either to investigate the correlation between performance in training

. and performance on the job or to determine whether Licensee undertakes such a correlation. Th a t is the single most important question, and the Committee f ailed to do what was necessary to answer it.

102. A review of the Staf f's proposed methodology as set out in Findings 61-70 leads to similar conclusions. The Committee did not do most of the interviews discussed by the Sta f f, but relied in the Special P.eport entirely on information from Licensee management, which can hardly be characterized as an The objective source of information in this particular context.

l Committee did no quality assurance checks, either formal or  ;

informal, to assure that the information and assurances provided l

j by the Licensee were correct. The Committee did not review job-tank analyses and related documents, nor did it even observe any training. The Committee did nothing beyond its general document review to addresa the issues of excessive meme r t :s tion

or impacts of and potential for cheating. The Committee did not

' observe the administration of any examinations or any other evaluations on which the Licensee relies to decide whether to permit trainees to become qual,1fied reactor operators.

103. Finally, we refer to what we identified in Finding 76 as the essential elements of an adequate methodology. The Committee met none of these relatively general standards. It did nothing to assure that various aspects of the training program work as intended. Not oily did it fail to seek objective information, it did not seek any independent informa tion, however subjective. It relied upon documents and assurances provided by Licensee. It failed to obtain and make an independent judgment .

as to the significance of all relevant information. Since it did not review the job-task analyses and learning objectives on which the TMI-l training program is based, it failed to cover even the most important issue before it.

104. One major f ailing of the Special Report is representative of the Committee's overall failure to undertake a thorough and independent review of the TMI-1 training program.

At page 17 of the Special Report, the Committee discussed the qualifications of Edward J. Frederick, then Supervisor of Licensed Cperator Training at TMI-1 Edwa rd J. Frederick (Supervisor, TMI Licensed Cperator Training) provides the TMI training program with extensive education and experience in nuclear powe r plant operation. lie is a graduate of the Navy Nuclear Power Program and has completed courses to qualify him as a certified TMI-L senior reactor operator inceructor and is a licensed TMI-2 senior reactor opera tor.

t

S.R. at 17. The Committee f urther emphasized Mr . Frederick's significance to its review by noting that "as an experienced licensed TMI station operator, (hel directs and monitors the TMI-licensed operator program so as to ensure orderly transition f rom the Navy program to the TMI program. Additionally, his experience during the TMI-2 accident and his subsequent education and training on small break LOCA phenomena qualifies his credentials in this area." Id . a t 18.

105. This language indicates that Mr. Frederick plays an extremely important role in the training program. The committee identifies attributes and asperience on his part that apply to no one else. As it turned out, however, this language was misleading, at best. Although Mr. Frederick had taken the courses necessary to qualify as a certified THI-1 senior reactor operator instructor, he had also, in Ma rch of 1984, failed the examination for which those courses were intended to prepare him. Uh r ig , Tr . 31,7 50. Dr. Uhrig later testified not only that this f ailure was colovant to the committee's ovaluation of Mr.

Frederick, but that it was a very important concidoration.

Chrig, Tr. 31,960-961. Thus, the Special Report created a socious mintmpression as to the status of this important individual.

106. The Staf f testified that thn Committeet's f ailuret to obtain accurato information about Mr. Frederick was a limitation of its methodology. Personcky, Tr. 33,177. This is a considorable undoratatomont. This nattor concerna un for savocal con.:cna. Flect, thu Conmt::oe, failed to waaro t h+t

(

f' . .

accuracy of the Special Report on this point. The Committoo did not speak to Mr. Frederick about his status. Tr. 32,087. It i

also must not have made any other independent ef fort to assure the accuracy of its statement.

107. Second, the Committee included this misleading information in the special Report although it was apparently told by Dr. Coe at the beginning of its work about the status of Mr.

Uh r ig , Tr . 31,750, Coe , Tr . 3 2,3 54. Thus, through Frederick.

oversight or otherwise, the Committee failed to assure that the report was accurate and f ailed to reflect in the report what the chairman himself considered to be significant negative information concerning the TMI-1 training program.

108. Third, although many Licensen personnel reviewed draf ts of the Special Report in which this language would have appeared, they failed to correct the misiepression that the language creates. Uhrig, Tr. 31,829-834, Long , Tr . 3 2,216-218, 2 5 3, Coe ,

Tr. 32,366-370,21 Leonard, Tr . 3 2,4 69, Newton, Tr . 3 2,46 2.

This is significant with respect to the Committee's work as a whole, but particularly with respect to the Committee's work on the :pecial Report. In reaching thn conclusions stated in the Special report, the Committon relied almont entirely upon

• 1

• We are quito disturbed by Dr. Coe's tantimony on this point, llo asserto that the statomunt in the Special Esport is not inaccuratn becauno Mr. TruderickIndood, dDr.id Coo takeconsidored thu coursonthe mentioned in tho atatement. Th is autonianing statement to be candid. Tr . 3 2,3 6 6- 3 71.

testimony leads un to quantion Cr. Coo's judgment quito nortounty.  !! ho cannot undorstand wny the statomont in decoptivo, wo find inac we cannot givo his tost'.ony any ci jnt ticant weight tocauan it may te bagud on aa n poor juugment. '.;o nad thow;r.t that C r . 'Jo n .iJnt to: 9 an LM ortant part as a new anc objoettvo participant in tno . raining program.

Wo woro wrong on that 1.otnt.

i

I

• I assess the ef fect on its conclusions, both in the Special Report 1

i that Licensee submitted to the Commission, and at the first l

l opportunity in the testimony that accompanied the Special Re po r t . We consider Licensee's failure to make this correction l

to be estremely serious and to reflect very badly on Licensee's judgment and its integrity. We believe the committee was negligent in this matter, as it was in reaching the conclusions of the Special . Report as a whole, but _we do not believe that this matter raises'any serious questions about the integrity of the members of the Committee.

111. As this discussion suggests, this matter heightens the need to assure that the evaluation of the TMI-1 training program was thorough, objective, and independent. It also emphasizes the l

need to assure that the training program itself employs objective criteria and does not depend too much upon subjective judgments.

i l This is particularly true where Dr. Uhrig, the Chairman of the Committee, has publicly committed himself, through the AIF Committees, to promote restart of TMI-1. Stipulation concerning Dr. Uhrig. Ginco his mind has already made up on the ultimate innue, we have difficulty baning favorable conclusions on his or the Committon's subjectivo judgoments.

112. Thu memborn of the Reconstituted CARP Committee certainly havo the background and qualifications to ovaluato a training program and respond to ALAD-770. They know as well on l

anyone the intricacion, compieXitics, and inherent subjectivity Wo also enat charact9ri:o a training program such an this one.

havet no reason e.o doubt enoir porgonal integrity.

i i  !

113. That is why we are so disappointed in the quality of their work on the Special Report. There is simply no serious f

case to be made for the proposition that the work that led to the the conclusions stated by Special Report was adequate to support the Committee. Yet the Committee members allowed themselves to bo pushed into a producing unqualified conclusions on the basis of which the Commission might well decide to authorize the In fact, they knew that if restart of THI-1. They knew better.

someone was going to rely on their report, there was going to be a need for verification of what they had said. Uhrig, Tr.

l 31,973, 32,022-023, 32,104. They should simply have insisted for that they could not complete a reasonable evaluation in time the Commission's meeting. The fact that they did not taints not In this, of

)

only the Special Report, but their entire testimony.

l all cases, it must be clear that the Commission's and this Board's decisions are based upon objective, factual information The and upon evaluations that meet at least minimal standards.

Special Report.did not meet these standards.

3 l

i

e o July 1, 1985 - UCS DOCKETED C

UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE Ti!E ATOMIC SAFETY AND LICENSING APPEAL BOA 0FFICL UF SECRt,1 A,.

Epygl In the Matter of ) 00CKlig4

)

METROPOLITAN EDISON COMPANY ) Docket No. 50289 SP

) (Restart Romand on (Three Mile Island Nuclear } Management - Training)

Station, Unit No.1 )

)

CERTIFICATE OF SERVICE I hereby certify that copies of the foregoing Union of Concerned Scientists Brief on Appeal of the Partial Initial Decision on Licensed Operator Training were served by deposit in the U.S. mail, this lot day of July,1985, except those indicated by an asterisk.

.c . / &;f.. ,.c William,8. Jordan, III e

f' s +

e e UNITED STATES OF AMERICA NUCLEAR REGULATORY COMMISSION BEFORE THE ATOMIC SAFETY AND LICENSING BOARD l

In the Matter of '

)

)

! METROPOLITAN EDISON COMPANY ) Docket No. 50-289

) (Restart Remand on

) Management)

(Three Mile Island Nuclear )

Station, Unit No. 1)

)

SERVICE LIST Administrative Judge G0ry J. Edles, Chateman

• Kary Wagner, Esq.

i Atomic Safety & Licensing Appeal Bd. Office of the Executive Legal Dir.

U.S, Nuclear Regulatory Commission U.S. Nuclear Regulatory Commiss'on 20555 Washington, D.C. 20555 Wcenington, D.C.

l Administrative .7udge f Jcnn H. Buch

• Deborah Eauser. Escuire

' Acc=tc Safety & Licensing Appeal Ed. Shaw, Pittman, Pott: & Trownridge U.S. Nuclear Regulatory Commtcston 1800 M Street, N.W.

20036 Wasnington, D.C.

Wconington, D.C. 20555 .

Ad=tnistrative Judge l

! Chrts tne N. Koni . Ms. Louise Bradford i Atomte Safety 4 Licencing Appeal Od. TMI Alert

, C.S. f:uclear Pegulatney Commission 1011 Green Street C.C. 20555 Harricourg, PA 17102 Wesr.tngton, l

' Ad=tnicerative Judoe Joanne Dorochaw, Esquire Ivan W. Smitn, Chateman The Cnetstic Institute

! Atomic Saroty & Licen:tng Board 1324 *.oren Capitol Street

U.S. Nuclear Requiatory Commte

Lon Ucontngton, D.C. 20555 Manninoton, D.C. 20002 l Admintctrative Judoe Ptr. and Mrs. Norman Aamodt l Snelcon J. Wolfe 200 North Churen Street i Atomic 3atety 4 Licenctny ACpeal Ud. Parkesburg, PA 19365

, U.S. *:ucioar Regulatory Commtanton Wauntngton, O.C. 20555 Ad=tntatrative Judae Lynne Bernabet, Eco.

Government Accountantltty Project

.1accave A. Linenneraer, Jr. 1555 Connecticut Ave.

Atunte 3acecy 4 Liconutn1 naard Wacntngton, D.C. 20009 U.J. *luelear Peoulatory Commtanton

' Waanington, D.C. ;0555

! Doc <oting and Service Section ti t en ae l F. McBride, Ecq.

Ofttee ot the Secretary Lesoeuf, Lamn, Letcy & MacHae e1100 13 3 3 f:ew Hamunnire Ave, N.W.

U.S. *:uclear Requiacory Comm1as1on Wacntngton, D.C. 20036 Wacntngton, D.C. 20555 8

.. "Q_

~

Micncol W. Mcupin, Ecq.

Hunten & Williens 0 ,* 707 Eact McIn Strcot -

P.O. Box 1535 Richmond, VA 23212 l Thomas Y. Au, Esq.

. Office of Chief Counsel Department of Environmental Resources l 505 Executive Houses -

I: P.O. Box 2357 ...

Harrisburg, PA 17120 ,

l

• HAND DELIVERED , .,

l.

I l . . .

l' l

8 1 .

5 1

I s

l=

4 0

l l

e -+~ ____ . _ _ _ _ _ _ _

o * * ~