ML20217K081

Methodology for Identifying Financial Variables for Trend Analysis

ML20217K081
Person / Time
Issue date:	10/31/1999
From:	Lloyd R, Raughley W NRC OFFICE OF NUCLEAR REGULATORY RESEARCH (RES)
To:
Shared Package
ML20217K077	List: ML20217K074 ML20217K081
References
NUDOCS 9910260042
Download: ML20217K081 (39)

Text

{{#Wiki_filter:.- METHODOLOGY FOR IDENTIFYING FINANCIAL VARIABLES FOR TREND ANALYSIS October 1999 ) Prepared by: William S. Raughley Ronald L. Lloyd j

• This report provides the results of analyses performed prior to August 1998 Regulatory Effectiveness Assessment and Human Factors Branch i

Division of Systems Analysis and Regulatory Effectiveness Office of Nuclear Regulatory Research 9910260042 991020 PDR ORG NREB PDR 1

h. \\ Abstract 1-This report summarizes the statistical and practical methods used to select and evaluate financial variables for possible use in the Nuclear Regulatory Commission's Senior Management Meeting processc The report is based on analysis of financial data from 1990 to , 1995._ A draft of this report was made availabte for public comment in August 1998. No changes were made to the draft as a result of comments received. This final report documents analyses performed prior to August 1998. Based on the analysis, several financial variables, 4 - when considered together, cre determined to exhibit a good statistical correlation with plants discussed at past Senior Management Meetings._ Given the analysis to date, the financial l variables are revenue factor, nonfuel operation and maintenance costs, coverage, and total i production cost per megawatt hour. Co', paring the four variable trends to earlier single-unit - and multiunit trends in the nuclear industry identifies changes that often preceded decisions to , discuss a plant at a Senior Management Meeting. The analysis confirms that the financial variables should not be used alone or in a financial ranking system. Subsequent to the work on financial variables, the NRC with extemal stakeholder input developed a revised reactor oversight process to focus on evaluating performance in key areas related to safety through use of risk-informed performance indicators and inspection, and taking action based on pre-established thresholds. The financial variables are not a part of the revised reactor oversight process. f i j lil l I

p:: u. t.. Acknowledgments "o The authors wouldlike to express th'eir appreciation to Howard Stromberg and Jeff Einerson of

the Idaho National Engineering and Environmental Laboratory, and Michael Dusaniwskyj of the -

U.S. Nuclear Regulatory Commission. The efforts and guidance provided by these contributors . during the analysis documented in this report made the analysis prompt and complete, l l 1 l i i V I il' [.

mr i 4 Contents U Ab stra ct..................................................................iii

Ac k n owled g m e nts.... :..................................................... v i

' Ab breviatio n s.............................................................. ix Executive S u m mary......................................................... xi 1 Introd u c ti o n............................................................. ~ 1 2 : Analysis a nd Results.................................................... 2 2.1. Criteria for Selecting Financial Variables.................................. 4 2.2 D a ta S o u r ce s..................................................... 4 2.3 ' Selection of Candidate Variables....................................... 5 - 2.3.1. Selection of Corporate Candidate Variables......................... 5 2.3.2 Selection of Plant Candidate Variables............................. 8 2.4 Identification of Past Senior Management Meeting Discussion Piants........... 8 2.5 Identifying the Financial Variables and Trending Methodology From Statistical Analysis............................................. 8 2.6 Financial Variable Trends........................................... 10 2.7 Analysis of Financial Variables '........................................ 10 2.7.1 Analysis Using Plants Discussed at Past Senior Management Meetings.. 10 2.7.2 Statistical Findings......................................... 14 3 Plans To investigate Additional Candidate Variables,,....................... 16 3.1 ' Assessments in the Financial Community................................ 16 3.2 Moody's Bon d Rating............................................... 16 3.3 Plant Forced Outage and Bulk Power System Capacity Margin............... 16 4 C o n c l u s i o n s...................... -.................................... 1 7 5 Refe re n ce s.......................................................... 18 Tables Table 1 Candidate financial variables -.......................................... 6 . Table 2 Lead analysis results................................................ 14 vii

y __O Contents (cont.) ') Figures i Paae -r

Figure 1 : Steps to identify an evaluated set of financial variables.......................3

- Figure 2 Typical revenue distribution........................................... 7 Figure 3. Financial variable trend chart for a sample site............................ 11 -_ Figure 4 - NERC Regional Councils.............. -.............................. 12 2 Appendix ~ A' ' Definitions of Candidate Variables ........................................ A-1 y 'B Detailed Statistical Methods and Results..................................... B-1 . Tables Table B-1 Summary of results '................................................ B-3 . Table B-2 Single-unit plants discussed versus not discussed....................... B-4 Table B-3 Multiunit plants discussed versus not discussed................,....... B-5 e viil e

c Abbreviations AEOD ' Analysis and Evaluation of Operational Data (Office for) FERC. Federal Energy Regulatory Commission INEEL. Idaho National Engineering and Environmental Laboratory kWe kilowatt electric MWe - megawatt electric MWH megawatt hour NERC North American Electric Reliability Council NRC . U.S. Nuclear Regulatory Commission O&M operation and maintenance costs SAS~ a statistical software system SMM. Senior Management Meeting -S&P Standard and Poors [ stock reports) UDI Utility Data Institute I i i { IX

x u e-Executive Summary '~ < As a result of recommendations made by the firm Arthur Andersen for monitoring financial e ' stress, the NRC's Office for Analysis and Evaluation of Operational Data' identified and H - evaluated a set of financial variables that could point to financial stress with the potential for i compromising plant safety. The purpose of this report is to summarize the work pertaining to e sthe identification of the financial variables that correlated with plants that had been discussed at 1 Senior Management Meetings (SMMs) from 1990 to 1995. A draft of this report was made available for public comment in August 1998. No changes were made to the draft as a result of

comments received.

This final report documents analyses performed prior to August 1998.

L The analysis confirms that the financial variables should not be used alone or in a financial ranking system.- H 1 ' Subsequent to the work on financial variables, the NRC with the input of extemal stakeholders ' developed a revised oversight process to focus on evaluating performance in key areas related to safety through use of risk-informed performance variables and inspection, and taking action / based on pre-established thresholds.' Financial variables may be useful in informing NRC management regarding financial stress. However the financial variables are not a part of the ^ revised reactor oversight process, ~'A technical review identified four financial variables: (1) the revenue factor, (2) nonfuel operation and maintenance cost, (3) coverage, and (4) total production costs / megawatt hour. The revenue factor is the ratio of the actual site revenue to the maximum possible revenue. . Nonfuel operation and maintenance cost is the annual cost for material, labor, and supervision l for level of effort activities and projects such as testing, corrective action, and preventive i 1 maintenance.: Coverage is the site revenue less the total site production costs dMded by the total production' costs; it is a comparative measure of how many times the site covers its . production costs. The total cost / megawatt hour is the total production costs per megawatt hour generated! Comparing these site variable trends to earlier single-unit or multiunit median - i trends in the nuclear industry pointed to financial trends and pattems that had often preceded a . decision to discuss a plant at a SMM. ' The 1996 trends in financial variables were added to the ! -information base for the January 1998 SMM process. ] The resultant set of financial variables were identified from a larger set of publicly available corporate and site financial variables. As there were many variables to consider, selection criteria based on practical and empirical considerations were u' sed to systematically reduce the number of variables. The candidate variables were selected on the basis'of their faca validity, that is, their relevance to the purpose of a company, their relevance to deregulation initiatives,

and peer and consultant recommendations. The candidate variables were statistically -

evaluated, both individually and collectively, to identify the financial variables based on i correlations with plants that had been discussed at earlier SMM meetings. The financial variables were trended with single unit and multiunit trends in the nuclear industry. The ~ ' ' Effective' March 28,1999, the Office for Analysis and Evaluation of Operational Data was < disbanded. The work described in this report which was initiated by AEOD is being completed by the Regulatory Effectiveness Assessment and Human Factors Branch of the NRC's Office of Nuclear Regulatory Research. xi.

,1 : k ' financial variable trends were bench marked again'st earlier decisions to discuss a plant at a SMM to identify past trends, pattems, and lead times. Based upon the analysis completed to date, several observations are made as a result of - statistical and practical evaluations of financial variables. The report concluded that: 1 1 - Financial variables of most interest are revenue factor, nonfuel operation and - maintenance costs, coverage, and total production cost per megawatt hour. These ' variables exhibit a goo stat s ca corre a on with the plants discussed at past SMMs. d i ti l l ti Comparing the trends of the four variables to single-unit and multiunit industry trends Identifies adverse trends that often preceded decisions to discuss a plant at a SMM. The analysis' also concluded that the revenue factor is the most predictive variable, and that a site is likely to be discussed at a SMM if its revenue factor is below 65 to 70 percent for 2 consecutive years. Mathematical analysis concluded that financial variable trend analysis alone should not

2..

be used to determine whether a plant should be discussed or not discussed in the SMM - process. However, analysis of the trends of financial variables does provide'useful information to _ e used in conjunction with plant performance trends. The financial trend b analysis also helps to distinguish sites at which financial stress is not affecting plant performance trends from sites at which financial stress may be a precursor to future adverse trends in plant performance. - 3. The four financial variables are not an exclusive set. They were selected from a larger ' set of variables based on statistical correlations with plants discussed at past SMMs.and - face validity by the financial community. Other financial variable correlations or assumptions may result in additional sets of variables that produce similar overall . observations. 4J ' Corporate financial variables did not exhibit good statistical relationships to plants . discussed at past SMMs.' Several reasons exist for this lack of correlation; principally too many types of organizational structures, multiple corporations having ownersh!p of a plant or site, the types of investments owned by the corporations having an investment

in the plant or site, and the percent nuclear investment by plant or site owners.

5. Financial data are _not always available on a timely basis to support the SMM process. The publicly available financial data for the revenue factor, nonfuel operation and maintenance costs, coverage, and total production cost per megawatt hour are -generally not available until the third quarter following the close of a calendar year, +

making them dated for use in the SMM process held during the mid-year.

_ The staff has assessed the use of other financial information. The following was provided for ~ I l i the July 1998 SMM process: (1) the observations of the financial community when it highlights . specific site problems or assesses deregulation initiatives, and (2) Moody's bond ratings ~ (particularly downgrades or speculative grade ratings). n xil l Ak 4

y 1 Introduction This report summarizes the statistical and practical methods used to select and evaluate financial variables for use in the Nuclear Regulatory Commission's Senior Management Meeting (SMM) process as described in SECY-97-072, " Staff Action Plan to Improve the Senior Management Meeting Process," April 2,1997 (Ref.1). The SMM process involves three significant events occurring on a semiannual basis: screening meetings; the SMM; and a Commission briefing. In preparing for SMMs, the NRC staff analyzes licensee performance information from inspection reports, performance indicators and analyses, enforcement history, and other pertinent data. Within, approximately, the 2-month period before each SMM, screening meetings are held with each NRC region to discuss individual plant performance. From these screening meetings, the NRC will determine which plants should be discussed at a SMM. It is primarily during the screening meeting forum that discussions would take place regarding the potential that an adverse plant performance trend was caused by financial stress, or that a future adverse plant performance trend may develop because of financial stress. The Office for Analysis and Evaluation of Operational Data' evaluated several site and f corporate ilnancial variables that had the potential to indicate financial stress that could i compromise plant safety. Analysis of these financial variable trends could be used to indicate when the NRC should be particularly attentive to the possibility that a licensee is compromising plant safety to reduce costs. The words financialand economic are sometimes used interchangeably. However, as a point of clarification, this report identifies financial variables, not economic variables. Financial matters are generally controlled by the site or corporation. Economics are concerned chiefly with the description and analysis of production, distribution, and consumption of goods and services. Economic matters we not controlled by the site or corpo.h. Providing an analysis of financial variables is part of an NRC initiative to improve the information base used to assess plant performance in Staff Requirements Memorandum M960625, " Briefing on Operating Reactors and Fuel Facilities," June 28,1996 (Ref. 2), the Commission requested that the staff, with the assistance of contractors, evaluate the development of improved performance indicators. In response to this request, AEOD obtained the assistance of the consulting firm, Arthur Andersen. Arthur Andersen published a report, " Recommendations To improve the Senior Management Meeting Process," December 30,1996 (Ref. 3), with a recommendation to make use of economic (financial] Indicators. In response to the Arthur Andersen report, Staff Requirements Memorandum M970218B," Briefing on Ana!ysis of Quantifying Plant Watch List Indicators (Arthur Andersen Study)," March 14,1997 (Ref. 4), provided guidance for implementing the consultant's recommendations. The staff's plan to exarnine indicators of financial stress and to monitor plants to determine what effect, if any, such stress has on an individual facility was subsequently described in SECY-97-072. In addition, the staff was directed to present an evaluated set of financial variables at the January 1998 SMM. AEOD was given responsibility to provide an evaluation of financial l variable trends. i Effective March 28,1999, the Office for Analysis and Evaluation of Operational Data was disbanded. The work described in this report which was initiated by AEOD is being completed by the Regulatory Effectiveness Assessment and Human Factors Branch of the NRC's Office of Nuclear Regulatory Research. 1 !L.

l D t 1 Arthur Andersen supported its recommendation to use financial variables with observations - about the relationship between safety and economics: o ( Given the economic forces behind production and safety,' / assessing indicators of economic [ financial] stress and management's response to them ahead of time should allow the NRC to achieve these improvements: earlier identification of problems, fewer safety risks to the public, earlie'r and less costly.. resolution of problems. ~ Experience shows that financial stress compromises safety when extemal or intemal conditions Lemphas!ze operations to enhance short-term financial performance. For example, in 1996, J State audits at Millstone and an NRC assessment at Maine Yankee confirmed that financial difficulties compromised safety. Overemphasis on cost issues was identified as a root cause of weaknesses at both plants.1The Millstone audit also found that cost issues overwhelmed the existing safety culture. The Maine Yankee assessment found that the emphasis to become a low-cost energy producer limited the resources available to address corrective actions and. some plant improvement upgrades. Similarly, NRC diagnostic evaluations completed at . FitzPatrickl South Texas, Quad Cities, and Palisades documented observations about limited resources affecting safety performance. Deregulation has _the potential to cause. financial stress that compromises safety and heightens - the need to broaden the NRC information base to includs an analysis of financial variable . trendo; The 1992 National Energy Policy Act and various state' initiatives have fostered the - m vement from regulated to market-based rates. Before deregulation, utility rate commissions o and the Federal Energy Regulatory Commission (FERC) ensured that the selling price of electricity was sufficient to provide the utility with enough revenue to cover its costs.' With

deregulation, market based rates may not provide.for recovery of costs. A potential concem is -

that utilities may offset revenue reductions caused by competition by implementing cost

reductions as investors pressure management to maintain the same level of fiscal performance.

> Cost reductions that compromise safety activity would warrant monitoring. 2-LAnalysis and Results U n overview of the steps in the analysis to ident!'y an evaluated set of financial variables is A illustrated in Figure 1, TSteps to identify an evaluated set of financial variables," and is J discussed in this section of the report.-The analysis first considered numerous publicly ~ available site and corporate financial variables and ended with a sct of four financial variables. " A process of elimination based on selection criteria was used to systamatically reduce the inumber of variables ~under consideration. The first reduction produced a set of candidate Jvariables that were statistically evaluated, both individually and collectively, to identify the

variables that correlated with plants that had been discussed at earlier SMMs. SAS (Ref. 5), a i statistical software system, was used for the statistical analyses that were done by the Idaho

' National Engineering and Environmental Laboratory (INEEL).- The final set of financial variables'was selected following more quantitative and qualitative analyses to include detailed

evaluations using earlier decisions to discuss plants at SMMs.

. 2 ./ m t" e '- i L ._U L

P. L-w - 1 i ~ l l Develop Selection Criteria i l Review Public Data Sources. i - Utility Data institute (UDI) - Corporate Annual Reports - Standard and Poors (S&P) - Moodys FERC - Value Line identify Candidate Variables l Data Collection & Database Development j Statistical Analysis of Statistical Analysis of Variables Further Assessment of Variables Collectively individually Face Validity l 1 . Develop Candidate Variable Sets Additional Quantitative and Qualitative Evaluation l l i Evaluated Set of Financial Variables o Figure 1 Steps to identify an evaluated set of financial variables 3 l c.

4 ]-Y q ~ : As there were many variables to consider,- the candidate variables were systematically selected r < on the basis of their face' validity, that is, their relevance to the purpose of a company, peer and consultant recommendations.- U L AEOD initially.used sets of the financial variables in'a'model derived from the statistical' i analysis. This approach was abandoned because the staff was not trying to rank the licensees fiscally. Comparing the individual financial variable trends to industry single-unit and multiunit trends produced a meaningful assessment. " 2.1 Criteria for Selecting Financial Variables Financial variable selection criteria were developed through meaningful practical and empirical - considerations.4 The following criteria were used to select the variables: (1) their data were . publicly available, (2) they had face validity to assure the results were relevant and had practical meaning so as to compel their acceptance, (3) they exhibited a meaningful statistical . relationship to plants discussed at past SMMs, (4) they led previous SMM decisions by at least 1 year, and (3) they were comparable between different types of electric utilities (public and-i investor-owned), as well as different corporate financial structures, it was concluded that the selection criteria did not provide an exclusive set of variables. The

resultant financial variables were selected from a larger set based on statistical correlations with the plants discussed at past SMMs, face. validity,' and experience. Other financial correlations or assumptions may result in additional sets of variables that produce similar overall observations.

2.21 Data Sources Public data s'ources were reviewed to identify candidate variables and obtain data. The site data were purchased from the Utility Data Institute (UDI) (Ref. 6), a supplier that routinely - obtains and consolidates public information that utilities report to the FERC, the NRC, and other ~ ' Federal agencies.. Samples of the UDI data used were verified against the source documents. The corporate data were obtained from company annual reports and correspondence submitted- - to the NRC, S&P stock reports, Moody's Corporate Bond Ratings, Moody's Municipal and Govemment Manual Public Utility Manual and the FERC Forms 1 and 826. Moody's and S&P obtain and consolidate public information from corporate annual reports, FERC, and the Securities and Exchange Commission, and provide commonly used ratios that reflect financial . performance and position. Information on capacity margins was obtained from Value-Line - company reports and corporate annual reports. Data were collected for the candidate variables, organized by site, and separated according to = whether a site _was either a single-unit or multiunit site.. Corporate data for the major equity

owner were obtained and were associated with each nuclear site owned by the corporation.

in collecting data, it was found that, in addition to the publicly and investor-owned utilities, there were many types of investor-owned utilities which tended to diminish the relevance of some ~ l variables. For example, some companies' fiscal statements reflect nonelectric businesses or ' the collective results of several subsidiaries or operating companies. Even more noticeable 1 were the company structures that changed year to year, changing the meaning of the data and the ability to produce meaningful trends. u 4 q b: a i 1'

Publicly available data may not be timely. The financial data from a given calendar year is generally not available until the end of the third quarter of the following year. 2.3 Selection of Candidate Variables The candidate variables are listed in Table 1 and defined in Appendix A in this report. As there were many variables to consider, the candidate variables were systematically selected on the basis of their face validity. Sections 2.3.1 and 2.3.2 summarize the selection of corporate and plant candidate variables, i 2.3.1 Selection of Corporate Candidate Variables \\ A primary purpose of an investor-owned utility is to continually maximize the earnings, or net income, for the owners. This resulted in the selection of " net income,"" net income change," I and " fixed charge coverage" as candidate variables. A corporation maximizes net income by j maximizing revenue, minimizing operating costs, and reinvesting capital to further maximize ) revenues or minimize costs. As stated earlier, deregulation could result in revenue reductions j that may be offset by' reducing operating costs. These considerations resulted in the selection of corporate " revenue,"" change in revenue,"" revenue-to-sales ratio"(average selling price of electricity), and " percent return on revenue" as candidate variables. Costs that could be reduced were identified from analyzing a typical corporate income statement to identify variable costs that could be easily changed. Figure 2," Typical revenue distribution," shows the major cost items for a typical corporation. TI 3 operation and maintenance (O&M) costs are the only costs that can be changed easily by utility management. ' Other costs, such as taxes and depreciation, are either fixed or very difficult to change. A corporate " operating ratio," the ratio of the O&M costs to revenue, was also selected as a i candidate variable. Several corporate financial ratios were analyzed since they are widely used by the financial community as a means of normalizing the dollar variables to facilitate comparisons such as the " operating ratio" from the S&P stock reports. Other S&P stock report corporate ratios were used including " percent eamed on net property,"" percent return on common equity," and " percent return on invested capital." " Percent retum on common equity,"is a value regulated by j State utility rate commissions or FERC. " Capacity margin" was also selected as a candidate variable, because having more capacity than can be sold may burden corporate finances. q Excess " capacity margin" diminishes the financialincentive to fix problems. Arthur Andersen i recommended " debt-to-equity ratio" and " percent nuclear generating capacity of total utility j capacity"(" percent nuclear") as corporate candidate variables. The same factors were used for the public utilities, recognizing there would be gaps in the data as the public utility financial goals are different from the investor-owned utility financial goals. A public utility has no { investors,is a break-even operation, and attempts to minimize costs to limit revenues needed l from its customers. l l l 5 n

Table 1 Candidate financial variables PLANT CANDIDATE VARIABLES CORPORATE CANDIDATE VARIABLES 4 Capital additions Capacity margin Capital Additions year moving average Debt-to-equity ratio Contribution Fixed charge coverage Coverage Moody's bond rating Design electrical rating Net income Loss Netincome change Nonfuel O&M costs Operating ratio Nonfuel O&M cost change Percent eamed on net property Nonfuel O&M costs year moving average Percent nuclear - 3 Nonfuel O&M costs per megawatt electrical Percent return on common equity. (MWe) rated Production cost per megawatt hour (MWH)- Percent return on_ invested capital 2-year moving average Production cost per gross MWe rated Revenue Production cost per gross MWH generated Revenue change Produ'etion cost per gross MWe rated - Revenue-to-sales ratio 3-year average Revenue factor Utility type Site operating ratio Site revenue i I g 6

1l l )s t dse egs nns iia an 2 t rh ea s REa C ( s dn e 4 d 0 iv 1 iD e m ocn x i a 2 laT 7 r N e d O e F ITUB no I t t R sb ~ ee 6 r T eD 6 t S n I ID n E o U ita N ic 4 e 2 r 1 E pe V D ER dn L a e sc A en xa ar 4 C Tu 5 s I lan P cI o Y L T dna e gcs nne ias tann retee 6 pn p 5 Oix aE ,M leu F W3zW>WE ?a PJ45 n E 3 s E. h g. E fi3 s

x; .T.- p 2.3.2 i Selection of Plant Ca'ndidate Variables - Plant candidate variables were selected in a similar manner as were the corporate variables. A t " revenue factor was calculated. "Nonfuel O&M cost" was selected because it is the only major

plant cost. hat utilities can easily change. A site ~ operating ratio was also calculated to reflect t

L the percentage of revenues that is absorbed by O&M costs.1 The fixed plant costs are not - publicly available. " Capital additions" was also selected, since it relates to maximizing corporate income; The plant "nonfuel O&M and capital costs" are also the costs related to safety activities. Arthur Andersen also recommended including operating cost per kilowatt hour, ~ V . operating cost trend for 3 years,' and capital spending trend over 3 years as plant candidate ' variables.- j . (. . Deregulation raises concems about a licensee's capability to cover plant costs such as O&M, ~ --depreciation, interest and principal on the debt, and taxes. A nuclear plant would likely continue u q X ?' to operate as long as the revenues from electric sales exceed these costs. However, since the- . site O&M expenses are the only publicly available data, it is not possible to assess if the revenues exceed the total costs.; Altematively, " contribution" and " coverage" were calculated from publicly available data to measure the plant's ability to cover costs; . 2.4 Identification of Past Senior Management Meeting Discussion Plants The candidate variables were correlated with plants that were discussed previously at SMMs. Discussion plants are those discussed, discussed and receiving a trend letter, discussed and ' classified as a Category 1 plant (plants taken off the Watch List), discussed and classified as a ' Category 2 plant (Watch List), and discussed and classified as a Category 3 (shut down and needs NRC permission to restart).1 A data file was developed that contained the month and year that each plant / site was discussed. t 2.5 ' identifying the Financial Variables and Trending Methodology From Statistical Analysis !!n this study, logistic regression, which is described in more detail in Appendix B, was used to identify a set of financial variables from the candidate list in Table 1. The logistic regression evaluated the relative importance of the candidate variables babad on their correlation with earlier SMM " discussed" and "not discussed" plants. The analysis used 1990-1995 data. The

1996 data was to be used in the SMM process.

The analysis identified several sets of variables that exhibit a statistically significant relationship to plants discussed at earlier SMMs. The earlier logistic regression analysis results identified the number of units at a site as a meaningful variable. This result led to the single-site and multiunit site groupings in Table B-1 (Appendix B). In addition, the grouping in the statistical analysis led to the conclusion that' site variables should be trended and compared to earlier i single-unit or multiunit trends in the nuclear industry..The sets of variables that best correlated . with the plants discussed at SMMs are shown in Table B-1.. As there was not necessarily a . single best set, the statistical selection guidelines in Appendix B were used to evaluate the results. Consistent with the selection criterion to have one set of variables for all types of - utilities and sites,' the decision was made to use the variables discussed below. Statistical analysis of candidate corporate and site financial variables identified four site M ' variables /?These financial variables are revenue factor, nonfuel O&M, coverage, and total .8 ,l.

r production cost per MWH. These variables are defined below; are derived from publicly available data; have face validity as discussed below; and exhibit good statistical correlation with plants discussed at past SMMs as shown in Tables B-1, B 2, and B-3. The financial variables, the definitions, and assessment of face validity are: Revenue Factor is a site ratio of the revenue obtained from the annual sale of electricity to the annual maximum revenue it could have theoretically obtained from the sale of electricity. Since the dollars cancelin the ratio, it results in a measure of MWHs the site has produced to the maximum the site could have produced - a site capacity factor. The value of a generating plant to its corporate orc'anization is measured by its prospective ability to produce revenues. The generating planto produce the revenues for an electric utility. The generating plants in aggregate must sell enough electricity to meet a minimum revenue requirement to cover plant operating costs and corporate costs. Meeting corporate finar,o,m objectives to maximize net income by maximizing revenues and minimizing costs, drives companies to consider revenue in their decision process. Deregulation may result in lower revenues through reduced rates or a loss of customers. A lower or declining revenue factor may indicate financial stress. Nonfuel Operation and Maintenance Cost is a variable cost for labor, supervision, material, electrical auxiliary power, overhead, and other costs for activities for the operation and maintenance of the site. It does not include site costs for fuel, depreciation, interest on debt, repayment of loan principals, property taxes, income taxes, dividends, retained earnings, or capital costs (costs for additions). In addition, it does not include the corporate costs of doing business or construction. Examples of activities covered by nonfuel O&M are level of effort and projects for operations, training, testing, corrective and preventive maintenance, repairs, engineering, most modifications, and licensing. In practice, many activities are a combination of nonfuel O&M and capital. Nonfuel O&M cost can be easily changed by utility management. Consequently, this is a cost generally targeted for reduction. Other site and corporate costs are generally fixed or more difficult to change. The corporation generally allocates nonfuel O&M dollars to each site based on site-specific staffing and work plans. Since staffing is approximately 60 percent of this cost, lower costs are often obtained by reducing staffing. Meeting financial objectives drives j companies to minimize this cost in the decision process. However the decision procese also recognizes that this cost is important to the operation and maintenance of the equipment that ] produces the revenue and other equipment that provides safety capability. This may be a cost targeted for further reduction in deregulation. 1 The nonfuel O&M cost varies widely across the industry because of variations in labor rates, labor and management efficiency, and economics of scale associated with tk number of units at a site. As a result of these differences, each site has its own level for nonNI O&M costs. Deviations from normal site trends, either high or low, can indicate financial stress. i Coverage is site revenue less total production costs divided by total production costs. This is a comparative measure of how many times the site covers its production costs. It also indicates the level of funds available to cover other costs. Site revenues in aggregate must cover all other site costs (depreciation, interest, taxes, etc.) and its po: tion of the corporate costs (taxes, 9 l

Its share of the costs of non-revenue-producing departments, etc.) for the plant to stay in business. Figure 2 shows the major costs that must be covered as a percent of the revenue for - a typical utility. Total Production Costs per Megawatt Hour is a ratio of the total site production costs to the total site gross generatio'n (e.g.,(production costs)/[MWH generated)). The total production costs include fuel and nonfuel costs, but not construction costs. This ratio represents economic efficiency. It was not identified by the logistic regression. However, individual statistical analysis found that it was significant. 2.6 Financial Variable Trends Trend charts, such as those shown in Figure 3, were developed for comparing the site variable trends to past single-unit or multiunit trends in the nuclear industry. Figure 3, " Financial variable trend chart," displays the financial variable trends and referenced median for a typical multiunit site. Single-unit site plots are similar. In this case the cost per MWH was compared to the industry cost per MWH median as well as the NERC regional cost per MWH median, it was subsequently decided that future cost per MWH displays would use the NERC regional median. The NERC regional cost per MWH median was judged to be the most meaningful, since the utilities are electrically interconnected in these areas and can readily compete. A map of the NERC regions is shown in Figure 4,"NERC Regional Councils." 2.7 ' Analysis of Financial Variables 2.7.1 Analysis Using Plants Discussed at Past Senior Management Meetings - lons of Financial Stress - An examination of Figure 3 shows that this site is financially stressed, Cost per MWH competitiveness has decreased, and both the revenue factor and coverage variables show an adverse trend and are worse than the industry median. In general, the site's financial future is in doubt and the site may have a reason to reduce costs. ' The site did reduce costs after mid-1993 as shown by the nonfuel O&M trend in this case, declining safety performance . followed as did SMM discussion and a trend letter. ' Common Financial Trends and Patterns of Past Plants Discussed at Senior Manaoement Meetinos Review of plots similar to those shown in Figure 3 and the analysis in Table 2," Lead analysis ^ results," identified common trends that preceded earlier discussion of plants at SMMs as well as differences in plant financial trends once they were discussed. Table 2 notes the plants and the year each was first discussed, and their four financial variables. The trends of the financial 10

i COST PER MWH ($) VERSUS NUCLEAR COST PER MY!H ($) VERSUS NERC AREA ' INDU7TRY - COMPETITION 0 60 60 60 6 l l' 50 - 50 50 - 50 40 - 40 40 - 40 30 30 - 30 30- ~ ~~- 20 20 - 20 20 ' 10-10 10-10 0 0 0 0 1990 1991 1992 '1993 1994 1995 1996 1990 1991 1992 1993 1994 1995 1996-i I . REVENUE FACTOR COVERAGE l .1.0 1 7 7 6-6 l' O.8 - 0.8 5- -5 4-4 l 0.6 - 0.6 l 3-3 O.4 - 0.4 ' 2-2 l 1- -1 i 0.2 - 0.2 0- -0 l ' O.0 0

• 1
• 1 l

1990. 1991 1992 1993 1994. 1995 '1996 1990 1991 1992 1993 1994 1995 1996 l i l NONFUEL 0 & M COSTS ($M) l - 260 280 l.or and 240 - 240 200- -200 industry median for multi-unit 160 - 160

==="a I 120-120 industry median for the North ., 80 80 American Electric Reliability Council (NERC) steam electric l 40 generating unas in the area in which the alte is w *M 1950' 1991 1992 1993 -1994 1995 1996 l Figure 3 Financial variable trend chart for a sample site 11 C

\\ N C AA C j M CRF CCP N C t i R R cn E u C S o c E y t i l ib a i l y t eR P A_ P I c P - ir S T t P O Sc = e C LE l M R Cn I E ac Ni r Ue c /. m OA Chtr Lo ANe C Nht C Oy I b S Gd W 1 e Ed Rivor Cp 1 Rr e Eu g NiF g$*, gR [ gh

r 4 variables were reviewed in order to find and enter in the table the first year an adverse trend was noticed. No entry indicates there was no lead time. The reviews determined the following: . Decreases of the revenue factor below the industry median often leads SMM discussion by 2 years. Table 2 shows that adverse revenue factor trends led the initial discussions of January 1998 SMM discussion plants for 69 percent of the plants discussed. Review of this finding with earlier SMM discussion plants revealed simi'ar findings and the overall conclusion that a plant is likely to be discussed if its revenue factor is below 65-70 percent for two consecutive years. Adverse nonfuel O&M trends led the January 1998 SMM discussion for 75-80 percent of the plants when used in conjunction with the plant performance trends. Analysis of earlier operating data shows that 60 percent of the industry has reduced O&M costs without affecting plant performance trends, and 20-25 percent are holding costs constant without affecting plant performance trends. The review found that the nonfuel O&M trend needs to be evaluated with plant performance trends to avoid a large number of false predictions. Common nonfuel O&M trends of earlier discussion plants are the following: - Historical nonfuel O&M cost trends were below the industry median. I Nonfuel O&M costs decreased during periods of SMM interest. This was not expected in view of the need to improve plant performance. - Significant increases in nonfuel O&M over a 1-year to 2-year time period generally preceded removal from the Watch List while earlier spending was insufficient. Decreases of coverage below the industry median led the January 1998 SMM discussion plants for 38 percent of the plants. A negative or marginal coverage over time indicated a significant trend leading toward plant closure. Cost per MWH above the NERC regional median led SMM discussion for 2 plants out of 13 shown on Table 2. This data does not make a strong case for its use. Review of trend charts similar to Figure 3 for all sites also determined the following: Cyclic financial variable trends were apparent for poorer performing plants. It appears these sites had difficulty achieving consistent results. . Better performing plants showed gradual movement in financial indicator values from year to year. . Using financial trends along with plant performance trends helps to distinguish sites at which j financial stress is not affecting safety performance from sites at which financial stress may be compromising safety performance. ) i 13 l 1 1

Table 2 Lead analysis results First year First year First year revenue coverage fc;l cost /MWH was factor fell below below the Year first above NERC the regional industry Nonfuel O&M cost Plant discussed regional median median median variations A 1997 1996 1995 Lowin view of plant performance data. B 1993 Underspent in past. B 1998 Underspent peers in 1996. C 1996 D all years 1991 1991 1991 Low in view of plant since 1992 performance data. Underspent in past. E 1997 1994 1994 F 1994 1992 Low in view of plant performance data. Underspent in past. G 1991 Cuts in 1987-1988. H 1998 1996 1 1997 1996 Historically underspent. J 1993 1992 1992 Lowin view of pfant performance data. K 1994 1992 1992 L 1995 1994 1992 Low in view of plant performance data. Underspent in past.

• 'No entry" corresponds to no lead time before sMM discussion.

2.7.2 Statistical Findings Data Lao Made No Difference I The candidate variables used in the logistic regression analysis were compared (1) unlagged and (2) lagged 1,2, and 3 years from the date of each SMM. There was no significant difference between the lagged and unlagged results because some plants had been discussed repetitively. j 14

o Corporate Data Should Be Analyzed Case-bv-Case Corporate variables did not exhibit a statistically significant relationship to the plants discussed at past SMMs. The initial logistic regression analyses identified that too much corporate data was missing from publicly owned utilities to obtain valid results. At this point, it was evident that the corporate and site data should be separately analyzed. The corporate data were separated into two groups, one for the public utilities and one for the investor-owned utilities, to investigate each on their own merits. There were not enough public utility data to perform a meaningful statistical analysis. Review of the investor-owned utility data indicated that the data could not be compared among corporations because of the diverse business and organizational structures. Table B-1," Summary of results"(Appendix B), shows that corporate variables did not exhibit a statistically significant relationship to plants discussed at past SMMs. Of all the investor-owned utility corporate data considered, fixed charge coverage and percentage earned on net property showed weak correlations with the plants discussed at past SMMs. This shows that case-by-case analysis must be done to identify problems at the corporate level. Cluster Analysis Shows That Financial Data Should Not Be Used Alone l The logistic regression analysis identified financial variables for use in the SMM process based on correlations with past decisions to discuss a plant at a SMM. INEEL observed that there may be circular logic in this approach. INEEL suggested using a cluster analysis to validate the critical assumption that the analysis of financial variables alone could identify the SMM discussion plants. In this study, the cluster analysis, which is described in more detail in Appendix B, offers an independent, mathematical look at the financial variables to see if they could be used alone to form the "should be discussed" group and the "need not be discussed" group. To quantify how well the financial variables formed these two groups, a cluster analysis misclassification rate was calculated using the actual plants discussed and compared this value to the misclassification rate obtained by using the logistic regression model. The results in Table B-1 show the cluster analysis misclassification rate is greater than the logistic regression model misclassification rate. INEEL advised that from a mathematical perspective, the results generally indicate more information is needed to better form the " discussed" and "not . discussed" groups, end to evaluate the results obtained from the two approaches. This led to the conclusion that financial data should not be used alone or in a financial plant ranking system, but could be used to supplement the information base for the SMM process. Comparison of Discussed and Not Discussed Cateaories Usina Peer Group Medians j This analysis statistically compared the " discussed" and "not discussed" groups for several candidate variables. The statistical methods and terms used are described in Appendix B. Medians for the " discuss'ed" group and the "not discussed" group are presented in Table B-2, " Single-unit oiscussed versus not discussed," and Table B-3,"Multiunit discussed versus not discussed." ) 15 1

j a Tables B-2 and B-3 show that only a few variables are significant for both the single-unit and J imultiunit groupsf The revenue factor and coverage are highly significant (p <.0001). Nonfuel .O&M costs for single-unit sites is significant (p <.0001), as are the nonfuel O&M cost changes for multiunit sites (p <.0089). Cost per MWH is also significant (p <.0107). Although contribution, loss, and site operating ratio are also significant, when correlated in the logistic regression,'they did not change the overall statistical results and added no new information. jBagprtina of Caoital Additions Data Was inconsistent , Review of the data for capital additions found it includes both additions and deletions. The deletions sometimes exceeded additions, so the amount added was often masked by deletior's. As such, the reported data are not usable for statistical analysis or trending. i 3. - Plans' to Investigate Additional Candidate Variables --The NRC will continue to assess the use of other financial information such as observations of - the financial community when it hightshts specific site problems or assesses deregulation Initiatives, Moody's bond ratings (particularly downgrades or speculative grade ratings), the fiscalimpact of forced outages, and excess bulk power system capacity margin.

3.1' Assessments in the Financial Community Assessments of S&P stock reports and Moody's' bond ratings were provided for the SMM process when they contained relevant s'.te information, announcement of cost-cutting measures that may impact sites, or an assessment of deregulation initiatives. Occasionally, S&P stock reports provided observations and evaluations about sites that led, or supported SMM interest.

3.2 = Moody's Bond Rating i Evaluations of Moody's bond ratings could provide useful information regardi.ig financial stress. Moody's primary business is to continually assess and rate a company's capability to pay

interest and ' principal'on its debt. The Moody's bond rating is a u'seful measure of a utility's financial condition because it incorporates expert professional opinion and analyses, it is a composite of most other relevant measures of financial condition, it evaluates the company's

understanding and response to issues and legislative activi,ty facing the industry, it provides a Lforward-looking rating on the basis of past performance, and it is an established and recognized -

rating system that is updated frequently. Downgrading of bond ratings because of stranded cost concems, or speculative grade ratings may identify company sites to which the NRC should be particularly attentive.

3.3 m Plant Forced Outage and Bulk Power System Capacity Margin The bulk power system is operated so as to generate all the power needed by the customers

• and interconnections at the lowest practical costs. A forced outage of a nuclear plant results in an immediate increase in fuel costs for replacement power. Typically, a 1-percent increase in i forced outage rate (of 3.6r days per year) of a nuclear unit results in $750,000 to $1,000,000

' increase in fuel costs and a corresponding decrease in net income. There are additional and 16- + liiieyig. e,i' .s a s g i. i .,,,,,a,, m,, i

p r g l greater cos's for reserve power requirements needed to maintain the capability to accept forced. t

outages without power interruption. ilncreases in plant forced outages, particularly above j

planned values, result in significant financial stressc j 3Thc9ulk power system needs a minimum reserve capacity margin to ensure continuous - {opSration.: Excess power system capacity margin diminishes the financialincentive to fix or

upgrade the material condition of a plantJ If a corporation has more power than it can sell, it is Enot likely to invest in plant improvements beyond those which are essential.

~ L 4 . Conclusions ' Based upon the analysis completed to date, several obcervations are made as a result of f' i statistical and practical evaluations of financial variables. The report concluded that: - 1. - Financial variables of most interest are revenue factor, nonfo qwation and maintenance costs, coverage, and total production cost pe regaatt hour.- These . variables exhibit a good statistical correlation with the plants F.,ussed at past SMMs. - Comparing the trends of the four variables to single-unit and multiunit industry trends identifies adverse trends that often preceded decisions to discuss a plant at a SMM. The analysis also concluded that the revenue factor is the most predictive variable, and that a site is likely to be' discussed at a SMM if its revenue factor is below 65 to 70 percent for 2 consecutive years.- 2. . Mathematical analysis concluded that financial variable trend analysis alone should not - be used to determine whether a plant should be discussed or not discussed in the SMM , process. -However, analysis of the trends of financial variables does provide useful Information to be used in conjunction with plant performance trends. The financial trend i analysis also helps to distinguish sites at which financial stress is not affecting plant . performance trends from sites at which financial stress may be a precursor to future adverse trends in plant performance. -3. The four financial variables are not an exclusive set. They were selected from a larger set of variables based on statistical correlations with plants discussed.at past SMMs and face validity by the financial community.. Other financial variable correlations or. assumptions may result in additional sets of variables that prcduce similar overall . observations. 4. ~ Corporate financial variables did not exhitHt good statistical relationships to plants . discussed at past SMMs. Several. reasons exist for this lack of correlation; principally too many types of organizational structures, multiple corporations having ownership of a 2 - pla' t or site, the types of investments owned by the corporations having an investment n in the plant or site, and the percent nuclear investment by plant or site ownars. 5. Financial data are not always available on a timely basis to support the SMM process. The publicly available financial data for the revenue factor, nonfuel operation and maintenance costs, coverage, and total production cost per megawatt hour are t Q '17 l

. generally not available until the third quarter following the close of a calendar year, making them dated for use in the SMM process held during the mid-year. 5: References 1. U.S. Nuclear Regulatory Commission, SECY-97-072," Staff Action Plan to improve the , Senior Management Meeting Process," April 2,1997. 2. U.Sl Nuclear Regulatory Commission, Staff Requirements Memorandum M960625, ~ " Briefing on Operating Reactors and Fuel Facilities," June 28,1996. 3.~ ' Arthur Andersen,

• Recommendations to improve the Senior Management Meeting Process," December 30,1996.

4. U.S. Nuclear Regul:3 tory Commission, Staff Requirements Memorandum M970218B, " Briefing on Analysis of Quantifying Plant Watch List Indicators (Arthur Andersen Study)," March 14,1997. 5. Institute Inc.,"SAS/ STAT User's Guide, Release 6.03 Edition," Cary, NC: Institute Inc., 1988. 6. Utility Data Institute, A Division of McGraw-Hill Companies," Nuclear Plant Data Model Database," July 1996. i i 18 2

D e e APPENDICES I i l t 4 i i i

y +s 'c m . APPENDIX A' , Definitions of Candidate Variables Capacity Margin is a corporate variable. Capacity margin is a calculated value using the Jequation: [{ capacity at peak)-{ peak load))(100%)/(capacity at peak) Capacity margin represents the amount of excess power (above peak load) expressed as a . percent of full capacity. A high capacity margin is not good because it represents lost revenue, whereas a low margin is also not good because it means that sufficient power is not available . under all conditions.' if the peak load exceeds the capacity at peak, the margin would ba J negative and the licensee would pay to import power to meet its load requirement. Capital Additions is a site variable. The capital is the amount of money invested in the site for the land, structures, and equipment. The site uses capital for additions. Licensees make - additions and retire portions of the structure and equipment throughout the life of the plant. This results in additions and deletions to the total capital base. The capital additions variable is

calculated by subtracting the capital base of one year from the preceding capital basa. This

could be a negative value or zero because retirements may exceed additions.

Capital Additions 3-Year Moving Average is a calculated site variable. The value is

calculated _by averaging the capital addition costs over three consecutive years, (e.g.,3-year average for.1996 = [(1996 capital addition) + (1995 capital addition) + (1994 capital addition)]/3.

Contribution'is site revenue less the site total production costs. It is calculated by subtracting the total plant production costs (plant fuel costs plus plant nonfuel O&M costs) from the plant revenue.- Coverage l's (site revenue less the site total production costs)/ total production costs.~ This is a ' comparative measure of how many times the site covers its production ecsts. It also indicates j the level of site funds available to cover other site costs. ] i bebt-to-Equity Ratio is a corporate variable. The debt-to-equity ratio is calculated by the j equation: ' (long-tenn debt)/[(common equity) + (preferred stock)] Common equity includes the value of common stock and retained eamings. The capital structure of a utility is based on its debt, equity, and liabilities as reported on financial statements. The debt-to-equity ratio is a measure of financial position at one point in . time. Interpretation of changes of the ratio in the same direction differs between the holders of the. debt and the owners who hold the equity. A-1

) m p ' Design Electrical Ratir', is the' design electrical output rating for the site in megawatts. . Fixed Charge Coverage is a corporate variable: It is the number of times income before i interest charges (operating income plus'other income) after taxes covers total interest charges and preferred dividend requirements. The larger the number, the better the performance. 1 Loss is a site factor that is (1-revenue factor) times the site revenue. . Moody's Bond Rating is a corporate variable.' The Moody's bond rating is obtained for the majority equity owner.- Not income is a corporate variable. It is the ' profit or the amount of eamings for the year, which 'is available for preferred and common stock dividend payments, and retained eamings.

Net lAcome Change is'a corporate variable. It is calculated by subtracting the. net income in a

. previous year from that in the current year. The change in net income is computed using the ' following equation: i [(1996 net income)-(1995 net income) = (1996 net income change) Nonfuel Operation and Maintenance Cost is a site variable.' It is the operating and l maintenance costs less the fuel costs. It includes the costs for items such as labor, j supervision, staffing, material, and overhead associated with activities to operate and raalntain the plant; Examples of activities are testing, repair, replacement, and preventive maintenance. 1 Nonfuel Operation and Maintenance Cost Change is a calculated site variable. The value is calculated by subtracting the nonfuel O&M cost for a previous year from the current year as: ~ (change in nonfuel O&M costs for 1996) = (1996 nonfuel O&M) - (1995 nonfuel O&M) Nonfuel Operation and Maintenance Cost 3-Year Moving Average is a site value obtained by averaging the nonfuel O&M costs over 3 consecutive years: (3 year moving average nonfuel O&M cost for 1996) = } [(1996 nonfuel O&M) + (1995 nonfuel O&M) + (1994 r anfuel O&M)]/3.. l Nonfuel Operation and Maintenance Cost per Megawatt Electrit 1 Nated is a calculated j l site variable. The value is calculated using nonfuel O&M cost divide by the sum of the MWe . rating of the units at the site.. Operating Ratio is a corporate variable. It is the ratio of corporate operating costs to operating - revenues or the proportion of revenues absorbed by costs. j " Percent Earned on Not Property is a corporate variable. It is obtained by dividing operating ~ income by average net property for the year. Percent Nuclear is a corporate variable. It is based on the percent of the 1995 corporate j power generation that was nuclear.- it varies slightly from year to year but not significantly. { A-2

t p Percent Return _ on Common Equity is a corporate variable. It is a value that is regulated by the public utility commissions.' It is the percentage obtained by dividing income available for common stock (net income less preferred dividend requirements) by average common equity. Percent Retum on Invested Capital is a corporate variable. It is the percentage obtained by dividing income available for fixed charges by average total invested capital. ] Percent Return on Revenue is a corporate variable. It is the net Income divided by the revenue x 100 percent.- It is also referred to as the " percent retum on sales." Production Cost is a site variable equal to the sum of the fuel and O&M costs.' Preliminary analysis did not find that this. variable exhibited a good correlation with plants discussed at the SMM. Its definition is provided since it is used in other definitions. Production Cost per Megawatt Hour Generated 2-Year Moving Average is a site variable. It is calculated by averaging the costs /MWH oveir two consecutive years (e.g.,2-year average). ' (1996 2-year average cost per MWH) = [(1996 cost per MWH) + (1995 cost per MWH)]/(2) l Production Cost per Gross Megawatt Electrical Rated is a site variable, which is a measure of efficiency. It is calculated by dividing total production costs by total rating of the unit. Production Cost per Megawatt Hour Generated is a site variable, which is a measure of l efficiency. It is calculated by dividing total production costs by total net electric generation as reported to FERC. i Production Cost per Gross Megawatt Electrical Rated 3-Year Average is a site variable, which is a measure of efficiency. It is calculated by dividing total production costs per gross . megawatt rated for 3 consecutive years by 3 (e.g.,1996 Cost /MWe= [1996 cost /MWe + 1995 cost /MWE +1994 cost /MWe]/3. Revenue, a corporate variable, is the income received from the sale of electricity.~ Revenue Change, a corporate variable, is calculated by subtracting the revenue in a previous-j year from the revenue in the current year. The following equation is used: . (1996 revenue)-(1995 revenue) = (1996 revenue chanje) l i - Revenue Factor is a site variable. It is the ratio of the actual site revenue to the maximum possible revenue the site could have produced. Since the dollars per MWH cancel each other in the ratio, it is a site capacity iactor. L Revenue to Sales Ratio, a corporate variable, is the average corporate selling price of electricity. It represents the average selling price of electricity to the customers. Site' Operating Ratio is a site variable. It is the ratio of site total production costs to the site revenues or the proportion of site revenues absorbed by site costs. A-3

r;' Site Revenue is a site variable. It is calculated by multiplying the site net electrical generation in MWHs times the average corporate selling price of electricity in dollar per MWH. Utility Type is a corporate variable. It notes whether the site major equity owner is a publicly owned company, an investor-owned company, or an investor owned holding company. 1 l 9 A-4

Fr, l APPENDlX B Detalled Statistical Method and Results l I Logistic regression was used to identify the variables that correlated with the plants discussed at SMMs. Logistic regression is a standard equation (model) used to define a relationship between a response probability and explanatory variables based on the best fit of the data in l the equation. The logistic regression provides the coefficients or " parameter estimates" for the explanatory variables. Criteria used to evaluate models: (1)' First, each variable in the model must meet the p s0.05 significance level criterion. A significance level of 0.05 means there is a 95 percent chance the correlation is real, and not by chance alone. (2) The second crite'rion is based on the Chi-Square statistic, which measures the model's goodness of fit, given the data set. The larger the Chi-Square, the better the fit. A p-value is associated with the Chi-Square for the model. This p-value is the probability that the resultant Chi-Square could have occurred due to chance. A small p-value (p s0.01) indicates that It is highly unlikely the calculated Chi-Square occurred due to chance, and it can be concluded that a good fit is achieved. The candidate models were examined both in terms of the probability value associated with their Chi-Square statistic and the Chi-Square statistic relative to the other models. Comparison of Chi-Square results using other data sets is not valid. (3) The third criterion is a misclassification rate. Using a candidate model, the probability of discussion is calculated for each observation in the data base. If the probability of discussion is greater than 50 percent, but the plant / site was not discussed, then the model misclassified the observation. Likewise,if the probability of discussion is less than 50 percent, but the plant / site was discussed, the model misclassified the observa-tion. The misclassification rate is the percent of the plants that were misclassified. A probability of 50 percent is used as the misclassification threshold simply because if the ] probability of discussion is greater than 50 percent, the expectation is more likely than not that the site will be discussed. (4) INEEL recommended a cluster analysis of financial variable sets to validate the critical assumption that the financial variables alone could be used to select the plants for discussion at SMMS. Cluster analysis was performed on the data for each of the models using the variables associated with that model. The cluster analysis places objecta into groups or clusters mathematically suggested by the data, not defined a priod (i.e., the response variables l " discussed,""not discussed," is not provided) such that objacts in a given cluster tend to be similar to each other in some sense, and objects in different clusters tend to be dissimilar. In this study, the cluster analysis grouped the " sites" into two categories using just the variables and ignoring the plants actually discussed at SMMs. Two B-1

a 4 distinct clusters were produced mathematically corresponding to the " discussed" group and the "not discussed" group as identified by the financial variables alone. A misclassification rate was calculated using the plants that were actually discussed at SMMs. If a " discussed" site as predicated by the financial indicators is placed into a group of plants "not discussed" at a SMM, or vice versa, misclassification has occurred. INEEL advised that from a mathematical perspective, the results generally indicate more j information is needed to better form the " discussed" and "not discussed" groups, and to evaluate the results from the two approaches. The results of the logistic regression that 4 showed the best correlation with the plants discussed at past SMMs and the cluster analysis misclassification are shown below in Table B-1. The results in Table B-1 show the cluster analysis misclassification rate is greater than the regression model .'l misclassification rate. ) (5) As the last criterion, the face validity of the candidati. models was evaluated at every step. ) Stati tical Methods U' sed in Trend Analvsis f 4 ' Wilcoxon's rank sum test is used to compare the two SMM categories, it is the nonparametric analogy to tne t-test. i ~ The t-test requires that the data under analysis follow a normal distribution. Wilcoxon's rank sum test requires no distributional assumption. For several variables analyzed he e, the assumption of a normal distribution is not reasonable; therefore, the rank sum test is used for all of the variables. . The test statistic is based on the ranks of the data rather than the actual data. Mean rank scores are calculated for each category (discussed /not discussed). The null hypothesis is that there is no difference between the mean rank scores for the two categories. This is analogous to a t-test of equal means. A large test statistic causes rejection of the null hypothesis in favor of the altemative hypothesis that a statistically significant difference exists. The significance probability (p-value) associated with the statistic is the probability that the resulting rank mean scores could occur by chance when no difference exists. The smaller the probability, the more statistically significant the ~ difference between the two categories in terms of the variable being analyzed. A p-value of

approximately less than 0.01 can be considered significant.

An inspection of the direction of the difference between the mean rank scores for the two categories provides interpretation of the direction of the relationship. B-2

. Table B-1 Summary of results Model Data Analyzed Statistical Results ( Regression ) l Model Cluster Analysis ] Misclassification Misclassification ) Model Variables Chi-Square Rate (%) Rate (%) j i 1 Corporate-

• Fixed charge coverage 18,6 17.7 17.5 j

investor owned

• Return on net property i

1 l 2 All plants + Revenue factor 58 17.2 28 +Nonfuel O&M cost ] l + Coverage l 3 Plant-Single + Revenue factor 41 10.4 18.2 i Unit

• Loss

'Nonfuel O&M cost 4 Plant-Multiunit Coverage 66 22.4 24.5 l . Loss .Nonfuel O&M i 1 l l l l I l B-3 L.

G l'

i..

1 L 14 - Table B-2 $1ngle-unit plants discussed versus not discussed Discussed. ' Not Discussed. Significant - Variable ; Median Value Median Value Difference p Value - Capacity Margin (MWe). 2.9 8.66 No 0.5461 Production Cost per Gross MWe 941 No 0.3489 l-Rated ($/MWe) : 1891 1 l -- Contribution ($ Million) .-103 2461 .Yes 0.0002 l l. Production Cost per MWH : Generated ($/MWH) 28.8 22.14 Lyes 0.0107. Coverage 0.688 2.07 Yes <0.0001 Pmduction Cost per Gross MWe Rated 3-Year Average ($/MWe) ' 859 929 No-0.8001 .p Debt-to-Equity Ratio. 1.00 0.984 No 0.5602 Fixed Charge Coverage 2.7 2.7 No 0.7563 Netincome Change ($ Million) L 12 4.8 No 0.2278 Loss ($ Million) -184 92 Yes 0.0003 Net income ($ Million) 226 211.5 No. 0.8137 Nonfuel O&M Cost ($ Million) 108 83.5 Yes- <0.0001 NdnfuelO&M Cost per MWe 122 104 No <0.0741 l Nonfuel O&M Cost Change ($' Million). ~ 5.99 0.523 No 0.2562 Operating Ratio (%) 81.45 84.7 No 0.6089 Percent Nuclear - - 24 30 No 0.6875 Percent Retum on Equity 11.9 11.6 No 0.9451 Percent Retum on invested Capital 7.35 8 No 0.3891 Percent Retum on Net Propertyf 7.7 8.35 No 0.3906 i Percent Retum on Revenues 10.35 8.75 No 0.7785 . Revenue Change ($ Million) 116 69 No 0.2453 Revenue ($ Million)1-1368 2516 No 0.0221 'T Revenue to Sales Ratio (cents /kWe 12.965 8.18 No 0.0718 Revenue Factor - 0.589 0.799 Yes <0.0001 Site Operating Ratio' O.513 0.320 Yes 0.0006 B-4 j :. 1

.? Tchla 0-3 Multlunit pl:nts di: cuss 2d versus net dl2 cussed - Discussed Not Discussed Significant Variable Median Value Median Value Difference p-Velue Capacdy Margin (MWe) 54 13 Yes 0.0038 Production Cost per Gross MWe.- f Rated ($/MWe)- 841 .952.32 No 0.0198 Contribution ($ Million) .473 715 Yes <0.0001 Production Cost per MWH. Generated ($/MWH) 24.71 17.9 Yes <0.0001 . 0.0001 Coverage 2.103 3.032 Yes Production Cost per Gross MWe. - Rated 3-Year Average ($/MWe) 809 943 No 0.0133 Debt-to-Equity Ratio 1.090 0.960 'Yes 0.0002 Fixed Charge Coverage 2.25 2.7 Yes 0.0020 Netincome Change ($ Million) 67.5 19 No 0.2214 458 231 Yes <0.0001 Loss ($ Million) Net income ($ Million) 355 468 No 0.1924 Nonfuel O&M Cost ($ Million) 166 .159 No 0.3882 Nonfuel O&M Cost per MWe -Two Unit 85 80 No .2613 Nonfuel O&M Cost per MWe ' -Three Unit - 221 78 Yes-0.0002 Nonfuel O&M Cost Change ($ - Million) 15.15 2.289 Yes 0.0089 Operating Ratio (%) 82.1 80.05 Yes 0.0020 Percent Nuclear 39 30 No 0.0544 Percent Retum on Equity

11.4 11.8 No 0.1777 Percent Retum on invested Capital 7.3 7.9 Yes 0.0070 Percent Retum'on Net Property -

7.85 8.6 Yes 0.0004 Percent Retum on Revenues 8.5 10.65 Yes 0.0024 Revenue Change ($ Million) 145 114 No' O.3908 Revenue 5260 4489 No 0.2828 Revenue to Sales Ratio ' l (cents /kWe) : 7.56 .6.85 No-0.0868 Revenue Factor - .607 .0815 Yes. <0.0001 Site Operating Ratio -0.323 .248 Yes <0.0001 B-5

Public Comments to the August 7,1998, Federa/ Reglster Not!ce (FRN) Question on the "Special Study-Methodology For identifying Financial Variables For Trend Analysis" QUESTION 3.a: Financial indicators (variables] can be used to gain insight into licensee performance in conjunction with other assessment measures. They would not be relied upon solely to draw conclusions on licensee performance in an integrated assessment process. How should financialindicators be used in the assessment of licensee performance? COMMENTS:

1. The majority of the eleven respondents stated that the financial variables should not be used. In addition, the respondents stated the following reasons given for not using financial variables: (1) the NRC role should be to assure financial qualification and adequate decornmissioning funding, (2) the staff should not be looking for variables that would have predicted discussion at previous Senior Management Meetings, (3) the purpose of variables should be to determine how well licensees are performing in protecting the public health and safety, and (4) financial variables are statistical correlations that do not prove causality.

2. Given competitive pressures on the utilities and the potential for significant reductions in the financial resources available to support nuclear plant operations, the NRC should consider expanding its review of plant performance tc Oclude examination of financial variables.

3. Financial pertu ;ance variables may be useful in determining why a licensee's performance had declined. However, the regulators should be able to determine performance problems prior to evincing themselves in financial variables.

4. Only use as an " additional consideration" because there is no relevant data that shows a direct relationship between financial performance and plant risk that is not better measured by safety measures.

RESOLUTION: The comments do not affect the conclusions of the study. With regard to comments 1(1),1(3), and 1(4), the study quotes Arthur Andersen, who supported their recommendation to use the variables by stating "Given the economic forces behind production and safety, assessing indicators of economic stress and management's response to them ahead of time should allow the NRC to achieve these improvements: earlier identification of problems, fewer safety risks to the public, earlier and less costly resolution of problems." With regard to comment 1(2), since the proposed use of the financial i variables is in the information base for the SMM process, it is logical to correlate the financial variables with past SMM decisions. t I 1

E, With regard to comment 1(3), the study states that the role of financial - E variables is to add to the information base and to identify where the NRC may need to be attentive to the possibility that a licensee may be compromising safety to reduce costs. - Conclusion 2 of the report states that mathematical analysis concluded that financial variable trend analysis alone

should not be used to determine whether a plant should be discussed or not discussed in the SMM process. However, analysis of the trends of financial variables does provide useful information to be used in conjunction with plant

performance trends. The financial trend analysis also helps to distinguish sites at which financial stress is not affecting plant performance trends from sites at which financial stress may be a precursor to future adverse trends in

. plant performance. ~ With regard to comment 1(4) the study identifies cases where an NRC inspection and a State audit found that financial stress compromised safety when extemal or intemal conditions emphasize operations to enhance short- ' term financial perfonnance. Other cases are documented in the study. In discussions with the Advisory Committee on Reactor Safety on February 3, 4 1998, several NRC Special inspection Reports and NRC Diagnostic Evaluation Reports were used to show that financial stress was a cause of safety problems. Conclusion 6 of the report states that additional data evaluation is needed to enhance the identification of financial stress that may cause or contribute to an adverse effect on plant performance. Conclusions of the studh are consistent with comments 2,3,4. i QUESTION 3.b: _ Are there any financial methodology processes that will provide a more useful set of financial variables? COMMENTS: The majority of the seven respondents stated that this was an issue for utility management and the financial community. In addition, the respondents stated that financial variables are not a predictor of safety outcomes or plant safety. The respondents also stated that the financial variables would penalize plants that shutdown to correct problems and could cause inappropriate behavior to satisfy NRC indicators. Use the amounts of civil penalties as a financial variable. RESOLUTION: The comments did not affect the conclusions of the report. Civil penalties are already used in the information base for the SMM process. Preliminary statistica! analysis found that civil penalties alone weakly correlated with past SMM discussion plant decisions. Investigation found that a single civil penalty may reflect multiple enforcement actions over different periods of time causing the statistic to lag. The other comments did not offer additional financial methodology processes that would provide a more useful set of financial variables. However, other comments about the use of the financial variable trends were provided and did not affect the conclusions of the report. Their resolution follows. 2

---

^

With regard to the comment that the financial variables are not a predictor of safety outcomes or plant safety, the study identifies cases where an NRC inspection and a State audit found that financial stress compromised safety when external or internal conditions emphasize operations to enhance short-term financial performance. Other cases are documented in the study, in discussions with the Advisory Committee on Reactor Safety on February 3, 1998, several NRC Special Inspection Reports and NRC Diagnostic Evaluation Reports were used to show that financial stress was a cause of j safety problems. The financial variable trends are used responsibly with other information to preclude penalizing plants that shutdown to correct problems. The study i contains a quotation from Arthur Andersen that indicates the benefit of ) financial variables. Arthur Andersen stated "Given the economic forces ) behind production and safety, assessing indicators of economic stress and J management's response to them ahead cf time should allow the NRC to achieve these improvements: earlier identification of problems, fewer safety 1 risks to the public, earlier and less costly resolution of problems." k Emphasis on financial variable trends causing inapprcpriate behavior is a i concern. The study documents cases where the licensee compromised safety to enhance short-term financial performance. The financial variables are not new to the industry. Financial variables, to include revenue factors (capacity factors), non-fuel O&M costs, and total production cost per megawatt hour, have been commonly publicized by the industry as comparative measures of effectiveness and competitiveness by the industry. QUESTION 3.c: The financial variables are based on publicly available data. Are there other financial data that could be made available that would be more useful? COMMENTS: The majority of the six respondents stated no and that it would not be in the utility's interest to expand the financial data they provide to the public. Several respondents noted that at least one utility is presently providing less information to FERC to protect its competitiveness. There is not a financial variable that can provide accurate insight into licensee safety performance. RESOLUTION: No change to the study is required since no other publicly available data has been suggested. In addition, Conclusions 2 and 6 of the report address the comment on the accuracy of the insights into licensee safety performance. 3}}