ML060830449
| ML060830449 | |
| Person / Time | |
|---|---|
| Site: | Summer  |
| Issue date: | 03/24/2006 |
| From: | Moorman J NRC/RGN-II/DRS/OLB |
| To: | Archie J South Carolina Electric & Gas Co |
| References | |
| 50-395/05-301 50-395/05-301 | |
| Download: ML060830449 (65) | |
See also: IR 05000395/2005301
Text
SCE&G
March 24, 2006
South Carolina Electric & Gas Company
ATTN: Mr. Jeffrey B. Archie
Vice President, Nuclear Operations
Virgil C. Summer Nuclear Station
P. O. Box 88
Jenkinsville, SC 29065
SUBJECT:
REACTOR AND SENIOR REACTOR OPERATOR INITIAL EXAMINATIONS
POST-EXAMINATION COMMENT RESOLUTIONS - VIRGIL C. SUMMER
NUCLEAR STATION 05000395/2005301
Dear Mr. Archie:
This is in response to your letter dated January 25, 2006, providing post-examination comments
on the Reactor Operator (RO) and Senior Reactor Operator (SRO) initial licensing written
examinations administered at your V. C. Summer Nuclear Station on January 10, 2006. As you
requested, we have reviewed the information that you provided regarding the quality and
content of the examinations to determine if the examinations should be considered valid.
Each comment was reviewed, carefully and deliberately, to determine if my staff followed the
NUREG-1021, Operator Licensing Examination Standards For Power Reactors, Rev. 9
guidelines for content, operational, and discriminatory validity when preparing the examinations.
In addition, portions of the examinations and comment resolutions were independently reviewed
by examiners in Region I and NRC headquarters.
This review indicates that the examinations met the overall quality and content guidelines
specified in NUREG-1021, Rev. 9, and that the number of flaws was within the normal range for
examinations of this type. As such, we consider the RO and SRO examinations that were
administered on January 10, 2006, to be valid and reliable tools for measuring the competence
of your licensed operator applicants and dispositioning their applications.
I understand that you have completed a root cause analysis to assess the high failure rate on
the January 10, 2006, examination. At your request, we have scheduled a public meeting to
discuss the findings of your root cause analysis, any corrective actions you may have taken or
have planned, and your conclusion on the validity of the written examinations.
Enclosure1 to this letter addresses your general comments on examination content and
construction. Enclosure 2 responds to your comments on specific questions. Enclosure 3
contains the as-administered examinations.
In accordance with 10 CFR 2.390 of the NRC's "Rules of Practice," a copy of this letter will be
available electronically for public inspection in the NRC Public Document Room or from the
Publicly Available Records (PARS) component of NRC's document system (ADAMS). ADAMS
SCE&G
2
is accessible from the NRC Web site at http://www/nrc.gov/readingrm/adams.html (the Public
Electronic Reading Room).
Thank you for your cooperation in this matter. If you have any questions regarding this letter or
the enclosures, please contact Steven D. Rose at (404) 562-4609, (Internet E-mail:
sdr2@nrc.gov), or me at (404) 562-4647, (Internet E-mail: jhm2@nrc.gov).
Sincerely,
/RA/
James H. Moorman, III, Chief
Operator Licensing Branch
Division of Reactor Safety
Docket No.: 50-395
License No.: NPF-12
Enclosures:
1. General Examination Comment Responses
2. Specific Examination Question Comment Responses
3. Initial RO and SRO Written Examinations
cc w/encls: See next page
SCE&G
3
cc w/encls:
R. J. White
Nuclear Coordinator Mail Code 802
S.C. Public Service Authority
Virgil C. Summer Nuclear Station
Electronic Mail Distribution
Kathryn M. Sutton, Esq.
Electronic Mail Distribution
Henry J. Porter, Director
Div. of Radioactive Waste Mgmt.
Dept. of Health and Environmental
Control
Electronic Mail Distribution
R. Mike Gandy
Division of Radioactive Waste Mgmt.
S. C. Department of Health and
Environmental Control
Electronic Mail Distribution
Robert G. Sweet, Manager
Nuclear Licensing (Mail Code 830)
South Carolina Electric & Gas Company
Virgil C. Summer Nuclear Station
Electronic Mail Distribution
Robert M. Fowlkes, General Manager
Engineering Services
South Carolina Electric & Gas Company
Virgil C. Summer Nuclear Station
Electronic Mail Distribution
Thomas D. Gatlin, General Manager
Nuclear Plant Operations (Mail Code 303)
South Carolina Electric & Gas Company
Virgil C. Summer Nuclear Station
Electronic Mail Distribution
David A. Lavigne, General Manager
Organization Development
South Carolina Electric & Gas Company
Vigil C. Summer Nuclear Station
Electronic Mail Distribution
Gary Moffatt
Training Manager
Virgil C. Summer Nuclear Station
P. O. Box 88 (Mail Code P-40)
Jenkinsville, SC 29065
_________________________
OFFICE
RII:DRS
RII:DRS
SIGNATURE
NAME
SRose:pmd
JMoorman
DATE
3/ /2006
3/ /2006
3/ /2006
3/ /2006
3/ /2006
3/ /2006
3/ /2006
E-MAIL COPY?
YES
NO YES
NO YES
NO YES
NO YES
NO YES
NO YES
NO
Enclosure 1
General Examination Comment Responses
V. C. Summer (the facility) provided comments on General Concepts associated with the
development of the examinations. In general, these comments contend that the examinations
contained too many test items that required memorization of information outside of the scope of
an operators responsibility to perform or memorize. Specifically, the facility believes that
certain test items required the knowledge of specific procedural steps or parameters that need
not be committed to memory. The facility also implies that the examination authors may have
underestimated the difficulty of a concept in preparing the examination. Additionally, these
comments point to various psychometric flaws with certain questions.
References:
(1) Letter from Jeffrey B. Archie to W.D. Travers, dtd January 25, 2006 (ADAMS Accession
Numbers ML060320418, ML060320366, ML060320297)
(2) NUREG 1021, Operator Licensing Examination Standards For Power Reactors, Revision 9
(i.e., Examination Standards)
Background
In Reference (1), The facility seeks to invalidate the January 10, 2006, NRC license
examinations for which eight (5 RO and 3 SRO) applicants failed the written examinations. The
facility admits to possible internal issues (audit exam, validation, exam content and structure) as
possible causes of its high failure rate, but also faults the failure of exam developers to adhere
to several NUREG 1021, Rev. 9, requirements (Reference (2)), as a further basis.
The facility relies upon citations from Appendices A and B in Reference (2) to bolster its
arguments. In particular, Reference (1), Enclosure 1, II.A., cites that too many test items
assessed memory level knowledge vice application levels of knowledge not supported by the
Examination Standards; further, the facility contends that memory level items cited were
embedded in procedures not required to be memorized, citing that a high number of questions
required only memorization of fact or fundamental thought and resulting in an examination
providing marginal evaluation of the higher cognitive thought processes required of operators on
the job."
Comment:
Paradoxically, the facility makes a "reverse logic" argument, suggesting that operators should
not know the very fundamental thought or knowledge upon which understanding or application
is based. In fact, Blooms taxonomy (Reference (2), Appendix B) is predicated on the basis of
this hierarchy of knowledge, that in essence, understanding or application of knowledge
presumes the knowledge of "fundamental thought" as the fundamental basis for answering any
question at a higher level. Therefore, at a minimum, the NRC, through its license examination,
would reasonably expect an operator applicant to know the lowest level of fundamental
thought in a test item.
It is true that highly operationally valid test items -- to the extent possible -- seek to assess
applicant knowledge at the understanding and application levels; however, as recognized in the
2
Enclosure 1
Reference (2), Appendices A, B, and ES- 401, constructing every test item at the higher
cognitive level is not a requirement in examination development. In this regard and related to
assessing fundamental thought, examination test policy in Reference (2) (ES-401. D.2.c.),
states that, 50-60 percent of examination items should be written to test knowledge at higher
cognitive levels (HCLs). By requirement, therefore, 40-50 percent of the remaining items are,
by necessity, to be written at the lower cognitive levels (LCLs); therefore, testing at the lower
cognitive level (i.e., fundamental knowledge or thought) is required for examination purposes
for nearly one-half of the test items. (See Reference (2), Appendix A, C.3.c for a further
explanation of what constitutes LCL and HCL test items).
This policy concerning the balance between HCLs and LCLs was arrived at for two significant
reasons: (1) setting the level of knowledge parameters allowed overall examination levels of
knowledge (applicant understanding) and levels of difficulty to be controlled for purposes of
National examination consistency, meeting the requirement for Uniform Conditions (Section 107
of the Atomic Energy Act of 1954) and (2) setting the above parameters met industrys stated
need to control resources because, as argued, too many HCL items would require greater
industry resources (time) to develop.
Multiple Choice (MC) Format and Memory Recognition
Moreover, it should be noted that the MC format largely works in favor of the applicant. Indeed,
the correct answer is contained among the alternative four choices (i.e., distractor set). In fact,
the MC format is often referred to as a recognition or aided recall item type, in which the test
taker need only recognize the correct answer among the alternatives in the distractor set and
within the context of the test items stem -- not an unreasonable expectation for the well-trained
applicant.
Alternatively, the supply type format, or unaided recall item (such as the short answer or essay
format) requires the test taker to recall, or reproduce the answer from memory -- a more
mentally effortful process. This item is also referred to a supply-type because the test taker
must supply the answer without any assistance or aid and is considered to require memory
recall, in the truest sense. In sum, the unaided recall format is not required for the NRC exam
and thus, memory recall is not required to answer a test item; therefore, the argument that the
test taker must recall from memory steps in a procedure is moot and inapplicable.
It should be noted that the NRC does not require knowledge of memorized procedures in its
written test item format, but rather, requires that operators recognize, or reasonably infer, the
correct answer within the context of the given conditions and among the choices in which the
answer is already stated and available. Moreover, Reference (2), Appendix B, C.1.a supports
the appropriateness to develop questions regarding knowledge that is embedded in or covered
by procedures within written test items.
3
Enclosure 1
The reasonable question to be asked in a well-developed MC question is the following:
Given the conditions stated in the stem and the choices among the distractor set, for which one
of the options is visibly available as the correct answer, should the safety knowledgeable
applicant (i.e., one whose past performance on comparable measures has been above the 80
percent level in training performance measures) be able to select the correct answer, even
though some of the information stated in the question was not required to be explicitly
memorized?
The NRCs expectation is YES. Reason: The mental problem solving process contained in any
given test item may not rely upon the complete memory of every detail required to answer it;
however, given the availability of the answer already in the question, a knowledgeable applicant
who has been trained and is generally familiar with the job related procedures should be able to
deduce and arrive at the correct answer (Reference (2), Appendix A, C.3.c.).
Test Item Discrimination
Discriminant validity, one of three elements of validity, is the sina qua non (essential) of the
NRC license examination. If the examination does not intend to discriminate between the
knowledgeably safe from the knowledgeably unsafe applicant, then there is little reason to
administer it (Reference (2), Appendix A. C.3.a.).
Good discriminating distractors play a key role in both appropriate level of knowledge and level
of difficulty. By design, distractors are intended to distract. If distractors in a test item failed to
distract lower achieving test performers, then the test item is in doubt of its discriminant validity.
The distractors play a role in the test item since distractors must be ruled-out as answer choices
in the applicants mind. In so many words, the distractor, if plausible, must be determined to be
incorrect; the mental process in ruling out a distractor is just as important in selecting the correct
answer. To accomplish this, the applicant must apply his reasoning skills coupled with his
knowledge set of the system or procedure being tested; this process may require (in the mind)
understanding of system interrelations. In effect, good distractors aid in the process of HCL
thought through the linking and bridging of knowledge so as to make informed decisions to
eliminate incorrect options.
As stated in Reference (2), Appendix A, the stem of the question can also have conditions that
must be interrelated to the demands of the stem; in effect, the stems conditions can play a
similar role in the Ruling out/in mental process of separating the distractor in the distractors set
from the answers. Good discrimination test items cause weak applicants to miss the item, an
intended purpose of any valid license examination.
KA/Test Item Mismatches
It should be noted that not all psychometric issues referred to in Reference (2), ES-401 are
psychometric errors of the test item, per se. For example, a KA topic/test item mismatch is not
4
Enclosure 1
a test item error. As its category suggests, it is only a mismatch between the randomly selected
topic and its subsequent test item. In no way does a KA/test item mismatch violate, or
necessarily, invalidate the test item. In fact, all safety significant KA topics have an equal
chance of selection for the exam, and consequently, all operators are expected to be able to
answer a test question from any selected safety significant KA. In essence, the test item
mismatch likely refers to some other KA for which the operators would, nonetheless, be
expected to answer. Thus, the argument to invalidate the test item because it does not match
the KA is moot for operator knowledge testing purposes.
In addition, KA/test item mismatches may be subject to interpretation and its degree of fit is not
always an exacting process. Notwithstanding, the argument that the test item should be deleted
from the examination solely because it did not fit the KA is not a valid argument and misses a
very important point: While it is desirable to achieve as close a fit as possible to the KA topic,
the interpretation that the test item does not match, does not invalidate the test item, per se.
Reason: Since any number of randomly selected safety-important KAs might have been
selected and a subsequent test item developed from it, the applicant would be, nonetheless,
expected to answer the question. From the facilitys or applicants perspective, KA/test item
match is irrelevant toward being expected to answer the question. Had a different KA been
selected that corresponded to the subject test item, then the applicant would have been
expected to answer it, regardless.
The KA/test item match issue is a matter of concern to the NRC because of its potential for
abuse when the facility licensee develops the examination. In the past, the NRC has
determined instances where test developers used past test bank items to suffice, for fit, for
randomly chosen KAs. It was observed that some test items were chosen for matters of
convenience and resource savings and upon close inspection, did not match or fit the chosen
KA topic. This error did not invalidate the test item, but if detected before exam administration,
could be corrected with a replacement test item. If a KA mismatch was detected during the post
examination process, this served as a lesson learned with increased scrutiny in subsequent
exams.
In summary, the reason that the NRC is interested in KA/test item matches is to prevent test
developers from using inappropriate test bank items to fit the randomly and systematically
selected test topic KA. In the past, authors relied too heavily upon test bank items that were not
clearly linked to the KA. Controlling for KA topic/test item match is an issue of test developer
discipline, integrity, and art (skill of the craft), but the resulting test item, in the subject case, may
still be a valid test item since it could be linked to another KA topic.
Level of Difficulty
As it states in its Reference (1), Enclosure 1, II.A, the facility makes an incorrect interpretation
by referring to exam authors underestimating the difficulty of concepts. Rather, seasoned exam
authors sometimes tend to underestimate the difficulty of test items.
5
Enclosure 1
Seasoned exam authors, who develop or review test items, may tend initially to evaluate test
items as more easy in difficulty than they might otherwise be to the applicant. This
phenomenon occurs because the test author has seen, repeatedly over time, many similar,
like-kind items. This natural tendency, therefore, can result in an underestimate of its true
difficulty level to the applicant. Again, this is a tendency that the test authors are made aware of
in the Examination Standards and can be easily resolved by reexamination, discussion, and
validations. As stated earlier, the test item development and validation processes rely upon
subject matter judgment; this judgment determines technical validity, level of knowledge, and
level of difficulty (discriminant validity).
SPECIFIC ITEM COMMENTS
The facility contests a large number of test items; some items are contested on the basis of
technical accuracy, i.e., no correct or multiple correct answers, while others are contested on
psychometric flaws, most of which are commonly cited and apply to multiple items.
The common themes of cited psychometric flaws are as follows:
!
KA test item mismatch
!
Testing operator knowledge and procedures beyond expectation and from
memory recall
!
Testing at low, non-operationally valid levels of knowledge
The below sample of items is illustrative of these common themes, and the NRC comments are
set forth. Specifically, several of the facilitys post examination item comments are erroneous
and based upon misperceptions in the Examination Standards, Reference (2), and other test
development practices. As discussed above, those misperceptions are contained in the
facilitys Reference (1), Enclosure (1), II.A.
Item # 9 (p. 25) et al.: Facility comment states that information needed in the stem is not
essential to answering the question as a basis for its recommendation to delete the item.
NRC COMMENT: The facilitys statement is a misperception of test practices. Test items that
contain non-essential information toward answering the question is precisely what is desired in
a good, discriminating test item. As discussed earlier (Reference (2) Appendix A., C.3.a),
discriminating test questions require applicants to rule-out distractors. In essence, distractors
may occur in the stem as well as in the distractor set; moreover, such distractors, or conditions
within the stem, are intended to distract the unknowledgeable applicant.
The NRC trains its examiners to include non-essential information in the stems of its test items.
Examiners are encouraged, during training, to include either irrelevant or incorrect decisions
within the question stems or in the distractor set to identify the unknowledgeable applicant. A
question that contains no chosen distractors are questions that do not serve their purpose;
rather we expect those applicants who are relatively poorer performers, to choose a distractor.
6
Enclosure 1
In truth, knowledge is best assessed when test items require applicants to separate the relevant
from the non relevant and the essential from the non-essential. Test items that challenge
operators to perform these mental tasks promote the interest of public health and safety.
The facilitys comment further states that the item requires applicant memorization beyond that
of immediate actions as its basis to recommend the deletion.
NRC COMMENT: As stated earlier, the mental problem solving process contained in any given
test item may not rely upon the complete memory of every detail required to answer it; however,
given the availability of the answer already in the question, a knowledgeable applicant generally
familiar with the job related procedures should be able to deduce and arrive at the correct
answer (Reference (2) Appendix A, C.3.c).
Items # 26 (p. 59); # 30 (p.82); # 44 (p.96); #45 (p.102); #69 (p. 123); #77 (p.140): The facility
comments that these items require application of procedures beyond its expectation of
operators to recall from memory.
NRC COMMENT: Refer to above comment to item #9.
Items #29 (p.74); #30 (p. 82); #44 (p. 96); #55 (p.108); #77 (p.140); #80 (p.153): The facility
provides multiple reasons to delete these items, one of which is KA test item mismatch.
NRC COMMENT: Notwithstanding the technical merits of the item, the KA mismatch argument
is irrelevant, unless the sample plan is violated. See earlier discussion on KA test item
mismatches.
Conclusion
It has been a long-standing premise, among test developers, that no examination can be a
perfect instrument for measuring human knowledge or ability. All examinations, even those
among the best, well-known standardized examinations, contain some degree of error in their
measurements; these errors may be a combination of errors: sampling, technical, psychometric
and grammatical.
Examination guidance, such as the Examination Standards, Reference (2), seek to promote
National reliability, a condition necessary for making validity determinations. The Examination
Standards further ensure that measurement errors are held to a minimum before the
examination is administered. Moreover, such Standards govern the development and
administration of the overall examination to ensure that examinations fall within accepted
standards of psychometric quality; levels of knowledge and difficulty; sampling; test bank use
and new item development as well as consistent and uniform scoring and grading criteria.
Adhering to the Examination Standards thus fosters highly reliable NRC examinations.
Examination validations, including test developer reviews from the NRC as well facility
supported reviews, serve as additional checks and balances on the overall quality and integrity
7
Enclosure 1
of the instrument before it is administered. Despite all of the safeguard measures, it is not
uncommon to learn that after an examination is administered, further errors can be found to
exist in the examination; these post-exam errors can be the result of a closer inspection of the
examination, particularly in test items in which there were a high number of applicant misses.
Routinely, these errors, if substantiated, may result in one or more answers being considered
correct or an item deletion; routinely, post examination answer key changes do not exceed 5
percent of the examination.
To be certain, when a high number of applicants miss a given item, it is more closely examined
for its psychometric and technical validity. A test item, if determined to be valid, is never deleted
from an NRC examination. Moreover, the number of applicants who miss a particular test item
is never a basis to delete an item. Rather, when a high number of applicants miss a particular,
yet valid test item, its diagnostic value may point to other reasons as a cause. In sum, when
test performance or its outcome is less than expected, the answer is not -- invalidate or change
the test.
Technical errors and clarity errors may lead to more than one answer or an item deletion.
However, all psychometric errors are not equal in value; for example, individual test items
having either too low or too high levels of knowledge and/or difficulty have not been reasons to
invalidate an item or the examination overall. As stated above, some degrees of imperfection
on the test instrument are accepted as a premise in any examination process.
Invalidating an examination is a serious decision that would occur after an examinations
administration, and only, if in the judgment of the NRC, a sufficiently large number of items had
been deleted to affect the sample plan adversely so as to bring into question the reliability and
validity of the examination to make a license decision. The NRC review of test items on which
the facility provided comments resulted in minor changes to the RO and SRO examinations.
The impact on the sample plans as a result of these changes was assessed and determined to
be insignificant and not of sufficient magnitude to invalidate the examinations.
Enclosure 2
Specific Examination Question Comment Responses
The facility provided comments on 56 of the 100 questions on the written examination
administered on January 10, 2006. These comments were provided in a letter dated
January 25, 2006 (RC-06-0027) from Mr. Jeffrey B. Archie, Vice President, Nuclear Operations,
V. C. Summer Nuclear Station to Dr. W. D. Travers, Regional Administrator, Region II. This
document can be found in the Publicly Available Records (PARS) component of NRC's
document system (ADAMS) at Accession Nos. ML060320418, ML060320366, ML060320297 .
ADAMS is accessible from the NRC Web site at http://www/nrc.gov/readingrm/adams.html (the
Public Electronic Reading Room).
This enclosure provides the NRC analysis and disposition of the comments. The NRC
response to the comments is ordered according to examination question number. The Reactor
Operator Examination consists of questions 1 - 75 of Enclosure 3. The Senior Reactor Operator
Examination consists of questions 1- 100 of Enclosure 3. The page numbers included in the
comment resolution correspond to the comment page number in Enclosures 1 and 2 of the
above mentioned letter.
The facility contended that NUREG-1021, ES-401, Form 401-6, Written Examination Quality
Checklist, items 3, 9 and 10 were not adequately met. The NRC has reviewed these items on
Form 401-6 following the post-examination comment resolution and has determined that these
items were adequately met. The results of this review are addressed in the analysis of the
individual questions that follows.
Test question RO #2 (K/A 002A4.08): Facility comments on page 238.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question adequately tests the knowledge required by the
KA topic and satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan
is not warranted.
Analysis:
The original KA addresses safety parameter display systems as a reference to systems used to
monitor the reactor for verification of parameters related to safe operation. It is not limited to the
non-safety related Safety Parameter Display System as stated by the facility. The OT)T trip
setpoint is generated off of RCS parameters and which are monitored by the operators in the
control room. The question provided the operators a realistic indication, displayed as an alarm,
and tested the knowledge of expected plant response. To answer the question, the applicants
needed to understand displays of monitored RCS safety parameters.
2
Enclosure 2
The alarm presented in the stem of the question occurred as a result of maneuvering the plant
at an excessive rate. There was no faulty or erratic operation of detectors postulated in the
question as stated by the licensee.
KA/test item mismatching is subject to interpretation. A less than precise degree of fit does
not invalidate the test item. The question author attempts to match the KA topic as close as
possible. From the facility or applicants perspective, KA/test item match is transparent to
answering the question.
Test Question RO #4 (003AK1.17): Facility comments on page 238.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The RO examination is not limited to items that relate to a control room operators interface with
the physical controls in the control room. Operators should have a basic knowledge of the
fundamental principles of reactor physics as they apply to operation of the reactor core under
abnormal conditions. The question tests the applicants knowledge of the operational
implications to the reactor core of a dropped rod as they relate to changes in fuel temperature
coefficient (FTC). The operational implication is related to how an abnormal rod pattern affects
the operation of the core. KAs are randomly selected and may include some topics that are
fundamental knowledge. These topics may be included on the RO exam and are valid test
items since the importance factor of the K/A is above 2.5.
This question goes beyond the basic GFES in that it tests the applicants retained knowledge of
how a dropped rod affects the FTC. The operational effects of a dropped rod are not typically
addressed on the GFES exams.
The NRC assessed this question as a memory level question, was counted as one of the
allowed memory questions, and was designated as such on the 401-9, Written Examination
Review Worksheet.
3
Enclosure 2
Test Question RO #5 (004G2.4.49): Facility comments on page 238.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
Immediate operator actions are actions that are sometimes designated by high level procedures
as immediate operator actions. However, immediate operator actions can also refer to skill-
of-the-craft actions that would be taken in a prompt and prudent manner without necessarily
referencing a procedure.
This question is a reasonable match of the KA, given the dual nature of immediate operator
actions. Additionally, the closest match KAs provided by the licensee are within the same
Tier, Group, and Category of the sample plans the original KA. Therefore, changing the KA
would have no effect on the sample plan.
Test Question RO #7 (005K6.03): Facility comments on page 238.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and should test at the higher cognitive level.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question tests whether applicants can recognize the failure of the RHR system to perform
its design basis function resulting from an errant closure of the cooling water supply to the RHR
heat exchanger. In this question, the effect is that the RHR system cannot perform its design
function and the loss of the RHR heat exchanger results from isolating the cooling water supply
(CCW). This could occur if an Auxiliary Unit Operator (AUO) incorrectly implemented a system
isolation on the wrong train in support of a maintenance activity. An understanding of system
4
Enclosure 2
interrelationships, such as the RHR-CCW interface, is important operator knowledge since this
is the ultimate heat sink.
Test Question RO #9 (007A2.05): Facility comments on pages 24-42.
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of their operators (i.e. operators are not required to know specific alarm setpoints
only nominal operating parameters). The facility also contends that there are psychometric
problems with the stem in that there is information present that is not needed to answer the
question.
Facility comment not accepted. This test item is appropriate for the initial licensing examination
and satisfies the requirement of Form ES-401-6, Item 10.
Analysis:
An applicant could answer this question correctly without knowledge of the specific PRT alarm
setpoints. Knowledge of the nominal PRT operating parameters would allow the applicant to
recognize that PRT pressure is 240% of the nominal value and abnormal. They would also
recognize that temperature is only 116% of nominal and level is still within the nominal band.
The following licensee learning objectives related to this topic imply that applicants are expected
to demonstrate knowledge of the alarm setpoints:
Enabling objective AB-2-07: DESCRIBE the normal operation of the following reactor coolant
system components, including component types and applicable setpoints. Item 5: Pressurizer
Relief Tank (PRT)
Enabling objective AB-2-14: DESCRIBE the operation of the following instrumentation and
control systems. Include function, instrumentation and setpoints. Item 6: Pressurizer relief tank
indication alarms and cooling.
The question stem states that operators are aligning the RHR system for cooldown. The RHR
pump suction relief valves discharge to the PRT. An inadvertent lifting of these valves provides
a possible reason for the abnormal condition of the PRT. This information establishes an
operational basis for asking the question and is the sole reason why distractor A is incorrect.
The facilitys statement concerning including information in the question stem that is not needed
to answer the question represents a misunderstanding of acceptable question construction
practice. Test items that contain non essential information toward answering the question is
precisely what is desired in a good, discriminating test item. Discriminating test questions
require applicants to rule-out distractors. In essence, distractors may occur in the stem as well
5
Enclosure 2
as in the distractor set; moreover, such distractors, or conditions within the stem, are intended
to distract the unknowledgeable applicant.
The NRC trains its examiners to include non-essential information in the stems of its test items.
Examiners are encouraged, during training, to include either irrelevant or incorrect decisions
within the question stems or in the distractor set to identify the unknowledgeable applicant. A
question that contains no chosen distractors are questions that do not serve their purpose;
rather we expect those applicants who are relatively poorer performers, to choose a distractor,
In truth, knowledge is best assessed when test items require applicants to separate the relevant
from the non-relevant and the essential from the non-essential. Test items that challenge
operators to perform these mental tasks promote the interest of public health and safety.
Test Question RO #11 (007K1.01): Facility comments on pages 43-51.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of operators.
Facility comment not accepted. This test item is appropriate for the initial licensing examination.
Analysis:
Applicants should understand that nominal pressure is not maximum pressure. The following
licensee learning objective related to this topic implies that applicants are expected to
demonstrate knowledge of the rupture disk setpoint:
Enabling objective AB-2-07 : DESCRIBE the normal operation of the following reactor coolant
system components, including component types and applicable setpoints. Item 12 - pressurizer
relief tank rupture disks.
2)
Facility comment is from Section II, General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to delete the question since this question can have more than one
correct answer.
Facility comment partially accepted. The question will not be deleted; however, choices B and
C will be accepted as correct.
6
Enclosure 2
Analysis:
Choices B and C will be accepted as correct since the stem did not specify whether the solicited
response was actual pressure or indicated pressure inside the PRT. These two responses
cover the entire design range, including the nominal rupture pressure. Applicants should know
that nominal pressure is not maximum pressure, therefore the range encompassed by Choices
B and C is appropriate for the correct answer.
Test Question RO #12 (008A3.06): Facility comments on page 239.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and includes implausible conditions in the stem.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The stem is plausible and satisfies the requirement of Form ES-401-6, Item 10.
Analysis:
The KA statement is not narrowly restricted to monitoring auto operation of motor current as
stated in the licensee comment. The KA statement for 008A3.06 is: Ability to monitor automatic
operation of the CCWS, including: Typical CCW pump operating conditions, including vibration
and sound levels and motor current. The question is a reasonable match for the KA in that it
tests the automatic operation of the CCW system in response to CCW pump mechanical
binding as evidenced by increasing motor current.
The question stem is plausible since the CCW surge tank level is given as a steady parameter,
indicating no challenge to system integrity. The applicants cannot assume that this parameter
is changing. This is covered in the pre-examination briefing contained in NUREG 1021
Appendix E, Item 7: When answering a question, do not make assumptions regarding
conditions that are not specified in the question unless they occur as a consequence of other
conditions that are stated in the question. Given this, the question stem provides clear
indications to a knowledgeable applicant that the operating CCW pump is slowing (decreasing
discharge pressure) because it is becoming mechanically bound (increasing motor amps). This
sets the conditions for the applicant to monitor the automatic operation of the CCW system by
anticipating an automatic start of the standby CCW pump in response to lowering pressure
caused by binding of the running pump.
Discriminating test questions require applicants to rule-out distractors. In essence, distractors
may occur in the stem as well as in the distractor set; moreover, such distractors, or conditions
within the stem, are intended to distract the unknowledgeable applicant.
7
Enclosure 2
Test Question RO #14 (010G2.4.4): Facility comments on page 239.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and is not knowledge required for an RO.
Facility comment not accepted. This question tests the knowledge required by the KA topic at
the appropriate license level and satisfies the requirement of Form ES-401-6, Item 9. A change
to the sample plan is not warranted.
Analysis:
The question covers the following KA with respect to the pressurizer pressure control system:
G2.4.4, Ability to recognize abnormal indications for system operating parameters which are
entry-level conditions for emergency and abnormal operating procedures. (CFR 41.10 / 43.2 /
45.6) IMPORTANCE RO 4.0 SRO 4.3
The question stem provides abnormal indications that resulted from a failed pressurizer
pressure instrument. The applicant must analyze the abnormal indications to determine that
they meet AOP entry requirements and then identify the appropriate AOP from the two choices
provided. This is a reasonable match for the KA.
KA/test item mismatch may be subject to interpretation and its degree of fit is not always an
exacting process. While it is desirable to achieve as close a fit as possible to the KA topic, the
interpretation that the test item does not match, does not invalidate the test item. From the
facility or applicants perspective, KA/test item match is irrelevant toward being expected to
answer the question.
The licensee has an enabling objective that is similar to the KA. Enabling Objective 3081:
STATE the entry conditions/symptoms of AOP-401.5 (from Instructor lesson plan AOP-401.5)
applicable to both the RO and SRO.
Additionally, this knowledge is appropriate for an RO since it has both a high KA value for the
RO application and is tied to 10 CFR 55.41(b)(10)
All applicants answered this question correctly.
Test Question RO #15 (011EK2.02): Facility comments on page 239.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
8
Enclosure 2
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment accepted. This question does not test the knowledge required by the
originally selected KA topic and would not have satisfied the requirement of Form ES-401-6,
Item 9. A change to the sample plan is warranted in order to satisfy the requirement of Form
ES-401-6, Item 9.
Analysis:
While the stem of the question does not provide an exact match to the wording of the KA, the
concept tested is consistent with the knowledge associated with the KA. The KA is associated
with knowledge of conditions created by a large break LOCA and how these conditions affect
operation of pumps. This question concerns the effect of increased pressure in the containment
system caused by a large, high energy break on the operation of the reactor coolant pumps.
The stem of the question indicates that a Phase B containment isolation has occurred
(automatic initiation of the reactor building spray pumps). The Phase B isolation results in
isolation of CCW cooling to the reactor coolant pumps. When this occurs, the pumps are
required to be tripped in accordance with EOP-1.0, Reactor Trip/Safety Injection Actuation, Step
8. The test item is valid and there is only one correct answer.
The sample plan will be changed and this question will be linked to KA 040AK2.01, Knowledge
of the interrelationship between steam line rupture and the following: valves. (IMPORTANCE
RO 2.6 SRO 2.5) which is in the same Tier, Group, and Category as the original KA, therefore,
has no effect on the validity of the sample plan.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
Test Question RO #16 (012A4.03): Facility comments on page 239.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment accepted. This question does not test the knowledge required by the
originally selected KA topic and would not have satisfied the requirement of Form ES-401-6,
9
Enclosure 2
Item 9. A change to the sample plan is warranted in order to satisfy the requirement of Form
ES-401-6, Item 9.
Analysis:
The facility comment is correct in that the question does not test specific knowledge of the
Reactor Protection System. However, the questions content validity is established by the link to
KA 013A4.02, Engineered Safety Features Actuation System (ESFAS), Ability to manually
operate and/or monitor in the control room: Reset of ESFAS channels (IMPORTANCE RO 4.3
SRO 4.4) (CFR: 41.7 / 45.5 to 45.8). This KA is of the same Tier, Group, and Category as the
original KA, therefore, has no affect on the validity of the sample plan.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
Test Question RO #18 (013K1.06): Facility comments on page 239.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and only tests at the memory level.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question reasonably ties to the KA by testing that the applicant understands the effect of a
delay in starting ESF components will have on the eventual success of accident mitigation. The
facilitys comment is a narrowly focused interpretation of the KA. Many KAs are written such
that they can be broadly interpreted. As such, it is not unusual or inappropriate to have a
question match more than one KA. This KA does not necessarily have to test the actual
physical connections between the ESFAS and ECCS systems due to the inclusion of the
and/or connector in the KA statement. This question tests the applicants knowledge of the
plant design basis, which is a required topic by 55.41(b)(7). The question requires the applicant
to understand the effect of malfunctioning ESF sequencers and correlate this effect to the
10
Enclosure 2
eventual success of accident mitigation. When answering the question in this context, it is
reasonable to classify this as a comprehension/analysis question.
Test Question RO #21 (022A1.04): Facility comments on pages 52-58 & 240.
1)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question matches the KA in that the cooling water flow considered was Reactor Building
Cooling Unit (RBCU) drain flow and not, as assumed by the licensee, system flow internal to the
RBCUs (CCW system flow). Additionally, the design limits aspect of this KA was addressed,
in that, a 0.7 gpm RCS leak is just below the TS limit of allowed unidentified leakage of 1 gpm.
The licensee lesson plan GS-7, Leak Detection System, states that the RBCU drain flow is one
of the most sensitive leakage detection systems in the reactor building. Possessing the
knowledge to answer this question correctly would allow the applicant to predict and/or monitor
changes in RCS leakage to prevent exceeding design, license, and RCS leakage limits. While
there are no physical controls for an operator to manipulate associated with RBCU drain flow,
monitoring drain flow with the knowledge of the implications of abnormal conditions is equivalent
to operating the controls.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
This question also matches KA 022A1.03, Ability to predict and/or monitor changes in
parameters (to prevent exceeding design limits) associated with operating the CCS controls
including: (CFR: 41.5 / 45.5): Containment humidity (IMPORTANCE RO 3.1 SRO 3.4) This KA
is of the same Tier, Group, and Category as the original KA, therefore, would have no affect on
the validity of the sample plan.
11
Enclosure 2
2)
Facility comments from Section II. General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to delete the question since this question is technically inaccurate
and has no correct answer.
Facility comment not accepted. The question will not be deleted, answer choice D will be
graded as the correct answer.
Analysis:
The question asks the applicants to select which indication may provide an alarm for a very
small RCS leak. While the contention that the leak may condense equally between the RBCUs
is theoretically possible, the only possible answer to this question is choice D. Answer choice D
is further supported by the facility lesson plan GS-7 which states the RBCU drain flow is one of
the most sensitive leakage detection systems in the reactor building.
Additionally, this question was used successfully on the V.C. Summer initial examination
administered on September 17, 2002 associated with the same KA. Two of the six applicants
missed this question on this examination, selecting the same wrong answer as the current
group of applicants. There were no post-examination comments on this question at that time.
Test Question RO #24 (024AG2.4.6): Facility comments on page 240.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and implausible conditions exist in the stem.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The stem is plausible and satisfies the requirement of Form ES-401-6, Item 10.
Analysis:
The plausibility of conditions in the stem as to the cause of the malfunction of the PORVs to
open is not germane to knowledge required to answer the question. The question sets
conditions that require emergency boration flow and tests applicant knowledge of both the
mitigation strategy for the conditions presented as well as the reason for the strategy. This
appears to closely match the original KA as well as the additional KA recommended by the
licensee. Matching multiple, highly important KAs indicates that this question has strong
correlation to required operator knowledge.
12
Enclosure 2
Test Question RO #25 (025AG2.1.32): Facility comments on page 240.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question is a reasonable match for the knowledge required by the KA. The question
provides the applicant with a CAUTION from EOP-14 that states: RHR Pumps should NOT be
run longer than 90 minutes without CCW flow to the RHR Heat Exchangers. The applicant is
then asked to demonstrate knowledge of the precaution by recognizing the correct reason for
including the precaution in EOP-14. It is assumed that if the applicant knows the reason for the
precaution, the applicant would take the appropriate action to prevent loss of the RHR system.
If this caution is not observed, it could result in a common mode failure of both RHR pumps.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
An additional KA match would be with KA 025AG2.4.20 Knowledge of operational implications
of EOP warnings, cautions, and notes. (CFR: 41.10 / 45.13) (IMPORTANCE RO 3.3 SRO 4.0)
This KA is of the same Tier, Group, and Category as the original KA, therefore, use of this KA
would have no affect on the validity of the sample plan.
Test Question RO #26 (026AA1.01): Facility comments on pages 59-71 & 240.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of operators.
This question will be deleted from the initial licensing examination due to a technical flaw.
13
Enclosure 2
Analysis:
The spent fuel pool limits are not contained in AOP-118.1. Since the stem asks for which
maximum temperature limit has been exceeded per AOP-118.1, this makes the premise of the
stem incorrect. The NRC proposed question contained a reference to the charging pump oil
cooler outlet temperature limit having been exceeded, which is specified in AOP-118.1. This
was replaced with the reference to the spent fuel pool cooling temperature at the request of the
licensee reviewers. The change was made to satisfy the licensee reviewers concern that the
level of difficulty was too high. Both the NRC and licensee reviewer failed to identify that the
spent fuel pool temperature limits were not included in AOP-118.1 during subsequent reviews.
The facilitys comment that this is an inappropriate test item and that the operators do not need
to be able to answer questions of this nature is inconsistent with the enabling objectives
included in the facility training program. The facilitys training program contains the following
enabling objectives:
1)
GS-5-07 - DESCRIBE the normal operation of the Spent Fuel Pit Cooling System
Components. Include component types and applicable setpoints: Refueling
Water Storage Tank, Spent Fuel Pit, Spent Fuel Cooling and Transfer Pump,
Spent Fuel Cooling Pump, Spent Fuel Heat Exchangers, Fuel Transfer Canal,
Cask Loading Area, Spent Fuel Cooling Demineralizer, Spent Fuel Purification
Pump, Spent Fuel Purification Filters, RM-L4.
2)
GS-5-13 DESCRIBE the normal operation of the Spent Fuel Pit Cooling System
including level, flow and temperature parameters.
3)
IB-2-19 LIST the condition or setpoint associated with the following Component
Cooling Water System limitations: (Item 18) Spent Fuel Heat Exchanger Flow
Low Temperature High.
Additionally, the KA catalog requires operator knowledge of normal values and upper limits of
temperatures of components cooled by CCW on a loss of CCW. KA 026AA2.04, Ability to
determine and interpret the following as they apply to the Loss of Component Cooling Water:
The normal values and upper limits for the temperatures of the components cooled by CCW
(CFR: 43.5 / 45.13) (IMPORTANCE RO 2.5 SRO 2.9).
2)
Facility comment from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
14
Enclosure 2
Analysis:
The question is a reasonable match for the KA. The question tests the applicants knowledge
that a loss of CCW will typically be evidenced by the temperature of the components that are
cooled by CCW. In this case the CCW temperature indications were taken as that of the
component being cooled which is directly related to the loss of CCW to that component. With
the loss of CCW flow, the temperature of the water within the CCW system itself would provide
no useful information, therefore, the question asked about the temperature indications of those
components cooled.
Test Question RO #28 (026K3.01): Facility comments on page 241.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The stem provides conditions for loss of an electrical bus which causes a loss or malfunction of
the CSS (Containment Spray System). The applicants must analyze this information to
determine which equipment remains operational and able to cool the containment. In addition
to asking the candidates what effect this has on the Reactor Building Spray pumps, they also
need to know the effect on the RBCUs. The RBCUs are part of the CCS (Containment Cooling
System) referenced in the KA. This question appears to closely match the original KA as well
as the additional KAs recommended by the licensee.
Test Question RO #29 (027AK1.02): Facility comments on pages 72-78 & 241
1)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and this question tests knowledge required for the General Fundamental Exam
(GFE).
15
Enclosure 2
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The K/A requires the testing of knowledge of operational implications of a Pressurizer Pressure
Control System (PPCS) malfunction, which is accomplished with the safety valve leaking. The
safety valve has a pressure control function to keep the RCS from encroaching on the RCS
pressure safety limit. The K/A also requires testing the implications of expansion of liquids as
temperature increases. Therefore, the K/A is met because the RCS temperature goes up,
causing the liquid volume to expand. The safety valve leaks, reducing the pressure in the
pressurizer. Both of these items cause an insurge into the pressurizer, which lowers the
pressurizer temperature (operational implication)
The question author during the question construction focused primarily on the expansion of
liquids as temperature increases aspect required by the KA and related this to the insurge into
the pressurizer associated with the plant power change that had occurred. The applicant was
expected to discern that a power increase causes the RCS temperature to increase resulting in
an insurge into the pressurizer and operationally impacts the pressurizer by decreasing the
temperature. The leaking safety valve was identified as an additional contributor to the insurge.
The pressurizer code safety valves are not traditionally considered a part of the pressurizer
pressure control system. However, the KA manual includes the pressurizer code safety valves
in Section 010, Pressurizer Pressure Control System. Therefore, a malfunction of these valves,
could be considered as a malfunction of the Pressurizer Pressure Control System (PPCS) for
the purposes of selecting test item topics from the KA manual and establishing conditions in a
question.
The stem sets up conditions that result in an insurge into the pressurizer as a result of an RCS
temperature increase caused by Xenon burnout. One reasonable application of the tested KA
would be establishing conditions that resulted in an insurge to the pressurizer resulting from a
malfunction of the PPCS. Although the stem does not use a malfunction of the PPCS to set
conditions, the questions effectively measures whether or not the operators know the
operational implication of the expansion as temperature increases.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
16
Enclosure 2
The facility is correct in that the question stem did not specify the magnitude of the leaking
pressurizer safety valve contribution to the insurge therefore could be considered to have a
minimal effect on the insurge and a knowledgeable applicant would recognize this.
The GFE may test basic expansion and contraction of liquids, but it would stop short of testing
how this knowledge is applied to a specific station event.
2)
Facility comment is from Section II. General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to delete the question since there is insufficient information given in
the question to correctly answer.
Facility comment not accepted. The question will not be deleted, answer choice B will be
graded as the correct answer.
Analysis:
When reactor power is increased as stated in the question, Xenon will burn out, resulting in
increased heat output from the core to compensate for the Xenon. The rod control system will
respond to changes in RCS temperature by moving rods to keep Tavg and Tref within the 3
degree lockup band. Operators will observe an increase in RCS temperature, in that, Tavg will
rise to the top of the lockup band and Thot will increase. Rod movement will eventually
compensate for this temperature increase. In this scenario, the general trend of RCS
temperature will always be upward, which will result in an insurge to the pressurizer.
The steam space leak can only result in an insurge. There are no circumstances under which a
leaking pressurizer safety valve will result in a pressurizer outsurge. If the applicant assumed
that the leak was negligible compared to the insurge, this assumption would not affect the
assessment of the conditions that resulted in the insurge.
Test Question RO #30 (034A1.02): Facility comments on pages 79-95 & 241.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of operators and does not relate to a 10 CFR 55.41 item for RO examination.
Facility comment not accepted. This test item is appropriate for the initial licensing examination
and tests at the appropriate license level.
17
Enclosure 2
Analysis:
The level of 461 feet corresponds to the TS limit of 23 feet of water above the top of the reactor
pressure vessel flange. If water level drops below this level at any time during refueling
operations, the TS requires immediate suspension of fuel movement. Therefore, this
knowledge is important for both the SRO and the RO to have committed to memory. The facility
comment to RO question 9 indicates that operators are required to know nominal operating
parameters of systems. If this is the case, operators would have committed to memory enough
information that would allow recognizing the correct answer to this question. An applicant with
the knowledge of the nominal operating parameters could quickly eliminate two of the
distractors (A and D), in that they contain only level ranges that are inside of the alarm bands
(i.e. not within the nominal range). In fact, 7 of the 9 applicants correctly answered this
question.
The knowledge tested by this question is appropriate for the RO and is tied to 55.41 (b)(10) &
55.41 (b)(12). The following facility Enabling Objectives imply that this knowledge level is
required by an RO:
Enabling objective GS-4-21: STATE the TS limitations for the following items: 1. Refueling
operations. Applicable to RO, SRO, and SE.
Enabling objective GS-4-19: STATE the reason for the following indicators available for the fuel
handling system: 11. Refueling cavity water level indicator. (Applicability not listed in lesson
plan)
2)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
In order to assess an operators ability to operate controls, this questions tests an applicants
knowledge of system parameters that must be monitored to ensure the system is operated
within limits and acted upon if the parameters are outside of limits. In this case the operating
band includes a TS limit. Knowledgeable applicants should have an understanding of system
parameters, knowledge of the indications available, knowledge of the nominal values or
expected operating parameters, and an understanding of the implications of operating the
system outside of normal parameters. This represents a reasonable match to the selected KA.
18
Enclosure 2
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
Test Question RO #31 (035K6.02): Facility comments on page 242.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The malfunction is the opening of the SG PORV due to an instrument failure, which has the
effect of dropping steam generating system pressure until a low steam line pressure SI causes
a unit trip signal. This is a reasonable match for the selected KA.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The facility suggested best match for this KA is:
041K6.05 Knowledge of the effect of a loss or malfunction on the following will have on the
SDS: Sensors and detectors (CFR: 41.7 / 45.7) (IMPORTANCE RO 1.6 SRO 1.7)
or
041A4.06 Ability to manually operate and/or monitor in the control room: Atmospheric relief
valve controllers (CFR: 41.7 / 45.5) (IMPORTANCE RO 2.9 SRO 3.1).
The KAs suggested by the facility do not appear to match as well as the original KA on which
the question was based.
19
Enclosure 2
The importance of this knowledge is also supported by the following KA: 041K3.01 Knowledge
of the effect that a loss or malfunction of the SDS will have on the following: S/G (CFR: 41.7 /
45.6) (IMPORTANCE RO 3.2 SRO 3.3).
The additional K/As are from the same Tier and Group as the original KA and would not affect
the validity of the sample plan.
Test Question RO #33 (039A1.10): Facility comments on page 242.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA, the stem contains subjective wording, and tests at the lower cognitive level.
Facility comment not accepted. This question tests the knowledge required by the KA topic at
the higher cognitive level and satisfies the requirement of Form ES-401-6, Item 9. A change to
the sample plan is not warranted.
Analysis:
There are no controls associated with the Main Steam system that affect the operation of the air
ejector radiation monitor that are important enough to include on the written examination. To
include a reference to MS controls would have been window dressing.
This question tests an applicants ability to determine the difference between a SGTR and a SG
tube leak which develops slowly over time by analyzing the response of main steam system
radiation monitors. The question tests whether that applicants understand that RMA-9 is the
most sensitive indication of a tube leak.
The secondary radiation monitors which are relied upon to detect leaking steam generator
tubes, RM-A9 and RM-19A, are frequently referenced by operators and therefore, easily
recognizable by the names used in the question. This is due to the high sensitivity in the
nuclear industry to detecting and mitigating steam generator tube ruptures. The facility
reviewers did not comment that the applicants needed the noun names in order to differentiate
between the two detectors. Additionally, the applicants could have asked for clarification of this
during the examination, but failed to do so.
The wording that characterized the leak was intentionally subjective. Using specific values in
this question is not necessary to answer the question and could confuse the applicants.
The stem provides the applicants with a hypothetical situation in which a reactor trip occurs so
that their knowledge of the radiation monitor response can be tested. The applicants are not
20
Enclosure 2
tasked with determining whether or not a trip is required, which would be beyond the scope of
the question.
The question tests at the higher cognitive level. The applicants need to know several facts and
analyze these facts in order to correctly answer the question. They need to know what type of
radiation that the monitors measure, how this is affected by the sudden decrease in fission rate
associated with the reactor trip, and how this will be displayed in the control room.
The facility contends that this question tests 073K1.01, Knowledge of the physical connections
and/or cause effect relationships between the PRM system and the following systems: Those
systems served by PRMs. (CFR: 41.2 to 41.9 / 45.7 to 45.8) (IMPORTANCE RO 3.6 SRO 3.9).
The KA suggested as a better match by the facility is from the same Tier and Group as the
original KA and would not affect the validity of the sample plan.
In addition, the only KAs in section 059, Condenser Air Removal System that are above 2.5 are
related to how the system responds to increased radiation.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
Test Question RO #35 (045K5.17): Facility comments on page 242.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA, has no operational implication, and tests knowledge required for the General
Fundamental Exam (GFE).
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question tests the applicants knowledge of how the MTC changes with a decrease in boron
concentration (dilution) that was performed to support a Turbine Generator load increase.
While reactor physics is covered on the GFES, the KA for this question was randomly selected
for inclusion on the examination and is appropriate. The operational validity of this question is
21
Enclosure 2
established by testing the applicants knowledge of the concepts as they would be applied to
normal plant operations. Before performing licensed duties, operators should be aware of how
their routine activities, such as changing RCS boron concentration and increasing or decreasing
turbine load affect reactivity in the core.
While many reactors operate the full cycle with a negative MTC, some reactors can operate with
a slightly positive MTC at the beginning of life. This possibility was eliminated in the question
stem by including the statement about negative MTC to prevent the applicants from needing to
make an assumption or ask a clarifying question. Eliminating the consideration of Xenon and
stating that there was no change in rod position was done to simplify the question. Reactor
operation with rods at the all rods out position is a reasonable set of plant conditions for this
question.
Additionally, 7 of 9 applicants answered this question correctly.
The question is a reasonable match for the KA. The question establishes the operational link (a
boron dilution is required) between increasing turbine load and the relationship to moderator
temperature coefficient (MTC).
Test Question RO #37 (054AK3.01: Facility comments on page 242.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and tests at the lower cognitive level.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The KA requires testing the reason for tripping the reactor in response to a loss of main
feedwater. The question adequately tests this required knowledge.
This question was used on the 2002 V. C. Summer examination and was taken from the V.C.
Summer exam bank. The KA used in 2002 was 029EK3.01 (IMPORTANCE RO 4.2 SRO 4.5).
This KA is from the same Tier, Group, and Category as the current KA. The KA suggested as a
better match by the facility is from the same Tier, Group, and Category also. Therefore, any KA
mismatch does not affect the validity of the sample plan.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
22
Enclosure 2
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The bullets in the question set a point of reference for the applicants. There are several
reasons for tripping the reactor during an event. The reasons are the same for many of the
major plant transients. However, distractors A and D contain reasons for tripping the reactor
that would not apply to a loss of feedwater transient. Therefore, being able to recognize the
distractor that contains the correct reason for tripping the reactor, in response to a loss of
feedwater event, appropriately tests the KA.
The applicant must analyze the indications given in conjunction with determining the reason for
the reactor trip. This involves a higher thought process that requires this question to be labeled
as a higher cognitive level question.
Test Question RO #41 (058AA2.01): Facility comments on pages96-100.
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of operators and does not test the knowledge required by the KA.
Facility comment not accepted. This test item is appropriate for the initial licensing examination,
tests the knowledge required by the KA topic and satisfies the requirement of Form ES-401-6,
Item 9. A change to the sample plan is not warranted.
Analysis:
The facilitys comment states that the knowledge tested in this question is not operationally
significant and is inappropriate for operators to recall from memory. The facility training
program has an enabling objective indicating that knowing this information is appropriate:
Enabling objective GS-2-06: DESCRIBE the normal operation of the following Safeguards
Power System components. INCLUDE component types and applicable setpoints: Item 5.
NSSS inverters (applicable to AO, RO, SRO, and SE)
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
23
Enclosure 2
unreasonable expectation for the well-trained applicant. Moreover, NUREG 1021, Appendix B,
C.1.a supports the appropriateness to develop questions regarding knowledge that is
embedded in or covered by procedures within written test items. The distinction to be made
here is that NRC is not requiring the procedure to be memorized, but rather, is expecting the
well trained applicant to discern knowledge embedded in a procedure, in light of the answer
options already available in our recognition style MC format.
This question tests mastery of the facility enabling objective by testing the applicants
knowledge of the NSSS inverter operation in response to a loss of DC input power. It also tests
the applicants knowledge of the trigger that initiates a swap to the alternate power source.
To receive both of these alarms requires a significant drop in DC voltage, the applicant only
needs to know that a significant drop of DC voltage has occurred in order to correctly answer
the question.
The NUREG 1021, Appendix E, written examination briefing is read to every applicant before
the written examination. It states, in part, When answering a question, do not make
assumptions regarding conditions that are not specified in the question unless they occur as a
consequence of other conditions that are stated in the question. If the applicants follow this
direction they would know that all automatic actions occurred as designed. This is sufficient
information for the applicant to know that an alternate power supply has come on line.
This question is a reasonable, not perfect, match for the KA selected. The question test the
concept given a loss of DC power, name the substitute power source that has come on line.
The applicants ability to verify that an alternate power source has come on line is tested by
whether or not they know the alternate power supply to XIT-5901.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
An additional KA match from the same Tier, Group, and Category which indicates that this
information is appropriate is: KA 058AA2.02, Ability to determine and interpret the following as
they apply to the Loss of DC Power: 125V dc bus voltage, low/critical low, alarm (CFR: 43.5 /
45.13 (also 41.7)) (IMPORTANCE RO 3.3 SRO 3.6). If this KA was used it would have no
effect on the validity of the sample plan.
Seven of the nine applicants answered this question correctly.
24
Enclosure 2
Test Question RO #45 (062A3.02): Facility comments on pages 101-106.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of operators.
Facility comment not accepted. This test item is appropriate for the initial licensing examination.
Analysis:
The facility claims that this question requires knowledge of a detailed application of procedures
that is beyond expectations for recall from memory. To the contrary, this question test the
applicants knowledge of a when an inverter is ready for use. This is indicated by the DC BUS
CHARGED light which indicates that the capacitors in the inverter have fully charged and that
the inverter is ready to accept loading. While the reference for the question is SOP-310, this
question tests applicants knowledge of basic electrical principles (i.e. capacitors do not charge
immediately) and indications available to the operator for proper inverter operation.
It should be noted that NRC test items do not require knowledge of memorized procedures in its
written test item format, but rather, require that operators recognize, or reasonably infer, the
correct answer within the context of the given conditions and among the choices of which the
answer is already stated and available. In fact, the MC format is often referred to as a
recognition or aided recall item type, in which the test taker need only recognize the correct
answer among the alternatives in the distractor set and within the context of the test items stem
-- not an unreasonable expectation for the well-trained applicant.
2)
Facility comment is from Section II. General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to delete the question since there is no correct answer choice given.
Facility comment not accepted. The question will not be deleted, answer choice C will be
graded as the correct answer.
Analysis:
The facility misread the distractor and their comment is inaccurate. SOP-310, Note 2.6 states
that Five to ten seconds should be allowed for the capacitors to fully charge and the battery
charger to stabilize. Illumination of the red indicator lights indicates charged capacitors.
Therefore, the DC BUS CHARGED light must go on within 10 seconds in order for the operators
to be able to check the status of the light after 10 seconds have elapsed. The correct answer to
the question states that the DC BUS CHARGED light must go on within 10 seconds.
All SROs and 2 of the 5 ROs answered this question correctly.
25
Enclosure 2
Test Question RO #46 (062AA1.07): Facility comments on page 243.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and requires memorization of fact.
Facility comment partially accepted. This question tests the knowledge required by the KA topic
and satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The NRC does agree that the question tested at the lower cognitive level and not at
the higher cognitive level as originally indicated.
Analysis:
This question is a reasonable, not perfect, match for the KA selected. The correct answer to the
question is related to a loss of service water. The applicant must be able to recognize that low
flow in the RBCUs is an indication of a loss of service water, but that this is not one of the
automatic start features for the service water pumps.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The KA recommended by the facility is appropriate for the question, as are other KAs, equally
appropriate which are within the same Tier, Group, and Category of the sample plan. These
include: 056AA1.07, Ability to operate and/or monitor the following as they apply to the Loss of
Offsite Power: Service water pump (CFR 41.7 / 45.5 / 45.6) (IMPORTANCE RO 3.2 SRO 3.2);
and 054AA1.01, Ability to operate and/or monitor the following as they apply to the Loss of Main
Feedwater (MFW): AFW controls, including the use of alternate AFW sources (CFR 41.7 / 45.5 /
45.6) (IMPORTANCE RO 4.5 SRO 4.4). The selection of either one of these KAs would have
no effect on the validity of the sample plan.
Test Question RO #49 (064K6.08): Facility comments on page 243.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and that the wording in the stem is not correct (unable to be should be worded as
cannot be).
26
Enclosure 2
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
Use of the wording unable to be is not the best choice of wording for the conditions given.
This nuance was overlooked by both the NRC and facility reviewers in the exam review
process. However, none of the applicants asked for clarification of this portion of the stem
during the administration of the exam. In that 8 of the 9 applicants answered this question
correctly, it does not appear that this affected the applicants understanding of the stem.
Wording in the KA manual does not differentiate between the diesel fuel oil day tank and the
fuel oil storage tank. It should be noted that the KA states fuel oil storage tanks (i.e. more than
one tank).
There are basically two ways to postulate the loss or malfunction of a tank. The tank can be
either removed from the systems or the connection between the tank and the system can be
interrupted. For this question, the loss of the tank was caused by an interruption of flow
between the tank and the diesel generator. From this perspective, the question is an
acceptable match for the KA.
Test Question RO #51 (071K3.01): Facility comments on page 243.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and the noun name for RM-A-10 should have been provided.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The original KA is considered a reasonable match for the following reason: RM-A10 was
considered to be a part of the waste gas system and had failed while monitoring a release. The
malfunction of RM-A10 was thus considered to be a malfunction of the Waste Gas System. The
affect on RM-A10 (the Process Radiation Monitor) is that RM-A3 will be relied upon to provide
monitoring and to insure an automatic release termination.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
27
Enclosure 2
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
Although this question is a repeat question from the V. C. Summer 2002 examination with the
same KA, a better KA match for the question is 071K1.06, Knowledge of the physical
connections and/or cause-effect relationship between the Waste Gas Disposal System and
following system: ARM and PRM systems. (CFR 41.2 to 41.9 / 45.7 to 45.8) (IMPORTANCE RO
3.1 SRO 3.1). This is the same Tier and Group, as the original KA and does not affect the
validity of the sample plan.
The omission of the noun name for RM-A-10 did not result in any of the applicants having to ask
for clarification during the administration of the exam. In that all the applicants answered this
question correctly, it does not appear that this affected the applicants understanding of the
stem.
Test Question RO #53 (073K5.03): Facility comments on page 243.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and that the KA selected is typically thrown out since it is considered trivia.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. This test item is appropriate for the initial licensing examination.
Analysis:
The question is a reasonable match for the KA selected. The failed fuel postulated in the stem
of the question results in a substantial increase in the radiation levels that would be monitored
by the RM-L1 (letdown radiation monitor). This question tests the applicants ability to correlate
the increase in radiation intensity in the RCS process fluid and the potential increase in radiation
exposure to personnel.
The NRC disagrees that the information tested by this question is trivia. All operators should be
aware of plant conditions and events that result in a change in radiation levels and the possible
implications for personnel exposure. This is particularly true when the first fission product
barrier (i.e. fuel cladding) has been breached. Additionally, this KA is not typically thrown out as
the facility suggests. Very similar question were used on examinations given at Salem in 2002
and Prairie Island in 2000 for the same K/A.
28
Enclosure 2
The validity of the question for inclusion on a licensing examination as a test item is further
supported by the additional, reasonable KA match proposed by the facility.
Test Question RO #55 (076A1.02): Facility comments on pages 107-121 & 244.
1)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question is a reasonable match for the KA selected. Knowledge of the implications of
conditions is equivalent to operating the controls. The KA suggested by the facility as a better
match is from the same Tier and Group as the original KA, therefore, would have no effect on
the validity of the sample plan.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
2)
Facility comment is from Section II. General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to delete the question since this question has no correct answer.
Facility comment accepted. The question will be deleted.
Analysis:
There was insufficient information contained in the stem of the question to determine a definitive
answer, therefore, the question will be deleted from the exam.
The applicants were not provided with a time frame in which to assess the effect of the
temperature changes postulated in the question stem.
29
Enclosure 2
Test Question RO #56 (076A3.02): Facility comments on page 244.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and includes implausible conditions in the stem. The facility also contends that this
question overlaps with RO question #46.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The stem is plausible and satisfies the requirement of Form ES-401-6, Item 10. The
similarities between this question and RO question #46 are insufficient to cause overlap
concerns.
Analysis:
The question tests the applicants ability to monitor the automatic operation of the service water
system to supply cooling water flow to the emergency diesel generators under a loss of offsite
power condition. In this question, the emergency diesel generator is the emergency equipment
supported by the service water system. This is a reasonable match to the KA. The KA clearly
addresses monitoring the operation of the service water system and not the operation of the
diesel generator as stated by the facility.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The validity of this question is supported by the facility recommended KAs, which are also
reasonable matches for the question.
This question was used on the V. C. Summer NRC examination administered in 2000 and
added to the V. C. Summer question bank. Additionally, this question was used by V. C.
Summer on an internal audit examination. The same KA was referenced in both cases.
The facility contends the stem contains conditions that are implausible. The stem conditions are
consistent with the conditions that were asked by this question on previous examinations.
While undesirable, procedure violations have occurred at nuclear plants. The implausible
conditions, while unlikely to occur, set the condition for testing the knowledge. The
implausibility of the initial conditions is not relevant to whether the candidate has the knowledge
to answer the question.
30
Enclosure 2
Additionally, the initial conditions provided are specifically addressed in Nuclear Operations
Training document IB-1, Service Water System lesson plan, which states: It is important to note
that if both SW pump A and C are racked up on the train A bus, and neither pump was
running (neither MCB handswitch in after-start), then neither SW pump would start on a LOSP
or a SI signal. In fact, all applicants answered this question correctly.
This question does not overlap with RO question #46. Question #46 tests the knowledge of the
automatic start signals for the service water pumps with a perceived loss of service water. This
question tests knowledge of which service water pumps would or would not start given specific
switch lineups. The service water system is the plants ultimate heat sink. Knowledge of
system operation is important operator knowledge. Both KAs were selected through a random
selection process in accordance with NUREG 1021.
Test Question RO #58 (103K3.02): Facility comments on page 244.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA, is a collection of True/False statements, and tests the same basic knowledge as
SRO question #13 (question #88).
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The question does not meet the criteria for a collection of true/false statements and
satisfies the requirement of Form ES-401-6, Item 10. The similarities between this question and
SRO question #13 (question #88) are insufficient to meet overlap concerns.
Analysis:
The malfunction tested in this question is the failure of the equipment hatch O-rings. The
applicant is asked to recognize that failure of the O-ring has an affect on containment integrity.
This is a reasonable interpretation of a poorly worded KA.
The stem of the question is necessary to test this lower cognitive level item. By the very nature
of a memory level test item, the distractors will be either true in relation to what is asked in the
stem or false. The distractors are not stand-alone true/false items that could be assessed in
the proper context without the stem.
The question is similar to SRO question 13 (question #88) only in that they are both related to
the containment boundary. Question #88 is related to a specific piece of equipment, the status
of that equipment, and operability of the equipment. This question deals with how the loss of a
specific components function will affect containment integrity.
31
Enclosure 2
Test Question RO #62 (G2.2.11): Facility comments on page 244.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and may not be related to expected RO knowledge.
Facility comment not accepted. This question tests the knowledge required by the KA topic at
the appropriate license level and satisfies the requirement of Form ES-401-6, Item 9. A change
to the sample plan is not warranted.
Analysis:
The facilitys interpretation of the KA is a narrow interpretation. The KA is not specific as to
which type of temporary change should be tested and therefore, does not exclude temporary
procedure changes. The original KA is a reasonable match for the question.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The KA recommended by the facility is in the same Tier, Group, and Category as the original
KA, therefore, would have no effect on the validity of the sample plan.
This question was an RO question on the Summer 2002 exam. The facility pre-examination
review validated this question as at the RO level. This knowledge is tied to 55.41(b)(10) as
required RO knowledge.
Test Question RO #63 (G2.2.25): Facility comments on page 244.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required for an RO and only
tests at the lower cognitive level.
Facility comment not accepted. This question is appropriate for RO knowledge and tests at the
higher cognitive level.
Analysis:
32
Enclosure 2
The question tests at the higher cognitive level because it requires applicants to know the
Technical Specification (TS) bases as well as determine the parameters necessary to monitor to
verify that core safety limits are maintained.
Knowledge of core safety limits is fundamental to all levels of licensed plant operation. ROs
monitoring the control boards should know which parameters and indications to monitor to
ensure that safety limits are not exceeded. The TS involved has a 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> action statement. RO
knowledge of 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> TS action statements is an industry-wide expectation.
This question was answered correctly by 5 of 9 applicants. Those applicants who did not
answer the question correctly chose an answer that reflected knowledge of the TS bases, but a
lack of knowledge of the parameters to monitor to ensure compliance with TS 2.1.1, Reactor
Core Safety Limits.
Test Question RO #67 (G2.4.15): Facility comments on page 245.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and the question is too specific for a Generic KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The local actions directed by EOP-6.0 require the use of communications between the local
station and the control room. Important local actions in the implementation of the EOP step
tested (i.e. the rate of depressurization of the steam generator) are dependent on the availability
of communications. This question also tested knowledge of an important EOP note.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
This question could also be tied to other KAs in same the Tier, Group, and Category, such as
G2.4.7, Knowledge of event based EOP mitigation strategies. (CFR: 41.10 / 43.5 / 45.13)
(IMPORTANCE RO 3.1 SRO 4.0); G2.4.12, Knowledge of general operating crew
responsibilities during emergency operations. (CFR: 41.10 / 45.12) (IMPORTANCE RO 3.4
33
Enclosure 2
SRO 3.9); and G2.4.20, Knowledge of operational implications of EOP warnings, cautions, and
notes. (CFR: 41.10 / 45.13) (IMPORTANCE RO 3.3 SRO 4.0).
The applicant knowledge of communication procedures associated with EOP implementation
was tested by giving the applicants a specific situation. The applicants must know that
continuing SG depressurization at the maximum rate is contingent on the availability of
communications between the control room and the local station. Knowledge of the importance
of communications in this one instance provides assurance that the applicants would be aware
of its importance in other situations.
Test Question RO #69 (W/E04EA2.1): Facility comments on pages 122-129 & 245.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of operators and does not relate to a 10 CFR 55.41 item for RO examination.
Facility comment not accepted. This test item is appropriate for the initial licensing examination
and tests at the appropriate license level.
Analysis:
The knowledge tested in this question is appropriate to the RO as indicated by the following two
V.C. Summer lesson plans.
1)
The V.C. Summer lesson plan for EOP-2.5 contains the following enabling objective:
1883, STATE the entry conditions of EOP-2.5, a. symptoms, b. transition. This is
applicable to the RO, SRO, and SE.
2)
The V.C. Summer lesson plan for EOP-1.0 contains the following enabling objective:
1679, SELECT an appropriate transition out of EOP-1.0 given a set of plant conditions.
This is applicable to the RO, SRO, and SE.
Additionally, this item is included in 10 CFR 55.41(b)(10) and is appropriate for the RO
examination.
The facilitys contention that the applicants could assume values for other parameters, such as
containment sump levels, etc. is inconsistent with the direction given to the applicants during the
NUREG 1021, Appendix E, briefing. Appendix E, Item 7 states: When answering a question,
do not make assumptions regarding conditions that are not specified in the question unless they
occur as a consequence of other conditions that are stated in the question. The question stem
contains sufficient information for a knowledgeable applicant to determine that the leak
postulated is outside of the Reactor Containment Building.
34
Enclosure 2
To answer this question, the applicants must have a general familiarity with EOPs and the plant
conditions that would result in EOP implementation. The applicant then needs to merely
recognize the correct answer as given in question. It does not require memorization of the
exact steps.
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
2)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required for an RO.
Facility comment not accepted. This question is appropriate for knowledge expected of the RO
license level.
Analysis:
The knowledge tested in this question is appropriate to the RO as indicated by the following two
V.C. Summer lesson plans.
1)
The V.C. Summer lesson plan for EOP-2.5 contains the following enabling objective:
1883, STATE the entry conditions of EOP-2.5, a. symptoms, b. transition. This is
applicable to the RO, SRO, and SE.
2)
The V.C. Summer lesson plan for EOP-1.0 contains the following enabling objective:
1679, SELECT an appropriate transition out of EOP-1.0 given a set of plant conditions.
This is applicable to the RO, SRO, and SE.
Additionally, this item is included in 10 CFR 55.41(b)(10) and is appropriate for the RO
examination.
All 5 RO applicants and 2 SRO applicants answered this question correctly.
Test Question RO #72 (W/E10EK2.2): Facility comments on page 245.
35
Enclosure 2
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and the question uses confusing language.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The validity of the question was not affected by the wording used in the stem and
satisfies the requirement of Form ES-401-6, item 10.
Analysis:
The facility heat removal system consists of both the primary heat removal system (RCS -
primary coolant) and the secondary heat removal system (SGS). Ideally, these systems are
coupled to remove heat from the reactor core. When there is a void in the reactor vessel
head, the RCS and SGS systems can become uncoupled and adequate heat removal may not
occur.
This question tests the applicant knowledge of how to maintain the primary coolant system in a
condition that would support heat removal by natural circulation with a void in the vessel head.
This is a condition that must be taken into account when performing a natural circulation
cooldown with a steam void in the reactor vessel as evidenced by the note in EOP-1.4, Natural
Circulation Cooldown With Steam Void in Vessel. Therefore, it is related to the heat removal
process under these conditions as delineated in the KA.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The KA recommended by the facility, W/E10EK3.1, Knowledge of the reasons for the following
responses as they apply to the (Natural Circulation with Steam Void in Vessel with/without
RVLIS) Facility operating characteristics during transient conditions, including coolant chemistry
and the effects of temperature, pressure, and reactivity changes and operating limitations and
reasons for these operating characteristics. (CFR: 41.5 / 41.10, 45.6 / 45.13) (IMPORTANCE
RO 3.4 SRO 3.7), is in the same Tier and Group as the original KA and further supports this
question as a valid exam question.
The wording of the question did not seem to cause any confusion during the test. None of the
applicants asked for clarification of the stem during the administration of the exam. In that 8 of
the 9 applicants answered this question correctly, it does not appear that the wording affected
the clarity of the stem.
36
Enclosure 2
Test Question RO #73 (W/E12EK2.1): Facility comments on page 245.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and only tests at the lower cognitive level.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The question tests at the higher cognitive level.
Analysis:
Although the KA for W/E12EK2.2 was included with the question in the supporting information,
the question was written to, and is a reasonable match for, W/E12EK2.1: Knowledge of the
interrelations between the (Uncontrolled Depressurization of all Steam Generators) and the
following: Components, and functions of control and safety systems, including instrumentation,
signals, interlocks, failure modes, and automatic and manual features. This is transparent to
the applicant and does not affect the overall validity of the question.
The comment provided by the facility included an incomplete version of W/E12EK2.1. The
facility version of the KA omitted the reference to instrumentation, signals, and failure modes.
This question addresses the instrumentation used (or parameters monitored) when safety
injection flow needs to be reduced during an uncontrolled depressurization of all steam
generators. Additionally, this question meets the KA in that it tests an applicants knowledge of
an uncontrolled depressurization of all steam generators and the control of the safety systems
used to mitigate the event.
This question tests at the higher cognitive level since the applicant must evaluate the
parameters listed in each distractor for applicability. The applicants can decipher the correct
answer through the understanding that during a depressurization of all steam generators the
establishment of a secondary heat sink is not a prerequisite to SI flow reduction.
Test Question RO #74 (W/E13EK2.2): Facility comments on page 246.
37
Enclosure 2
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and only tests at the lower cognitive level.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The question tests at the higher cognitive level.
Analysis:
The facility states that this question requires memorization of a sequence of events from a
yellow path EOP. However, this question can be answered with knowledge of the general flow
path of EOP-15.1 or by analyzing the given conditions using adequate system knowledge. The
applicants could eliminate distractors A and B if they recognized that a loss of offsite power
prevented dumping steam to the condenser. Distractors A and C could be eliminated if they
recognized that the steam to the turbine driven emergency feedwater pump cannot be supplied
from steam generator A. One applicant selected distractor A, three applicants selected
distractor B, and two applicants selected distractor C.
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
This question is a reasonable match for the KA in that it tests the applicants knowledge of the
facilitys heat removal system as it applies to a steam generator overpressure condition.
With respect to the above, the KA recommended by the facility does not appear to be a better
match for the question.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
Test Question RO #75 (W/E15EK3.3): Facility comments on pages 130-138.
38
Enclosure 2
Facility comment is from Section II. General Concepts, C. Multiple or No Correct Answers.
Facility recommendation is to accept choices B and D as correct answers.
Facility comment not accepted. Answer choice B will be graded as the correct answer.
Analysis:
The correct action in response to a rupture in the RBCU, to secure and isolate the A SW
booster pump, is performed to prevent flooding of vital equipment inside the Reactor Building.
The fact that Reactor Building integrity is maintained by this action is a consequence of isolating
the A SW booster pump however, it is not the reason for performing the isolation.
The information provide in the facility comment concerning fission products being introduced
into the SW system is addressed as a result of the SW booster pumps not operating. In the
situation posed by the question, the SW booster pumps are operating.
The Instructor Lesson Plan for EOP-17.1 states that the basis for operator actions in Step 1 of
EOP-17.1 is to evaluate all potential sources of excess water in the RB before critical
components are flooded. The lesson plan also states that for a major leak or break in other
piping within the RB, isolation is essential to maintain water level below design basis level. The
information provided in the EOP-17.1 lesson plan does not state that isolation must take place
to prevent a fission product release path. Provided the SW booster pump is operating and there
is flow emitting from the ruptured RBCU into RB, then there is little likelihood that fission
products would be introduced into the SW system.
39
Enclosure 2
Senior Reactor Operator Examination:
Test Question SRO #77 (003A2.03): Facility comments on pages 139-151 & 232.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of SROs to know from memory.
This comment was not evaluated since the question is being deleted due to having no correct
answer (see below).
2)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
This comment was not evaluated since the question is being deleted due to having no correct
answer (see below).
3)
Facility comment is from Section II. General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to delete the question since this question has no correct answer.
Facility comment accepted. The question will be deleted.
Analysis:
The correct sequence of operator actions in accordance with AOP-117.1 is not contained in any
of the distractors. Therefore, there is no correct answer and the question will be deleted.
The question was originally constructed and proposed by the NRC to contain only a high level
list of the actions taken in the AOP without regard to sequence. The V. C. Summer exam
reviewers requested, and the NRC agreed to, the addition of the sequence requirement in the
stem and the inclusion of more specific sequence-related information in the distractors.
The facilitys post-examination comment that operators do not need to identify a correct AOP
mitigation sequence is inconsistent with the position taken during the pre-examination review
and validation.
40
Enclosure 2
Test Question SRO #80 (007EA2.01): Facility comments on pages 152-171.
1)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question is a reasonable match for the knowledge required by the KA. The question
provides the applicant with an available reactor power indication that is decreasing in response
to a reactor trip. The applicant must evaluate this available power indication as unusually high
and interpret that it indicates the reactor may not be tripped and then determine the procedure
path to mitigate the event.
2)
Facility comment is from Section II. General Concepts, C. Multiple or No Correct
Answers.
Facility recommendation is to accept choices C and D as correct answers.
Facility comment accepted. The answer key will be changed to accept answers C and D as
correct answers.
Analysis:
The facility comment fails to address the reason that the correct answer is distractor D.
Steps 1 - 5 of EOP-1 are immediate action steps. As such, it is expected that every operator
know these from memory. The immediate action steps and substeps are performed in
sequence. If one of the substeps cannot be completed, a transition to the Alternative Action
should be performed.
The stem of the question provides two conditions which require transition to EOP-13. EOP-1.0,
Step 1, substep 2 states Verify all [emphasis in original document] Reactor Trip and Bypass
Breakers are open. The question stem states that reactor trip breaker A failed to open. Step 1,
substep 4 states Verify Reactor Power level is decreasing. The abnormally high reactor power
level of 7% immediately after the trip, although decreasing, indicates that the reactor should not
be considered subcritical. Either of these two abnormal conditions is sufficient for a
knowledgeable applicant to recognize that transition to EOP-13.0 from step 1 of EOP-1.0 is the
correct response.
41
Enclosure 2
Test Question SRO #83 (022AG2.4.49): Facility comments on page 232.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and does not test the SRO knowledge required by 10 CFR 55.43(b).
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The question is related to 10 CFR 55.43(b) and satisfies the requirement of Form
ES-401-6, Item 3.
Analysis:
The facility is correct in that the technical validity of the actions tested by the question are
supported by the alternative actions of AOP-102.2, Loss of Charging. However, this question
only requires the SRO to have a general knowledge of the mitigation strategy for a cavitating
charging pump and to be able to recognize the general actions required to prevent damage to
the pump in a cavitation event. In some cases, prompt and prudent operator action must be
taken, without reference to procedures, in order to prevent damage to a cavitating pump. The
SRO should be aware of these actions to direct the proper actions of the crew. For the V. C.
Summer Station, the charging pumps are also the high head safety injection pumps. In some
cases, the cavitation event could result in a common mode failure of both Safety Injection
pumps, thereby necessitating that these actions be taken without reference to a procedure.
The KA does not limit the topic to determining the response to the failure of an automatic
function. While it is desirable to achieve as close a fit as possible to the KA topic, the
interpretation that the test item does not match, does not invalidate the test item. Since any
number of randomly selected safety-important KAs might have been selected and a subsequent
test item developed from it, the applicant would be, nonetheless, expected to answer the
question. From the facility or applicants perspective, KA/test item match is irrelevant toward
being expected to answer the question. Had a different KA been originally selected that
corresponded to the subject test item, then the applicant would have been expected to answer
it, regardless.
The KA tie to 10 CFR 55.43(b)(2) does not necessitate that the question be written such that it
matches both the KA and the specific 10 CFR 55.43 item as assumed by the facility. This
question adequately covers 10 CFR 55.43(b)(5).
The facility failed to identify this as inappropriate for an SRO during the pre-exam review and
validation. No supporting reference material was provided by the facility to support the
assertion that the information tested in this question is not appropriate in accordance with their
Conduct of Operations guidance.
All four SRO applicants answered this question correctly.
42
Enclosure 2
Test Question SRO #84 (032AA2.08): Facility comments on pages 172-177.
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of SROs.
Facility comment not accepted. This test item is appropriate for the initial licensing examination,
tests at the appropriate license level and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
Generic KA G2.2.12, Knowledge of Surveillance Procedures. (CFR: 41.10 / 45.13)
(IMPORTANCE RO 3.0 SRO 3.4) covers the topic of this question and further supports the
validity of including the topic on the SRO portion of the exam. Additionally, the KA selected that
resulted in this question, indicates the testing required as the result of a loss of power to the
Source Range monitor is important.
The facility training program contains the following Enabling Objectives supporting this
knowledge:
1)
AOP-401.9 Enabling Objective 2398, APPLY AOP-401.9 by predicting a discrete path
through the AOP, given a set of plant conditions.
2)
AOP-401-9 Enabling Objective 2399, PLAN an appropriate response to given conditions
that are not directly addressed in AOP-401.9.
This question covers the facility Enabling Objective in that each distractor plans a course of
action associated with a loss of power to the source range monitor. While the technical validity
of the information tested by the question is, in part, contained in the Technical Specifications, an
SRO need only recognize the appropriate course of action to answer this question. Additionally,
the correct answer to this question can be determined with knowledge of the general
requirement to perform a channel operational test on a monitor that had lost power. This is not
beyond the level of knowledge appropriate for an SRO.
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
Three of four SRO applicants answered this question correctly.
43
Enclosure 2
Test Question SRO #85 (035G2.4.20): Facility comments on page 233.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the SRO knowledge required by 10 CFR 55.43(b).
Facility comment not accepted. This question tests the knowledge required by the KA topic, is
related to 10 CFR 55.43(b) and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
The question can be answered correctly if the applicant can recognize the steam generator
conditions which require initiation of bleed and feed. The KA importance of this topic for an
SRO is 4.0 thereby, recognizing this as an important topic. This topic is related to 10 CFR 55.43(b)(5). The question distractors do not require the selection of a procedure by name.
However, recognition of the appropriate mitigation strategy is the same as selecting a
procedure. Additionally, this item is tied to a caution in EOP-15.0, Response to Secondary Heat
Sink, which is read by only the SRO during accident mitigation.
The facility training program contains the following Enabling Objective supporting this
knowledge:
EOP-15.0 Enabling Objective 2096, RELATE each CAUTION and NOTE in EOP-15.0 to
its corresponding step by identifying the bases or reason for the CAUTION or NOTE.
This should include, but not limited to, the following: Item #2. Importance of timely bleed
and feed initiation.
Two of four SRO applicants answered this question correctly.
Test Question SRO #86 (054AA2.03): Facility comments on page 233.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the SRO knowledge required by 10 CFR 55.43(b).
Facility comment not accepted. This question tests the knowledge required by the KA topic, is
related to 10 CFR 55.43(b) and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
44
Enclosure 2
In order to select a correct procedure, an SRO must understand plant conditions based on
indications and perform elementary diagnostics to recognize the initiating event. The question
requires some system knowledge in order to answer correctly. However, diagnosis of the
initiating event is SRO level knowledge. This topic is related to 10 CFR 55.43(b)(5). The
question distractors do not require the selection of a procedure by name. However, recognition
of the appropriate event is instrumental in determining the mitigation strategy which is the same
as selecting a procedure. The low level of difficulty may make it seem like an RO question.
This question was assigned a Level of Difficulty of 2" by the NRC. All four SRO applicants
answered this question correctly.
Test Question SRO #87 (068G2.1.20): Facility comments on pages 178-185.
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since the question test knowledge beyond that
required of SROs.
Facility comment partially accepted. This test item is appropriate for the initial licensing
examination and will not be deleted from the exam. However, the answer key will reflect answer
choices A and D as correct answers.
Analysis:
Facility Health Physics procedure, HPP-710, Sampling and Release of Radioactive Liquid
Effluents, supports distractor D as an additional correct answer. HPP-710, Step 5.3.3 indicates
that the liquid radwaste release permit be terminated if the release is terminated a second time
by a radiation monitor high alarm. This additional answer was not identified by the facility during
the examination review and validation activities.
The facility suggests that there is not a completely correct answer to this question since
distractor A does not contain a reference to a redundant analysis. However, the distractor
states that The tank must be sampled and activity levels verified, which is the equivalent of a
redundant analysis.
This question does not require knowledge of specific supplemental actions in a procedure as
the facility states. The occurrence of multiple radiological spikes during a release is considered
unusual. The applicants should know multiple, automatic terminations of a radiological release
necessitate a reevaluation of the radiological conditions prior to reinitiating the release.
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
45
Enclosure 2
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
Facility Enabling Objective GS-9-16, EXPLAIN the importance of the following items when
performing a gaseous or liquid waste discharge: Notification to count room of alarm on
discharge Rad monitor, supports the knowledge tested by this question. The information
provided by HPP-710, step 5.3.3 and SOP-108, step 5.c indicates that the count room must be
notified the second time a radiation monitor spike terminates a release.
This is SRO only in that it is tied to 10 CFR 55.43(b)(4) and decisions related to radiological
releases are made by SROs.
Three of four SRO applicants answered this question correctly.
Test Question SRO #88 (103G2.1.30): Facility comments on page 234.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
The question matches the KA in that, the stem provides the applicants with indication of the
operation and status of the personnel escape airlock. The applicants are tested on whether
they can interpret the local indications given and determine the operability status of the
containment system.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The KA recommended by the facility as a better match for the question is in the same Tier,
Group, and Category as the original KA, therefore, would have no effect on the validity of the
sample plan. The importance value for the KA recommended by the facility is 4.0, which
46
Enclosure 2
supports the importance of this topic. Additionally, this item is SRO only in that the assessment
of operability is related to 10 CFR 55.43(b)(1).
Two of four SRO applicants answered this question correctly.
Test Question SRO #89 (G2.1.13): Facility comments on page 234.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the SRO knowledge required by 10 CFR 55.43(b).
Facility comment not accepted. This question tests the knowledge required by the KA topic, is
related to 10 CFR 55.43(b) and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
This question tests if the SRO applicants understand their responsibility associated with the
conduct of operations as it relates to controlling personnel access within the protected area. It
is reasonable to expect that an SRO applicant would be able to recognize their specific
responsibilities when presented to them.
SAP-200, Conduct of Operations, states that the duty shift supervisor has the responsibility and
authority to have any person removed from site when conditions warrant. Specifically, inside
the protected area, the shift supervisor has the final authority. The facility contention that this is
not an SRO level test item appears to be inconsistent with their internal procedures.
One of four SRO applicants answered this question correctly.
Test Question SRO #91 (G2.2.20): Facility comments on page 234.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the SRO knowledge required by 10 CFR 55.43(b) and contains reverse logic.
Facility comment not accepted. This question tests the knowledge required by the KA topic, is
related to 10 CFR 55.43(b) and satisfies the requirement of Form ES-401-6, Item 3. The validity
of the question was not affected by the wording used in the stem and satisfies the requirement
of Form ES-401-6, item 10.
47
Enclosure 2
Analysis:
The facility is interpreting the statements in 10 CFR 55.43 as if they are KA items. Each test
item does not have to test the 10 CFR 55.43 item in its entirety. The KAs can be tied to 10 CFR 55.41/43 items and cover only a portion of the item.
The question tests the SRO applicants knowledge of troubleshooting activities for equipment
covered by the Technical Specifications. This is a reasonable match for the selected KA.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
This question did not appear to cause the confusion that is usually associated with a reverse
logic question in that no clarification was required during the examination.
All four SRO applicants answered the question correctly.
The facility comment states that this question is tied to 10 CFR 55.41(b)(10) and that it is not an
SRO only question. To the contrary, this item is tied to 10 CFR 55.43(b)(5) and is appropriate
SRO only knowledge.
Test Question SRO #92 (G2.4.7): Facility comments on pages 186-195.
Facility comments is from Section II. General Concepts, C. Multiple or No Correct Answers.
Facility recommendation is to delete the question since there is no correct answer choice given.
Facility comment not accepted. The question will not be deleted, answer choice D will be
graded as the correct answer.
Analysis:
Distractor D states: A review indicates that a full safety evaluation is required per 10 CFR 50.59." This means that one of the questions from SAP-107, Attachment III, was answered
yes. If SAP-107, Attachment III, 50.59 Evaluation, was performed resulting in all No
answers, then a full safety 50.59 safety evaluation would not be required. This case is not
addressed in the distractors. Note 6.3.6 of SAP-148, Temporary Bypass, Jumper and Lifted
Lead control states: A YES answer to any question of SAP-107, Attachment III, 50.59
evaluation, requires PSRC and NSRC review prior to implementation. The 50.59 evaluation
referred to above is a screening instrument to determine if a full safety evaluation is required to
48
Enclosure 2
meet the requirements of 50.59. As such, SAP-148, Attachment I,Section IV, 10 CFR 50.59
Review, provides guidance that a PSRC and NSRC review shall be required prior to
implementation and that a full safety evaluation IS required.
This question was taken from the V.C. Summer exam bank and was used on the V.C. Summer
NRC examination administered in 2000.
Three of four SRO applicants answered this question correctly.
Test Question SRO #93 (G2.3.2): Facility comments on pages 196-207 & 235.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of SROs to recall from memory.
Facility comment not accepted. This test item is appropriate for the initial licensing examination
and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
While the technical content of this question can be found in Health Physics procedure, HPP-
709, the applicants do not need to know this information to answer the question. If the
applicants do not know the health physics requirements of the plant, they could easily answer
the question with a general knowledge of plant equipment and the relative locations of the
waste gas release points and auxiliary building ventilation intake. This question tests the
applicants knowledge of the interface between a waste gas release and the auxiliary building
ventilation.
This knowledge is supported by the following facility Enabling Objectives:
1)
GS-9-04, DESCRIBE the radiation monitoring system interfaces with the following
systems and/or components: Auxiliary building ventilation.
2)
GS-9-11, STATE the plant locations for the components of the radiation monitoring
system.
3)
AB-18-09, STATE the plant location of the following components of the Auxiliary and
Fuel Handling Building Ventilation System Components: a. Auxiliary Building Supply
Plenums.
The distractors in this question were changed at the facilitys request during the pre-examination
review and validation to make a clearer distinction between the distractors with respect to the
wind direction. This made it easier for the applicants to eliminate distractors based solely on
knowledge of the geographical location of plant equipment.
49
Enclosure 2
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
2)
Facility comment from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and is a collection of True/False statements.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted. The question does not meet the criteria for a collection of true/false statements and
satisfies the requirement of Form ES-401-6, Item 10.
Analysis:
This question matches the KA in that if a release is made under adverse wind conditions,
personnel in the auxiliary building could be unnecessarily exposed to the released radioactive
gas. Unnecessarily drawing radioactive waste gas into auxiliary building is not consistent with
ALARA principles.
While it is desirable to achieve as close a fit as possible to the KA topic, the interpretation that
the test item does not match, does not invalidate the test item. Since any number of randomly
selected safety-important KAs might have been selected and a subsequent test item developed
from it, the applicant would be, nonetheless, expected to answer the question. From the facility
or applicants perspective, KA/test item match is irrelevant toward being expected to answer the
question. Had a different KA been originally selected that corresponded to the subject test item,
then the applicant would have been expected to answer it, regardless.
The KA recommended by the facility is in the same Tier, Group, and Category as the original KA
and has a higher SRO importance factor. This is additional support for the validity of the
question.
The facility comment that this question is inappropriate because it is a series of True/False
statements is not consistent with the guidance in NUREG 1021, Appendix B, Step C.2.c.
50
Enclosure 2
Test Question SRO #94 (G2.4.33): Facility comments on pages 208-218 & 235.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of SROs to recall from memory.
Facility comment not accepted. This test item is appropriate for the initial licensing examination,
tests at the appropriate level and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
The question tests applicant knowledge of what is required to extend a Bypass Approval
Authorization beyond the 96 hour0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> point. Only the Operations Manager can direct (i.e. approve)
the method of inactivation past 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />. The facility comment discusses the multiple approvals
that would be required after the 96 hour0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br /> approval has been granted by the Operations Manager.
This information is irrelevant to and not tested by the question.
The technical content of this question can be found in OAP-100.5. It is reasonable for an
applicant to recognize which level of approval is necessary when extending the time that an
annunciator is out of service for greater than 96 hours0.00111 days <br />0.0267 hours <br />1.587302e-4 weeks <br />3.6528e-5 months <br />.
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
This question was taken from the V. C. Summer examination bank. Changes recommended by
the facility were incorporated during the pre-examination review and validation. The additional
information provided by the facility in the post-examination comment was not available to the
examination author.
2)
Facility comment is from Section II. General Concepts, B . Consistency of Examination
Structure.
Facility comment is that this question does not test the SRO knowledge required by 10 CFR 55.43(b).
Facility comment not accepted. This question tests the knowledge required by the KA topic, is
related to 10 CFR 55.43(b) and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
51
Enclosure 2
The facility is interpreting the statements in 10 CFR 55.43 as if they are KA items. Each test
item does not have to test the 10 CFR 55.43 item in its entirety. The KAs can be tied to 10 CFR 55.41/43 items and cover only a portion of the item. This question is valid for an SRO applicant,
in that, it is the SRO function within the control room team to deal with this type of administrative
issue.
Two of four SRO applicants answered this question correctly.
Test Question SRO #96 (W/E02EG2.4.6): Facility comments on pages 219-246 & 235.
1)
Facility comment is from Section II. General Concepts, A. Level of Knowledge.
Facility recommendation is to delete the question since this question test knowledge beyond
that required of SROs to recall from memory.
Facility comment not accepted. This test item is appropriate for the initial licensing examination,
tests at the appropriate level and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
The applicants can answer this question if they understand that SI termination is in progress.
There is sufficient information in the stem for a candidate with knowledge of the general EOP
mitigation strategy to make this determination. The stem also includes indications that an SI is
no longer required.
The ability to recognize that establishing normal charging flow after a safety injection is an
indication that safety injection flow is being terminated is basic to an operators knowledge of
the EOP mitigation strategy.
NRC test items do not require knowledge of memorized procedures in its written test item
format, but rather, require that operators recognize, or reasonably infer, the correct answer
within the context of the given conditions and among the choices of which the answer is already
stated and available. In fact, the MC format is often referred to as a recognition or aided recall
item type, in which the test taker need only recognize the correct answer among the
alternatives in the distractor set and within the context of the test items stem -- not an
unreasonable expectation for the well-trained applicant.
The ability to recognize plant conditions and EOP foldout page criteria is a reasonable
expectation for SRO applicants.
The following two facility Enabling Objectives for EOP-3.1 support the knowledge tested in this
question as a valid topic:
52
Enclosure 2
1911, RELATE any systems/components operation, indication, or malfunction to its effect on
1912, SELECT an appropriate transition out of EOP-3.1 given a set of plant conditions.
2)
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA and is a collection of True/False statements.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample is not warranted.
The question does not meet the criteria for a collection of true/false statements and satisfies the
requirement of Form ES-401-6, Item 10.
Analysis:
The question tests the applicants knowledge of the EOP implementation strategy associate
with the given set of plant conditions. The stem contains enough information for a candidate
with adequate knowledge of the EOP mitigation strategy to correctly answer the question. This
required knowledge is supported by the high importance factor of KA, W/E02EG2.4.6, SI
Termination - Knowledge symptom based EOP mitigation strategies. (CFR: 41.10 / 43.5 /
45.13) (IMPORTANCE RO 3.1 SRO 4.0)
The stem of the question is necessary to test this higher cognitive level item. The distractors
are not stand-alone true/false items related to many separate issues that could be assessed in
the proper context without the stem. See Section C.2.d of Appendix B to NUREG 1021, Rev. 9,
for an explanation of presenting a collection of true/false statements as a multiple choice item.
The applicant must analyze the conditions in the stem and identify that SI flow is no longer
needed as evidenced by the recovering conditions and then identify the correct mitigating
strategy while recognizing the constraints required by the EOP-3.1 reference page.
Test Question SRO #98 (W/E09EA2.2): Facility comments on page 236.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the SRO knowledge required by 10 CFR 55.43(b).
Facility comment not accepted. This question tests the knowledge required by the KA topic, is
related to 10 CFR 55.43(b) and satisfies the requirement of Form ES-401-6, Item 3.
Analysis:
53
Enclosure 2
This question tests an applicants ability to assess a set of plant conditions and recognize the
appropriate mitigation strategy for a natural circulation cooldown under the given conditions.
This test item is appropriate as an SRO only question because it is related to 10 CFR 55.43(b)(5) and the KA SRO importance value is 3.8. Although the question did not ask the
applicants to select a procedure by name, they were tasked with identifying the mitigation
strategy provided in a specific procedure, which is the equivalent.
This question was taken from the facility question bank and modified for use on this
examination. The facility bank considers this question as an SRO only question and it was used
as such on the NRC examination administered in 2000 and on a facility SRO audit examination,
SRO-01-01 Audit exam (#73).
The facility did not comment on the appropriateness of this question as an SRO only question
during the pre-examination review and validation.
Three of four SRO applicants answered this question correctly.
Test Question SRO #99 (W/E12EG2.4.4): Facility comments on page 236.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
This question provides the applicants with a set of conditions that provide clear indication that
an uncontrolled steam generator depressurization is occurring that affects all steam generators
during an SGTR event and asks the applicant to select the correct emergency operating
procedure to enter. This clearly matches the KA, which states: Uncontrolled Depressurization
of all Steam Generators: Ability to recognize abnormal indications for system operating
parameters which are entry-level conditions for emergency and abnormal operating procedures.
(CFR 41.10 / 43.2 / 45.6) IMPORTANCE RO 4.0 SRO 4.3.
Test Question SRO #100 (W/E13EG2.2.25): Facility comments on page 237.
Facility comment is from Section II. General Concepts, B. Consistency of Examination
Structure.
54
Enclosure 2
Facility comment is that this question does not test the knowledge required by the randomly
sampled KA.
Facility comment not accepted. This question tests the knowledge required by the KA topic and
satisfies the requirement of Form ES-401-6, Item 9. A change to the sample plan is not
warranted.
Analysis:
While this question may not be a perfect match of the KA, it is a reasonable match. The KA
clearly addresses the generic aspects of the selected K/A (G2.2.25), knowledge of bases in
technical specifications for limiting conditions for operations and safety limits. The tie to the
W/E13, Steam Generator over-pressure, is reasonable in that the question relates to inoperable
code safety valves which are related to the prevention of steam generator over-pressure
conditions. The basis for the Technical Specification tested is to ensure that sufficient relieving
capacity is available to limit secondary system pressure to within 110% of design pressure;
therefore, a failure to perform the required TS actions could result in a steam generator over-
pressure condition.