ML17213A135

2017 Point Beach Nuclear Plant Initial License Examination Form ES-401-9

ML17213A135
Person / Time
Site:	Point Beach
Issue date:	05/23/2017
From:	Jesse Seymour Operations Branch III
To:	Point Beach
Shared Package
ML16319A340	List: ML17205A232 ML17205A239 ML17205A285 ML17205A290 ML17213A130 ML17213A135 ML17213A140 ML17213A216
References
Download: ML17213A135 (11)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only (Bank: Millstone 2009 Exam)

NRC: Asks for information normally received. Instead, give power 1* H 3 X B ES level and ask for combination that would cause direct reactor trip.

Bullets 2-4 in stem unnecessary.

Facility Response: change incorporated.

2* H 3 N S NRC: Place the word ONLY at the end of distractor A for clarity.

3* H 3 N ES Distractor B should say steps instead of step.

Facility Response: change incorporated.

(Bank: Prairie Island 2010 Exam) The exam which this question originated from associated this question with a K/A for LOCAs 4* H 2 X B U outside of containment, which is what this question focuses on. It does not address the LBLOCA aspect of the current K/A.

Facility: K/A reselected, question replaced.

4R H 3 N S 5* H 3 B S (Bank: Millstone 2009 Exam)

(Bank: Diablo Canyon 2012 Exam)

NRC: For answer C, wouldnt charging flow go to 0 (vs.

6* H 2 B ES lowering)? Better clarity if formatted as a 2x2 question. Distract analysis B doesnt match distractor.

Facility Response: change incorporated.

(Modified: PBNP 2005 Exam) Distractors C and D value of 1800-2000 gpm has no basis in SEP-1. 1550 gpm appears in SEP-1 and may be a more plausible alternative. To avoid 7* F 2 M ES confusion between the actions of steps 19 and 20, specify that sweeping air from the RHR system has been completed.

Facility Response: change incorporated.

8* H 2 N S 9* H 2 B S (Bank: DC Cook 2012 Exam) 10* F 3 N S (Bank: PBNP 2011 Exam)

NRC: Revise stem to have supervisor suggest starting A RCP versus starting RCP in the ruptured loop. Also, ask Which of the following would be the correct response, and basis, for the existing plant conditions? Specify that the first RCP is being started. Part 11* H 2 B ES two of answer D should begin with starting RCP A may since the stem will indicate that it is both the first RCP and is also in the ruptured loop. Distractor analysis for A contains typo (word the misspelled).

Facility Response: changes incorporated.

Note: asterisk symbol next to question number indicates that question was part of the initial 30 question review.

Note: all questions received independent verification of indicated references and K/A conformance review.

1

Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)

3. Check the appropriate box if a psychometric flaw is identified:

• Stem Focus: The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• Cues: The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).

• T/F: The answer choices are a collection of unrelated true/false statements.

• Cred. Dist: The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• Partial: One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• Job Link: The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• Minutia: The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• # / Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• Backward: The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable)

6. Enter questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).

2

ES-401 Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only (Bank/Last 2: PBNP 2015 Exam)

NRC: Original version of the question simply said restore to an 12* F 3 B ES acceptable alignment and omitted mention of Tech Specs.

Unclear why per Tech Specs was added.

Facility Response: change incorporated.

NRC: Stem should specify that RCPs are in operation. For clarity, format stem conditions into the format of given the following with a 13* F 4 N ES bulleted list of conditions.

Facility Response: change incorporated.

14* F 2 B S (Bank: PBNP 2005 Exam)

(Bank: PBNP 2003 Exam)

NRC: The question involves actions of EOP-1.4 step 37; this should likely be the stem condition given (versus having 15* H 3 B ES transitioned back to EOP 1.2). Second half of answer and distractor choices appears unnecessary and should be omitted.

Justification section incorrectly refers to A RHR Pump vs. B.

Facility Response: changes incorporated.

(Bank: Farley 2010 Exam)

NRC: Reword stem to The following annunciators have just been 16* H 4 B ES received in order to clarify that RWST level should still be well above 10% at this time.

Facility Response: changes incorporated.

(Modified: Ginna 2010 Exam)

NRC: Distractor D appears to be a second correct answer; it is similar to the actions of CSP-H.1 step 7. Question does not meet criteria to be considered modified; should be listed as Bank. ES-401 D.2.f requires modifying a pertinent condition in the stem.

17* F 3 X X MB US Format of answer would be improved if it included both PORVs opened if in order to better mirror distractor B. Typo in justification section (it is is reversed).

Facility Response: question revised.

NRC: revised question satisfactory.

(Bank/Last 2: PBNP 2015 Exam) 18* H 2 B S NRC: Question appears to have proprietary reference not submitted separately (Figure 3).

NRC: Distractor A is implausible; no interrelationship exists between P-6 and rod drop circuitry. Question does not meet criteria to be considered modified; should be listed as Bank. ES-401 D.2.f requires modifying a pertinent condition in the stem. No changes in the stem conditions are substantial enough to make a 19* H 2 X MB ES difference in analyzing the stem.

Facility Response: facility believes that original version is plausible based upon misconceptions associated with P interlocks and rod control. Other change incorporated.

NRC: retain original version of distractor A.

20* H 3 B S (Bank: Ginna 2010 Exam)

NRC: LOK is Higher vs. the listed Fundamental level. Question requires evaluating a graphic using memory information from a legend.

21* F 2 N ES Facility Response: facility believes question is at fundamental level due to being very close to just being a list of colors.

NRC: agree with fundamental LOK.

3

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only NRC: Question does not test the specific bases for EOPs of the K/A; it only tests recalling which step is procedurally valid.

22* F 3 X N U Question should require recall of bases information to eliminate distractors.

Facility Response: question replaced.

22R F 2 N S NRC: Question appears to be matched to wrong learning objective (Inventory CSF vs. Core Cooling CSF). Question should list 23 H 3 N ES steam tables as a reference to be provided during exam.

Facility Response: change incorporated.

24 H 3 B S (Bank: South Texas Project 2010 Exam)

NRC: The K/A for comes from Westinghouse EPEs/APEs E15 Containment Flooding. Question should be testing knowledge from CSP-Z.2 Response to Containment Flooding. Current question is limited to RHR system knowledge and OP P&Ls.

25 H 3 N S Facility Response: facility believes K/A match based upon scope of K/A section involved; question involves containment flooding that will lead to emergency procedures.

NRC: agree with K/A match.

(Bank: Callaway 2011 Exam)

NRC: LOK is Fundamental vs. the listed Higher level; question may be answered solely by recalling the note. Distractor analysis unclear; implies that cooldown may be performed without having 26 HF 2 B ES any available steam generators (?). Typo in third bullet of stem; should say there is a note.

Facility Response: facility agrees that LOK is fundamental.

Changes incorporated.

27 F 3 B S (Bank: Surry 2012 Exam) 28 F 3 M S 29 H 3 B S (Bank: PBNP 2009 Exam)

(Bank: Prairie Island 2016 Exam)

NRC: LOK is Higher vs. the listed Fundamental level (bank 30 FH 3 B SE question LOK was higher). Question requires evaluating indications using understanding of system.

Facility Response: facility agrees that LOK is higher.

(Modified: Diablo Canyon 2008 Exam)

NRC: LOK is Higher vs. the listed Fundamental level (bank question LOK was higher). Question requires evaluating 31 F 3 M S indications using understanding.

Facility Response: facility believes LOK is fundamental based upon the simplicity of the question.

NRC: agrees with fundamental LOK.

4

ES-401 Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only (Bank: Turkey Point 2011 Exam)

NRC: Wording of second part of question in stem is awkward.

32 F 2 B ES Should be stated roughly along the lines of PCT could exceed the 50.46 criteria of _____ for clarity.

Facility Response: change incorporated.

NRC: Distractor analysis C has a typo; word student is omitted.

33 H 4 N ES Facility Response: change incorporated.

34 F 3 B S (Bank: Seabrook 2009 Exam)

(Modified: Kewaunee 2011 Audit)

NRC: This question could be clarified substantially by simply providing a list of indications in the stem and asking for the location 35 H 3 M ES of the source of leakage.

Facility Response: facility believes that clarity is sufficient; no issues with clarity were noted during validation.

NRC: agree that clarity is adequate.

36 H 3 B S (Bank: Ginna 2012 Exam)

(Modified: Surry 2014 Exam) Question does not meet criteria to be considered modified; should be listed as Bank. ES-401 D.2.f 37 H 3 MB ES requires modifying a pertinent condition in the stem.

Facility Response: change incorporated.

38 F 2 B S (Bank: Surry 2010 Exam)

(Bank: Catawba 2013 Exam) Distractor analysis C contains typo 39 F 3 B S (word in spelled min) and D refers to more than two loops.

40 F 4 B S 41 H 3 B S (Bank: Ginna 2007 Exam)

NRC: Distractor B is rendered implausible by supplied reference; it is obvious there is a difference in P/T curves based upon the 42 H 3 X N ES number of shroud fans running. It is unclear why this reference is needed.

Facility Response: change incorporated.

43 H 3 N S 44 H 3 B S (Bank: PBNP 2009 Exam) 45 F 2 M S (Bank: Prairie Island 2010 Exam, Comanche Peak 2013 Exam)

(Modified: Diablo Canyon 2012 Exam) 46 F 2 M ES NRC: Typo in Cognitive Level discussion, AFW is misspelled.

Facility Response: change incorporated.

47 H 2 M S (Modified: PBNP 2005 Exam) 48 F 3 B S (Bank: PBNP 2009 Exam)

NRC: Stem contains a clue for the first half of the 2x2 answer. G-01 starting air pressure is given as 120 psig while G-04 pressure is given as normal. The implication of this phrasing is that G-01 pressure is abnormal. Instead, provide a G-04 starting air pressure value that is normal and to let the applicant assess it.

49 H 3 M ES Facility Response: change incorporated. Facility believes that issue should be considered as editorial in nature versus unsat since the word normal did not give away part of answer, although it did make it easier to get there.

NRC: agreed that issue was an editorial/enhancement in nature.

5

ES-401 Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only NRC: Typo in distractor analysis D; appears to confuse D-28 and 50 H 3 N ES D-40.

Facility Response: change incorporated.

51 H 2 B S (Bank: Palisades 2009 Exam)

NRC: LOK is Higher vs. the listed Fundamental level. Question requires evaluating indications using understanding of system.

Clarify distractor D by including that the valves are only designed to close on a Unit 2 SI.

52 F 2 N S Facility Response: facility believes that the graphic is acting as a list of valves and positions; based upon this, appropriate to consider the LOK as fundamental. Other change incorporated.

NRC: agree that LOK is fundamental.

NRC: Typo in distractor analysis C, word in misspelled tin.

53 H 2 B ES Facility Response: change incorporated.

54 H 3 M S 55 F 2 B S (Bank: Farley 2011 Exam)

NRC: Distractor B is potentially a second correct answer. As stated in in the distractor analysis, Tave is low (albeit by 0.5F) and Bank D demand position is considerable lower than expected.

56 H 2 X N US Facility Response: change incorporated. Facility requests consideration that original review status should have been E versus U.

NRC: original question, as submitted, was unsatisfactory.

57 F 2 N S 58 H 4 B S (Bank: PBNP 2005 Exam) 59 F 3 B S (Bank: Summer 2005 Exam)

NRC: Learning objective does not match question; objective is not associated with calculating time to boil. Capitalize word before in 60 H 3 N ES stem for emphasis. For clarity, make the first bullet in stem the third bullet to better depict the order of events.

Facility Response: changes incorporated.

(Modified: 2009 Sequoyah Retake Exam)

NRC: LOK is Higher vs. the listed Fundamental level. Question requires an extra mental step of first determining what the plant 61 FH 2 M ES effects of the generator lockout will be before answering the rest of the question.

Facility Response: facility agrees that LOK is higher.

(Bank: PBNP 2011 Exam)

NRC: Distractor analysis for B is unclear; it appears that 62 H 3 B ES reasoning for level misconception may be misstated.

Facility Response: change incorporated.

63 H 2 N S 6

ES-401 2 Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only NRC: Distractor analyses are unclear; they do not discuss the misconceptions that appear to be anticipated from plant 64 H 3 B ES configuration and available flowpaths (i.e. MSIVs, isolations, etc).

Facility Response: change incorporated.

(Bank: PBNP 2005 Exam)

NRC: Typo in distractor analysis B word same is misspelled 65 H 2 B E save.

Facility Response: change incorporated.

(Modified: 2011 South Texas Project Exam)

NRC: Distractor B is not plausible; this is not a large motor, why would it be considered?

66 F 2 M S Facility Response: facility contends that distractor is plausible based upon automatic equipment into manual control; another large motor might make for a second correct answer.

NRC: agrees, leave distractor as is.

67 F 2 B S (Bank: 2011 Callaway Retake Exam)

NRC: The distractor analysis for C and D makes reference to the valves in the stem being located in EOP E-0 Attachment A, 68 F 3 B ES however, this was not located in the procedure.

Facility Response: change incorporated.

NRC: ES-401, D.2.a, states to ensure Tier 3 questions maintain a focus on plant-wide generic knowledge and do not become an extension of Tier 2 plant systems. The submitted ES-401-3 lists K/A 2.2.14 in Tier 3 for this question. The submitted exam 69 F 2 X B U question is associated with 057G2.2.14 and has a system-specific focus, however. This is inappropriate and inconsistent with the outline. Typos in distractors A and B analyses (this be).

Facility Response: question replaced.

69R F 2 N S (Bank: PBNP 2011 Exam) 70 F 2 B ES NRC: Distractor analysis B contains a typo (the an).

Facility Response: comment incorporated.

71 F 3 B S (Bank: PBNP 2009 Exam) 72 F 2 B S (Bank: Vogtle 2012 Exam)

NRC: Level of difficulty of this question is too low. Its possible to identify correct answer using the noun names of the instruments 73 H 13 N US and the listed information from the LCO with little other knowledge.

Facility Response: question revised to raise LOD.

NRC: LOD of revised question is acceptable.

NRC: Reword the question in the stem to read What action, if any, should the crew take and what is the basis? Answer choices 74 H 2 N ES would be clearer with semi-colons used instead of commas.

Facility Response: changes incorporated.

(Bank: PBNP 2005 Exam)

NRC: Clarify the initial conditions by including as the first bullet in 75 H 3 B ES the stem that a LOCA has occurred.

Facility Response: change incorporated.

7

ES-401 2 Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only (Bank: PBNP 2009 Exam)

NRC: Distractor analysis D is unclear; if applicant incorrectly 76/1* H 3 Yes B ES calculates level available to Unit 1, wouldnt they then think that the TLCO was met for Unit 1?

Facility Response: change incorporated.

NRC: Distractor A is a potential correct answer; the stem is asking for a sequence of actions, and the required actions appear in order in this distractor. Distractor C should contain four items as well for consistency.

77/2* H 3 Yes N ES Facility Response: facility contends that AOP requires crew to verify immediate actions of EOP-0 (requires branching prior to tripping RCP); distractor A incorrect. Other change incorporated.

NRC: agree that distractor A is not a second correct answer.

(Bank: PBNP 2011 Exam)

NRC: The exam which this question originated from associated it with a K/A for use of AOPs in conjunction with EOPs. It does not 78/3* H 2 X Yes B U test the turbine/reactor trip aspect of current K/A in more than a superficial manner, however.

Facility Response: question replaced.

78/3 H 3 Yes M S R

(Bank: Callaway 2011 Exam)

NRC: LOK is Higher vs. the listed Fundamental level. Question requires diagnosing the event and then applying additional 79/4* FH 2 Yes B ES information from the stem to determine correct actions (multiple mental steps).

Facility Response: facility agrees, LOK is higher.

(Modified)

NRC: Question does not meet criteria to be considered modified; 80/5* F 3 Yes MB E should be listed as Bank. ES-401 D.2.f requires modifying a pertinent condition in the stem.

Facility Response: change incorporated.

81/6* H 4 Yes N S NRC: LOK is Higher vs. the listed Fundamental level. Question requires applying information from the stem to recalled information 82/7* FH 3 Yes N ES multiple times in order to determine correct actions (multiple mental steps).

Facility Response: change incorporated.

(Modified: PBNP 2007 Exam)

NRC: See comment for Question 93 (SRO Question 18) concerning a clue which that question would provide for this question.

83/8* F 2 Yes M S Facility Response: facility disagrees; student must know specifics of what happens to filter alignment, though question always involves a shift.

NRC: agree, no clue is provided.

(Modified: Ginna 2010 Exam)

NRC: The supplied previous question does not match question 84/9 H 3 Yes M ES #84 from the 2010 Ginna exam.

Facility Response: change incorporated, should have listed question as Ginna 2011 question #85.

8

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only (Modified: PBNP 2009 Exam)

NRC: Original version of question was associated with a different K/A. It does not test the generic portion of E10G.2.4.50, however 85/

H 3 X X Yes M U (or match the learning objective). None of the CST-related 10 information in the stem is necessary for answering the question (4 bulleted items worth of info).

Facility Response: question replaced.

85/

H 2 Yes N S 10R NRC: Stem information about loose parts and high RCS activity appears unnecessary. Use term steady instead of flat lined.

86/

H 3 Yes N ES Facility Response: first comment incorporated. Facility uses term 11 flat lined to describe stability.

NRC: agree with leaving term flat lined in question.

(Modified: Braidwood 2011 Exam) 87/ NRC: Include a pressure value/lowering trend in stem (similar to H 3 Yes M ES 12 original version) for clarity.

Facility Response: change incorporated.

88/

H 3 Yes B S (Bank: PBNP 2011 Exam) 13 89/14 H 3 B S (Bank: PBNP 2007 Exam)

(Bank/Last 2: PBNP 2015 Exam)

NRC: Per ES-401 Att. 2 (Clarification Guidance for SRO-only Questions)Section II.E, SRO-only knowledge should not be claimed for questions that can be answered solely using 90/15 H 3 No B E knowledge of plant parameters requiring direct entry into Red and Orange Functional Restoration procedures. Also, the Service Water part of the K/A match is very weak in the questions current form.

Facility Response: question replaced.

NRC: 55.43 link should be 5 instead of 2 (selection of 90/15 H 4 Yes N ES procedures).

R Facility Response: change incorporated.

NRC: With no time frame discussed in the stem, distractor D could potentially be a second correct answer; part one of the question statement should include the LCOs within one hour action time frame to prevent this. Question requires knowing from memory a 4% channel check acceptance criteria that is not in Tech Specs; this appears to be beyond reasonable memory level knowledge. LOK is Higher vs. the listed Fundamental level; question requires interpreting meter data in the stem and applying 91/16 FH 4 X Yes N US recalled information to determine correct actions (multiple mental steps).

Facility Response: facility agrees with partial flaw comment; question revised to alleviate flaw. Facility agree that LOK is higher; change incorporated. Facility disagree with minutia flaw comment; channel check criteria is recorded/verified/reviewed as part of shift logs.

NRC: agree with question not containing an additional minutia flaw.

92/17 H 3 Yes N S 9

ES-401 2 Form ES-401-9

3. Psychometric Flaws 4. Job Content Flaws 5. Other

1. 2.

6. Source 7. Status Q LOK LOD SR 8. Explanation Stem Cred. Job- #/ Back Q- (B/ M / N) (U /E /S)

(F/H) (1-5) Cues T/F Partial Minutia O Focus Dist Link Units ward K/A Only NRC: Stating that PAB exhaust automatically shifts filters in response to system effects on RE-214 provides a clue for answering Question 83 (SRO Question 8), a cueing concern between both questions. The stem will require modification to 93/18 H 4 Yes N ES alleviate this. It would also be preferable to put an image of TS 3.3.5 Condition A in the stem for clarity since TS 3.3.5 itself will not be provided as a reference.

Facility Response: first comment previously resolved (see question #83 for detail). Other comment incorporated.

NRC: The listed 10CFR55.43 link appears to be incorrect. Also, in distractor B the term CO is used. Is this synonymous with 94/19 F 2 Yes B ES Operator at the Controls?

Facility Response: comments incorporated.

NRC: The listed 10CFR55.43 link appears to be incorrect.

Facility Response: facility believes that link is correct since a 95/20 F 3 Yes M S procedure change is addressed.

NRC: agree that link is correct.

96/21 F 2 Yes B S (Bank: Farley 2013 Exam)

NRC: This question requires determining PMT requirements located in maintenance procedures from memory. This knowledge is too specific for a closed-reference exam question. Also, stem should provide definition of term RMP.

97/22 F 3 Yes B ES Facility Response: Facility disagrees with minutia flaw; facility will provide additional learning objective and lesson plan information to support that question is reasonable for a closed-book examination.

NRC: agree that question is reasonable and does not contain a minutia flaw.

NRC: ES-401, D.2.a, states to ensure Tier 3 questions maintain a focus on plant-wide generic knowledge and do not become an extension of Tier 2 plant systems. This question is focused on the CREFS system and associated TS requirements. Also, LOK is 98/23 F 3 X Yes M U Higher vs. the listed Fundamental level. Question requires applying information from the stem to recalled information multiple times in order to determine correct actions (multiple mental steps).

Facility Response: question replaced.

98/23 NRC: typo in justification section; cm is misspelled.

F 2 B ES R Facility Response: change incorporated.

(Modified: PBNP 2007 Exam) Per App. B, Section C.2.e, negatively stated stems are undesirable, but are not prohibited.

99/24 F 2 Yes M ES Distractor analysis D has typo; volunteer is misspelled.

Facility Response: change incorporated.

100/

H 3 Yes N S 25 10

Other Comments Regarding Written Exam:

• NRC: ES-401-6 Item #5: Audit Question 88 (SRO 13) and Exam question 89 (SRO 14) use the same K/A (062G2.44);

overlap concern.

o Facility Response: audit written exam question provided for comparison.

NRC: no overlap concern based upon review.

• NRC: ES-401-6 Item #5: Audit Question 96 (SRO 21) and Exam question 98 (SRO 23) use the same K/A (2.3.13); overlap concern.

o Facility Response: audit written exam question provided for comparison.

NRC: no overlap concern based upon review.

• NRC: ES-401-6 Item #7: RO section of exam exceeds higher cognitive threshold of 50-60% (actual = 61.3%).

o Facility Response: comment incorporation has lowered RO section higher cognitive value to within 50-60% band.

11