ML092810456

Initial Exam 2009-301 Final Administrative Documents

ML092810456
Person / Time
Site:	Watts Bar
Issue date:	10/08/2009
From:	NRC/RGN-II
To:
References
50-390/09-301, 50-391/09-301
Download: ML092810456 (59)

Text

ES-401 Written Examination Form ES-401-9 Review Worksheet Watts Bar 2009-301

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

@ The stem or distracters contain cues (i.e., clues, specific determiners, phrasing, length, etc).

@ The answer choices are a collection of unrelated true/false statements.

@ One or more distracters is not credible.

@ One or more distracters is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

@ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6 Based on the reviewer=s judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

RO/SRO Combined Question Generic:

NP = Not plausible NDV = No discriminatory value CAF = Check at facility WOOTF = Which one of the following DV = discriminatory value FACILITY RESPONSE COMMENTs: Please type all facility responses in blue. Do this for each question Put all periods and question marks inside the quotes where applicable Highlighting the references (and providing references to validate distracters) would have significantly reduced Chief Examiner evaluation time. I had to resort to digging into the supplied electronic reference to verify. The reference material did not do a good job in justifying distracters.

Some Q had KA justification statement and some didnt.

Many questions have an inadequate explanation as to why the distracters are plausible EX (Q 55 - KA 103K1.01). Ask the question to yourself, what would lure the applicant to select this choice?

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 000008G2.2.20 1

C 3 X Y N E If C was correct, D would be also. Modify D to read .close ONLY one .

Tail pipe temperatures are discrete. It makes no sense to close both PORVs. Distracter C is NP. Replace C.

RFA 04/01/09 Replaced Distractor C.

Modification to D no longer required due to change in C.

Modified stem and 2 choices Approved 04/29/09 RFA 2 000009EK2.03 C 3 X X Y N E Lower the RCS Temperature to make distracter A more plausible but not so low that C and D become correct answers too. Currently it is NOT. Force the applicant to have to reference the steam tables to answer the Q.

RFA 04/01/09 Modified question to identify 2 components needed to establish cooling. Could not lower RCS temperature without moving from a small break toward a large break where the argument could be made that the SGs may not be required.

Modified stem.

Approved 5/6/09 RFA 000011EA1.05 3

C 2 X Y N U Psychometrics:

B is also a correct answer. B needs an only Its common knowledge that suctions are in parallel on safety related systems. One failure on an SI will not prevent the system from performing its intended safety function.

Therefore, distracter A is NP Between the psychometric flaw and multiple correct answers, this Q is U.

RFA 04/01/09 Replaced question.

Reworded distractors to clarify overlap sequence.

Approved 5/6/09 RFA 00015/017AK2.10 4

C 2 X Y N U Since this is a high Cog Q, require the applicant to know two pieces of information. B is a good distracter but A and D are not. Use loop flow and amps twice and come up with a second part and set it up like Q 3.

The Q is U because of 2 NP distracters.

RFA 04/01/09 Used loop flow and amps twice, set up table and added second part. Also lowered the power level in the question stem to support plausibility of the second part.

Modified stem Approved 04/29/09 RFA 000025AK3.01

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 5 M 2 X Y N E D is a NP distracter because there is no indication of voiding in the tubes (for reflux boiling) and there wont be with T = 195 even at ambient pressure. Therefore, replace distracter D.

RFA 04/01/09 Distractor D replaced with Normal charging into RCS and removal of water via manual Pressurizer PORV control which is an alternate cooling method but not the preferred method.

Modified stem and all 4 choices Approved 04/29/09 RFA 000026AK3.04 6

C 2 Y N S No comment RFA 04/01/09 Approved 04/29/09 RFA 000027AK1.03 7

C 2 X Y N U Distracters A,B, and C make absolutely no sense even for someone with limited thermodynamic knowledge.

Distracters A, B, and C are NP and D is correct but for the wrong reason.

Rewrite C to look just like D except change cooler water to hotter water. For A and B, write two distracters that involve the Pzr heaters.

This Q is U because of 3 NP distracters.

RFA 04/01/09 Rewrote C to look like D except for hotter, rewrote distractors A and B to address thermodynamic principles and to include pressurizer heaters.

Modified stem and all distractors Underlined occurs in the Pressurizer (PZR) in the WOOTF statement to focus on the change in the pressurizer versus the change in the RCS.

Approved 05/14/09 RFA 000029EA1.12 8

C 2 X Y N E P-4 functions description was not provided in the references. Please explain why reactor trip functions will occur normally is a good part 2 distracter for A and B.

RFA 04/01/09 Modified to remove reactor trip functions occur normally aspect.

Approved 04/29/09 RFA 000038EK1.03 9

C 2 X Y N E In the answer (A), remove the upper head region becomes inactive and. This part gives the answer away. Since the whole RCS is inactive because the RCPs are off, the correct answer does NOT need to be telegraphed.

RFA 04/02/09 Modified the A distractor to read The fluid temperature in the upper head region significantly lags the temperature in the RCS loops. Could not remove the entire phrase because the wording left would not ID which region was lagging.

Approved 04/29/09 RFA

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 000054G2.2.22 10 N U This KA is supposed to reference a Loss of Feed CAF: re-write Q if necessary or explain.

This Q is U until this discrepancy is resolved.

RFA 04/02/09 NEW question written.

Modified stem by redefining bullets and changed SG setpoints, Approved 04/29/09 RFA 000056AK1.04 11 C 3 Y N S No comment RFA 04/02/09 Approved 04/30/09 RFA 000057AA2.19 12 C 2 Y N S No comment RFA 04/02/09 Modified one distractor 04/30/09 approved RFA 000058AA1.02 13 C 2 X Y N E* The fact that a loss of a vital inverter will ALWAYS result in some sort of TS entry renders A and B NP. However, this is an easy fix. For A and B second part add TS X and Y. For C and D state TS Y only.

This Q is E* because the Q is technically a U but can be fixed with a minor change RFA 04/03/09 Modified question to address 2 T/S conditions as described above. Added only in distractors C and D. Modified stem wording and question structure to accommodate the changes.

04/30/09 approved RFA 000062AA2.01 14 C 2 X Y N E The Q is NOT psychometrically balanced. In C and D. In C and D change A supply header has ruptured to A rupture has occurred As it stands, distracter B is NP RFA 04/03/09 Made the change identified above in C and D.

Modified one distractor 04/30/09 approved RFA 000065AK3.03 15 C 2 X Y N E CAF: Do you need to state in the stem With No Operator Action? If the board operator places the MFP in manual, will this have the same affect as if he left it in auto?

RFA 04/03/09 Added phrase (Assume no actions taken by the Main Control Room Operator during the event).

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 04/30/09 approved RFA E04EK2.1 16 N U Since the leakage is ultimately ending up back into containment via the PRT, this is NOT a LOCA outside containment. Eventually, the PRT rupture disc will blow and the water will wind up in the containment sump. Therefore the KA is not met.

Try to redirect the stem of the question so the leakage is collecting somewhere outside the containment.

This Q is U because the KA is not met RFA 04/03/09 Replaced question.

Rewrote distractors.

5/6/09 approved RFA E11EA2.1 17 C 3 Y N S No comment RFA 04/03/09 04/30/09 approved RFA E12G2.2.36 18 C 1 X Y N U It is common knowledge that code safety valves are and always will be TS. The applicant knows that there is only one correct answer. He will select B regardless whether he thinks one of the other distracters might be true or not. This Q, as is, has no DV. Rewrite the Q so that each distracter has a TS LCO entry and one that is not.

1. AA

2. AB (B has to be convincing)

3. BA (B has to be convincing)

4. BB (this distracter would have to be very convincing since neither one is an LCO entry)

RFA 04/03/09 Rewrote the question as described above. Added second condition and both elements in condition 2 involve T/S equipment. (The isolation of the manual removes the applicability of the bypass reg valve LCO). Selected 2 of the original conditions and modified wording.

04/30/09 approved RFA 000001AA2.01 19 C 2 X Y N U For this condition, distracters A and B are NP. The procedure is clear and this is NOT an uncommon action. Replace distracters A and B or rewrite the Q.

This Q is U because of two NP distracters RFA 04/03/09 Revised question to make A and B plausible.

04/30/09 approved RFA 000003G2.2.44 20

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only rd C 2 X X Y N E Remove in back D, Group2) from 3 bullet. Otherwise, distracter A is NP.

RFA 04/03/09 rd Removed in back D, Group2) from the 3 bullet. Modified distractors A and B to pevent having to know which group M-12 was in. Restructured stem of question.

Restored question to the original format with minor modification.

04/30/09 approved RFA The KA # is incomplete 21 C 2  ? N E* CAF: Resolve KA RFA 04/06/09 Added 000028 to complete the KA# - 000028 AK1.01.

04/30/09 approved RFA 000033AA2.01 22 C 3 Y N S No comment RFA 04/06/09 04/30/09 approved RFA 000059AK2.02 23 C 2 X X Y N E The distracter analysis does not support any of the choices. B stares the reason is correct when in fact it is NOT. D states that the radiation monitor is correct when in fact it is not.

CAF verify plausibility of distracters.

RFA 04/06/09 Reworded distractors to address plausibility issue.

5/6/09 approved RFA 000076AK3.05 24 M 2 X Y N U B and C are NP since there is no reason to isolate LD.

Replace these two distracters.

This Q is U because of two NP distracters RFA 04/06/09 Added phrase in conjunction with normal letdown to B is indicate letdown not isolated.

Replaced the C distractor with The cation bed is placed in service at 120 gpm flow in order to reduce the amount of fission products in the reactor coolant system.

Modified wording on correct answer D.

Modified 2 distractors 04/30/09 approved RFA E08EA1.3 25 C 2 X Y N E With PTS in play, distracter C (increasing RCS pressure) is NP. Replace C.

RFA 04/06/09 Replaced distractor C.

Modified stem - Must validate subcooling numbers Changed subcooling value to 80 deg. F.

5/6/09 approved RFA

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only E14EK2.1 26 C 2 X Y N U C is also a correct answer because the SI signal cannot be reset until Containment pressure is less then the SI setpoint (15 psig)

This Q is U because of two correct answers.

Replace C.

RFA 04/06/09 Modified the question to eliminate multiple correct answer issue by addressing the Auto SI block function when SI is reset.

Reworded stem.

5/6/09 approved RFA E15EK1.2 27 M 2 X Y N U The second part of A and B is not plausible. Its a LOCA as stated in stem. Its obviously RCS water from somewhere.

This Q is U because of two NP distracters.

RFA 04/06/09 Replaced distractors A and B, second column. Made stem modification. Added discussion in the distractor analysis to explain plausibility, added reference.

Added clarification to two choices to include a more accurate description per the applicable reference; i.e., TSC aspect.

Approved 05/14/09 RFA 003K2.02 28 M 2 Y N S No comment RFA 04/06/09 Adjusted distractors B and D 04/30/09 approved RFA 004A2.05 29 C 2 X X Y N E* C and D: The phrase after 3-5 minutes) is a key phrase. Applicants will be able to eliminate A and B immediately. Move this phrase after 3-5 minutes to the stem.

This Q is E* because the Q is technically a U but can be fixed with a minor change RFA 04/06/09 Moved the phrase after 3-5 minutes to the stem. Also moved the word close from all 4 choices to the stem. Phrase now reads after 3-5 minutes close.

Modified stem, 04/30/09 approved RFA 004K5.08 30 C 2 X Y N E The CR calculation is good. There was no proof regarding the amount of primary water to be added.

CAF: Prove the first part of the calculation.

RFA 04/06/09 Added discussion of water added in the distractor analysis.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 04/30/09 approved RFA 005K3.06 31 C 2 X Y N E Delete (Evaluate each condition separately). There is only one correct answer.

The words less than adequate are somewhat purple. Change to nonfunctional, inoperable, unavailable, etc.

RFA 04/06/09 Deleted (Evaluate each condition separately) and changed wording from less than adequate to unable to be aligned and restructured question sentence.

Modified stem 04/30/09 approved RFA 006A4.05 32 M 2 X Y N E Change B to read NOT occur because an SI signal is still present to the RHR sump valves because B, as written, does not call out the manual actions and is therefore, NP because will be immediately eliminated.

See reference 3-OT-SYS063A, P 20 of 67 RFA 04/06/09 Changed B to read NOT occur because an SI signal is still present to the RHR sump valves as identified above.

04/30/09 approved RFA 007K4.01 33 M 2 X X Y* N E* A and C are also correct answers because the stem does not have a qualifier. Add the word maximum to the stem and make this Q a fill in the blank.

This Q is E* because the Q is technically a U (multiple correct answers) but can be fixed with a minor change to the stem.

RFA 04/06/09 Added qualifier and made fill in the blank.

• Question replaced to better match the K/A.

05/19/09 approved RFA 007K5.02 34 M 2 X Y N E Neither the reference material nor the distracter analysis demonstrate why A, B, and C are wrong or plausible.

CAF RFA 04/06/09 Rewrote question.

Modified Stem, correct answer, and one distractor.

Re-modified stem wording and first part of each choice 05/19/09 approved RFA

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only Distracters B and D are NP. Common knowledge dictates that a trip is not immediately 35 required for a isolating LD as a result of a HX leak. Its common operator knowledge that if M 2 X Y N U a CCS pump malfunctions, it wont lead to a RT.

Replace B and C.

This Q is U because of two NP distracters.

RFA 04/06/09 Replaced the two implausible distractors.

04/30/09 approved RFA 010K6.04 36 M 2 X Y N U The distracter analysis is not adequate in explaining why A and B are plausible. The reference material does not lend any support in justifying the plausibility of these two distracters either.

CAF: Justify A and B. This Q is U until justified. Just because it was on a previous exam does not mean it is plausible.

RFA 04/06/09 Replaced the question.

Minor rewording.

5/6/09 approved RFA 012A2.01 37 C 3 Y N S No comment RFA 04/06/09 Reworded distractor D.

5/6/09 approved RFA 012G2.4.35 38 M 2 Y N S No comment RFA 04/06/09 Question was not a complete technical description of procedural guidance as originally written; i.e., BOTH actions must be completed for a reset of the SI. There was no correct answer for the second part of the question due to the wording used. Revised the question to make the original answer accurate. Modified 2 distractors to provide plausibility for the new wording in the second part of the question. The new distractors identify the required action that follows the correct action and the question wording asks for which would be performed first.

Approved 05/14/09 RFA 013K2.01 39 C 2 Y N S No comment RFA 04/06/09 5/6/09 approved RFA 022A3.01 40 M 2 X Y N E The reference material provided is inadequate. It doesnt explain the logic. Distracter B states only 7 minutes has elapsed since actuation but doesnt state which actuation. Are talking SI or phase B? Which logic requires 9 minutes elapsed time SI or Phase B? If it is Phase B then change the WOOTF statement to 0354 to put it closer to the setpoint.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only RFA 04/06/09 Added wording to the distractor analysis.

Changed the WOOTF statement to 0354 Eliminate fan response aspect. Expanded to reason for damper response. Modified all distractors.

5/6/09 approved RFA 025K1.02 41 M 2 X Y N U Even if C or D were correct, A would be also. Therefore, C and D are NP.

Change A and B by adding an only at the end. Change C to match D since D is more plausible, then add A to C and add B to D.

This Q is U because distracters C and D are NP because of A and B.

RFA 04/06/09 Added only to A and B. Changed C to match D and added A to C and add B to D as identified above.

Underlined "directly" in stem.

5/6/09 approved RFA 025K4.02 42 C 2 Y N S No comment RFA 04/06/09 5/6/09 approved RFA 026K4.09 43 M 2 X Y N E Need an IAW a procedure ____ in the stem to bullet proof the correct answer.

RFA 04/06/09 Added in accordance with ES-1.3 at the start of the question sentence in the stem.

Rewrote to emphasize knowledge item of interlock feature.

5/7/09 approved RFA 039A3.02 44 C 2 X Y N E Add the word only to D and add D to A. This will add plausibility to A. Currently A is NP relative to D by itself.

RFA 04/06/09 Added only to D- now reads in 5 seconds, only. and added D to A . Now reads 700 psig or a rapid steam line pressure drop of 100 psig in 5 seconds.

Reverted to original form of question. Changed a distractor value to 675 psig.

5/6/09 approved RFA 059A2.03 45 C 2 Y N S No comment RFA 04/06/09 5/6/09 approved RFA 061K5.03 46 C 2 Y N S No comment

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only RFA 04/06/09 Minor wording change in stem.

5/6/09 approved RFA 062G2.4.11 47 C 2 X Y N U It is not clear to me why A and C are plausible when basic knowledge dictates that all safety related pumps have redundancy. A and C are NP. Add something in the stem to make the applicant question whether or not the B train pump is available or replace A and C distracters. This Q is U due to two NP distracters.

RFA 04/07/09 Modified the stem and distractors.

Remodified the stem to clarify initial conditions.

5/6/09 approved RFA 063A1.01 48 M 2 X Y N E* Add a WOOTF statement in the beginning to be consistent.

If A was correct, B would be also.

If C was correct , D would be also.

In C and D, why is 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> plausible. The reference lends no credibility. CAF: explain.

The stem needs a qualifier in the second part for a maximum of _______

This Q is E* because the Q is technically a U but can be fixed with a minor change RFA 04/07/09 Added statement to the question Which ONE of the following identifies the times the Vital Batteries are designed to have adequate capacity to supply the loading conditions listed below?

Added phrase a maximum of _______."

The LCO time of 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> for an inoperable battery provides plausibility for the use of 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> as a distractor.

Minor wording change in stem: added "at least".

5/6/09 approved RFA 064K6.07 49 C 2 X Y N E If C was correct, D would be also. Therefore C is NP. Must put in a qualifier to eliminate this ambiguity.

RFA 04/07/09 Added qualifiers to C and D.

Minor wording change to distractors C and D: "indicates" vs. "is" for pressure value.

5/6/09 approved RFA 064K6.08 50 M 2 Y N S No comment RFA 04/07/09 5/6/09 approved RFA 073K3.01

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 51 M 1 Y N U B is such an obvious answer that this Q has very low DV. The DV needs to be increased to at least a 2.

This Q is U because of low DV RFA 04/07/09 Replaced the question.

Minor correction to distractor analysis for the correct answer: C.

5/6/09 approved RFA 076A1.02 52 C 2 X Y N E Given all these annunciators, D is NP. Replace D RFA 04/07/09 Replaced the question.

Modified to change the pump that trips (A-A) which changes the correct answer to D.

5/6/09 approved RFA 078A4.01 53 M 2 X Y N E In distracter C, why is 75 a good number? I do not see the plausibility.

CAF: re-evaluate C.

RFA 04/07/09 Provided a statement for why 75 psig is plausible for distractor C.

Restructured the stem and changed 80 to 75 in second part of distractor A.

5/6/09 approved RFA 078K1.01 54 C 2 X Y N E If D was correct, C would be also. In C add only before 6.

RFA 04/07/09 Added only to C, now reads but only 6 dump valves will Modified stem, and rewrote all four choices.

Added controller nomenclature for clarity due to validation comment.

5/19/09 approved RFA 103K1.01 55 C 2 X Y N U Why is D plausible if loss of cooling water to 1A-a fan would not trip the fan or anything else.

Why not say the lower containment cooler 1D-B will have a reduced cooling capacity of approximately 10.5% due to isolation of ERCW Since air return fans start on a phase B isolation, why not say for B, the lower compartment cooler 1 D-B starts on a phase B isolation signal only and in C state the lower compartment cooler 1 D-B starts on a phase B isolation signal and the TCVs for the running cooler open more In distracter add the word only This Q is U due to multiple NP distracters RFA 04/07/09 Modified to incorporate the distractor changes as identified above. Made cooler plural in

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only distractor C.

Wrote new question.

5/7/09 approved RFA 001K6.03 56 C 2 Y N S No comment RFA 04/07/09 Modified stem and all distractors.

Eliminated "and Safety Injection in the second bullet to make the question technically accurate. If Safety injection had occurred an additional feedwater isolation signal would have been generated and closed the feedwater isolation valves.

Approved 05/14/09 RFA 002K3.03 57 C 2 X Y N E Delete (Evaluate each condition separately). There is only one correct answer.

Distracter A: Why would the tank overflow at 78%? What would cause the tank to overflow? The reference is inadequate for explanation. If the RCDT relief lifts, where does it go? Possibly the PRT? If so, then distracter A should state ..... and level on 1-LI-77-1 indicates 100% and rising Replace distracter A.

RFA 04/08/09 Deleted phrase(Evaluate each condition separately).

Changed distractor A to 100%.

Replaced distractor A.

5/7/09 approved RFA 016K5.01 58 M 2 Y N S No comment RFA 04/08/09 5/6/09 approved RFA 071K1.01 59 M 2 Y N S No comment RFA 04/08/09 5/7/09 approved RFA 027K2.01 60 M 2 Y N E The Cog level is Memory not High. This is a systems recall Q.

No comments on the Q S RFA 04/08/09 Changed cog level to Low.

Disregard comment. Restore to higher cog. level designation.

Question is SAT as is.

5/6/09 approved RFA 033K4.01

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 61 M 1 Y N U All SFP have vacuum breaks. This Q, as written, has no DV.

Replace the Q This Q is U due to low DV.

RFA 04/08/09 Modified the question.

Modified C and D level value for plausibility.

5/7/09 approved RFA 034A1.02 62 M 2 Y N S No comment RFA 04/08/09 Note: K/A match is not exact (but acceptable) because the cavity seal is not part of the fuel handling system, however, the second part of the K/A is met because stem conditions do affect refueling canal water level.

5/7/09 approved RFA 045G2.2.37 63 C 2 X Y N E Underline complete in the stem to bullet proof the choices.

RFA 04/08/09 Underlined complete in the stem.

Based on validation changed operable to available 5/19/09 approved RFA 068A3.02 64 M 2 Y N S No comment RFA 04/08/09 Minor editorial changes.

5/7/09 approved RFA 072A2.01 65 M 2 X Y N E* Since the area monitors instrument malfunction annunciator is lit, the second parts of distracters A and C are NP as written.

Delete evacuate nonessential personnel from the affected area on both.

This Q is E* because the Q is technically a U but can be fixed with a minor change.

RFA 04/08/09 Deleted evacuate nonessential personnel from the affected area on distracters A and C.

Modified stem to clarify procedure referenced.

5/7/09 approved RFA G2.1.4 66 C 2 X Y N U Since a Unit Supervisor is always required for all modes, B is NP.

Since there is never a situation where 2 NLOs are required, distracter D is NP.

This Q is U because of two NP distracters.

RFA 04/08/09 Rewrote all three of the distractors to eliminate the NP distractors.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only Modified structure of all choices for readability.

5/7/09 approved RFA G2.1.42 67 M 2 X Y N E Since 5 are allowed to be outside of approved storage, Distracter A (1) is not a good number. The distracter analysis was not convincing.

Suggest consider using 6.

RFA 04/08/09 Changed the number from 1 to 6. Then arranged choices in numerical order with D becoming 6 and A,B, C becoming 3,4,& 5 respectively. C becomes correct answer.

Minor editorial to stem.

5/7/09 approved RFA G2.2.20 68 M 2 X Y N U What is the difference between A and B? They are subtle and in a different order.

CAF: re-evaluate A and B. I believe this Q currently has two correct answers. This Q is U until resolved due to two possible correct answers.

RFA 04/08/09 Added identified by the troubleshooting to the end of B distractor to identified that the work order was not written prior to the troubleshooting activity.

Restructured all choices for readability.

5/7/09 approved RFA G2.2.21 69 M 2 X Y N E* If A was correct, B, C, and D would also be correct. Re-write A distracter and put the word only after B and D. Cap the word and in C.

This Q is E* because the Q is technically a U but can be fixed with a minor change.

RFA 04/08/09 Added only to A and B, Capped the word and in C, and replaced the A distractor.

approved subject to validation comments due to no direct procedure requirement.

5/7/09 RFA G2.3.12 70 M 2 Y N S No comment RFA 04/08/09 Modified distractor format.

5/7/09 approved RFA G2.3.14 71 C 2 Y N S No comment RFA 04/08/09 5/7/09 approved RFA G2.3.15 72 M 2 Y N S* This Q is week. Distracter A is minimally plausible especially when you plug the words back into the stem.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only However, will accept as is.

RFA 04/08/09 Reformatted choices and stem.

5/7/09 approved RFA G2.4.9 73 M 2 X Y N E In the WOOTF statement, add .describes (1) the initial expected .)

RFA 04/08/09 Added initial to the WOOTF statement as described above.

Modified stem conditions to clarify not solid water ops.

To improve operational validity, the question was changed to solid water operation.

Comments were received concerning how the pressure could rise rapidly to 600 psig with the question setup as originally written; i.e., a steam bubble would prevent a rapid pressurization due to changes in inventory.

Approved 05/14/09 RFA G2.4.19 74 M 2 Y N S No comment RFA 04/08/09 Replaced distractor B and reordered it with A.

5/7/09 approved RFA G2.4.27 75 C 2 X Y N U Make column (1), distracters A and B Condenser steam dumps or atmospheric steam dumps Make column (1), distracters C and D Atmospheric steam dumps ONLY A and B as they are, are NP.

This Q is U because distracters A and B are NP RFA 04/08/09 Added only to C and D in column 1, changed Atmospheric Steam dumps to Atmospheric Relief valves to match plant terminology.

Added Atmospheric relief valves to A and B in column 1, 5/7/09 approved RFA SRO ONLY KA: 000008G2.1.20 76 C 3 Y Y S No comment RFA 03/24/09 4/29/09 approved RFA

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only KA: 000011EA2.01 77 C 3 X Y Y E CETs are NOT 1200 but 820 and rising slowly IAW the stem. Therefore, distracter C is NP.

RFA 03/24/09 Revised question to add clarifying RCP status in stem, replaced distractor relating to SAMGs (CETs at 1200 degrees).

Modifications made 04/29/09 approved RFA KA: 000025AA2.06 78 C 2 X Y Y U Distracter C and D: are NOT plausible because it is common knowledge that single failure criteria applies. One PORV being OOS should not prevent a safety function. Replace C and D.

This Q is U because of 2 NP distracters RFA 03/24/09 Modified question; changed stem, correct answer and all distractors in the first part of the question.

Replaced question.

Provided reference for LCO 3.4.12 due to validation comments.

05/19/09 approved RFA KA: 000056G2.4.45 79 C 3 Y Y S No comment RFA 03/24/09 04/29/09 approved RFA KA: 00065AA2.03 80 C 3 X Y Y E TS are NOT involved in the answer selection. Where are they solicited in the stem? Only distracters C and D make reference to TS. Therefore, how are they applied per the KA match and SRO only statement?

Added "in accordance with Technical Specifications" in stem.

4/29/09 approved RFA With a LOCA outside Containment, sump recirc would not be in play. Therefore, distracter 81 C 3 X Y Y E A is NP (Discuss plausibility of A with Chief Examiner; ECA-1.2 directs implementation of ECA-1.1).

RFA 03/24/09 Modified stem and distractors To make the question operationally valid, the subcooling number was changed to 0°F. With RVLIS at 32 % the vessel level is covering only approximately the bottom 3.5 feet of the fuel, thus the system is at saturation and subcooling would not exist.

Approved 05/14/09 RFA 028AA2.01 82 C 3 Y Y S No comment RFA 03/24/09

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only Editorial only to spell out Reactor Trip System Instrumentation (RTS) Instrumentation,"

versus RTS Instrumentation in choices C & D.

Approved 05/14/09 RFA 000076AA2.02 83 C 2 X Y Y U Why would the applicant reject 10CFR 50? Since control room habitability is such a big concern, distracters A and C are NP. Replace the first part of A and C.

This Q is U because of 2 NP distracters.

RFA 03/24/09 Modified stem and distractors Added Steam Generator Tube Rupture to more accurately identify the analyzed events per the Tech. Spec. Basis.

Approved 05/14/09 RFA W/E03G2.4.30 84 M 1 X Y Y U Distracter A is obvious that the TSC must be notified. The other distracters are questionable in the applicants mind. From a psychometrics point of view, the applicant will choose A regardless what he knows about B, C, or D. Make this Q a two part Q where the applicants needs to know 2 pieces of information. This Q meets the KA and is SRO only but the COG level is 1 as written.

RFA 03/24/09 Revised question to make it a two-part question.

4/29/09 approved RFA E10G2.2.37 85 C 3 X Y Y E What prompts the applicant to raise CD rate? Nowhere in ES-0.3 directs these actions.

CAF RFA 03/25/09 Reworded ES-0.3 distractors to match the actual procedural guidance.

Modified stem and distractors 4/29/09 approved RFA 005A2.03 86 C 3 Y Y S No comment RFA 03/25/09 Removed one unnecessary bullet.

4/29/09 approved RFA 008G2.4.41 87 C 3 X Y Y E From a psychometric perspective, all choices are different except A and B. If the applicant has no clue to the correct answer, he will choose between A or B. Change distracter B.

RFA 03/25/09 Replaced distractor B.

Modified stem and all distractors 4/29/09 approved RFA 026G2.2.38

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only 88 M 2 Y Y S No comment RFA 03/25/09 Modified stem and all distractors 4/29/09 approved RFA 062A2.06 89 C 3 Y Y S No comment RFA 03/25/09 4/29/09 approved RFA 063G2.2.37 90 C 3 X Y Y E The choices are psychometrically unbalanced. Generate another inoperable statement and replace distracter D. If the applicant doesnt know the answer, he will eliminate D first.

RFA 03/25/09 Generated a new distractor to replace distrator d and placed in B, moved the original B distractor to C and the original C distractor to D to balance.

Modified stem and all distractors Editorial only to prevent the time line from moving past the first of the next month. All listed dates were revised to be 2 days earlier. This will prevent testing the applicant on how many days are in a given month. Question is otherwise unchanged.

Approved 05/14/09 RFA 028G2.4.49 91 M 2 X X Y Y E What is the difference between ABI and manually shutting off Train 1A purge and shutting dampers? It would appear the A and C are the same and B and D are the same.

CAF RFA 03/25/09 The ABI should have shut down containment purge. The RED light indicates that the ABI failed to trip the purge fan. The purge supply fans share a common suction plenum with the AB supply fans. Re-initiating ABI would be manually initiating the same initial signal, where manually stopping fans and closing dampers would be from different controls.

Rewrote distractors, added bullet to stem for clarification of LCO entry time.

5/06/09 approved RFA 055G2.1.19 92 M 2 Y Y S No comment RFA 03/25/09 Modified the WOOTF statement 4/29/09 approved RFA 079A2.01 93 C 3 N Y U Per the KA, the malfunction must occur on the SAS not the IAS. It appears that the malfunction has occurred on the IAS.

Y FAC: Re-evaluate. Q is U pending re-evaluation.

S RFA 03/25/09 Discuss during prep week to better understand comment.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only Due to the air systems being all one in the same, this question is satisfactory 4/29/09 approved RFA G2.1.8 94 C 2 X Y  ? E Would you not expect an RO to know the answer to this one. This appears to be RO knowledge because they are basic procedural directives.

CAF: Please re-verify SRO only RFA 03/25/09 Discuss during prep week. The question condition involves an off normal alignment of the DG and the room ventilation fans. Based on temperature, the DG may be emergency started thus running without normal protection.

Modified stem and 2 distractors. Question was determined to be SRO only.

4/29/09 approved RFA G2.2.15 95 M 2 X Y Y E A breach permit requirement for a NO on 30 days is not plausible. However, a procedural reason for a breach permit may be thus requiring a more in-depth knowledge vs. a simple yes or no.

Replace second part of distracters.

RFA 03/26/09 Added reason for a breeching permit being required or not required to second part of each distractor.

Modified stem and 1 distractor.

4/29/09 approved RFA G2.2.42 96 C 2 X Y Y U A mode change with a safety system inoperable in not permitted and this is common knowledge. Distracters A and B are NP.

This Q is U because 2 distracters are NP RFA 03/26/09 Modified question to require knowledge of RWST T/s and the use of LCO 3.0.4b which allows mode change with while relying on an LCO.

Modified stem.

4/29/09 approved RFA G2.3.4 97 C 3 X Y  ? E Increase the expected dose to a values close to the emergency exposure limit to lend more plausibility to distracters C and D. Can the RP supervisor authorize any kind of exposure?

If not, then change RP Sup to SED and re adjust the question.

Re-verify that this is NOT RO required knowledge too RFA 03/26/09 Replaced question.

Modified stem.

4/29/09 approved RFA G2.3.15 98 M 2 X Y Y E* If choice A was correct, C would be correct also.

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q#/ LOK LOD (C/A) (1- Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation

5) Focus Dist. Link units ward K/A Only If choice B was correct, Choice D would be correct also.

Therefore, there are 2 NP distracters. This can be corrected with a minor change to the stem. Add in 1. the minimum fission product barriers RFA 03/26/09 Changed 1 to read the minimum fission product barriers that have a loss or.

Revised stem and 2 distractors 04/30/09 approved RFA G2.4.35 99 C 3 X X Y Y E CAP the word not in (1). I missed it at first. The distracter analysis in D states that this method of Pzr heater control is plausible because aux control circuits do exist. D is the correct answer. Is this plausibility statement supposed to go with distracter C or A and B?

When is Only manual control available if any? Unless there is a condition, only manual control is available may not be plausible.

CAF to re-verify plausibility.

RFA 03/26/09 Capitalized the not , removed plausibility statement from the correct answer, added information to distractor A and B to identify that when AUX is selected only manual control is available on some components.

Modified 2 distractors Added backup heater nomenclature.

5/19/09 approved RFA G2.4.38 100 M 2 Y Y S No comment RFA 03/26/09 Replaced question due to overlap created when another question was changed.

05/19/09 approved RFA

Question Status S (31 ) Question is Sat with NO Comments S* (1) Week but acceptable E (36) Editorial Changes Needed E* (8) Technically the Q is U but a word or two will fix (usually a psychometrically flaw)

U (24) Question is Unsat