ML12240A306

CP-2012-06 Draft Written Exam Comments

ML12240A306
Person / Time
Site:	Comanche Peak
Issue date:	06/11/2012
From:	NRC Region 4
To:	Luminant Generation Co
References
Download: ML12240A306 (17)

Text

ES-401 CP-2012-06 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 B S B, CR, 41.7 D, CR, 41.3 - If seal injection flow is increased, the seal bearing temperatures will decrease, and vice versa. The applicant knows this without any system specific knowledge, so he/she could immediately eliminate disctracters B and C. With what is E given in the question, the second parts of these distracters are implausible.

2 H 3 X N S B is plausible if the lower seal fails due to the reduced flow that would go to the bearing. Seal construction knowledge. C is plausible if the flow through the thermal barrier heat exchanger increase is enough to cause the bearing temperature to decrease. Revised explanations to clarify.

C, CR, 41.7 - Is it standard RO level of knowledge to remember content of Caution E

3 H 4 N statements?

S Caution statements are required in their learning objectives for the SOPs.

4 H 3 N S C, CR, 41.10 - Question on Step 9 of EOS-1.2B - the correct action is stated in the Caution statement prior to the continuous action step. However, the procedure step Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only doesnt direct the operator to start RHR pumps from a logical procedure adherence perspective. Discuss.

Because the Caution supports a continuous action step, it is supposed to be enforced at all times in the procedure. No changes.

5 F 4 N S D, CR, 41.5 6 F 3 B S B, CR, 41.7 C, OR, 41.3 - SOP-109A says that the PRT rupture disks rupture at 91 psig. Is this based on containment being at 0 psig or some other value? Since a containment pressure is given in the question, and the discussion is appears to be based on 7 H 3 X B S differential pressure, we need to ensure that all units are consistent.

Discussed, the procedure should be enhanced to say 91 psid vice 91 psig. Question is satisfactory.

8 F 3 B S B, CR, 41.10 9 H 2 N S A, CR, 41.5 10 F 2 N S A, CR, 41.7 11 H 2 N S C, CR, 41.7 D, CR, 41.7, .10 - The K/A tests the ability to predict the impacts of an inadvertent ESFAS actuation AND based on those, tests the ability to use procedures to address the situation. The given question attempts to address the first part of the K/A statement, but doesnt address the second part at all. Also, it is hard to see how distracter A is plausible (nothing changes with CCW flow, and no indications given of issues with the RCPs, so trip the RCPs).

E Also, the applicant only needs to know the first part of answer to make the correct 12 H 2 X X M choice. Knowledge of the status of RCPs is not needed.

S The second part of the question deals with their knowledge that if there is an auto alignment of a system that doesnt happen, the operator is always responsible to take immediate action to place the system in the correct alignment. (PROCEDURE REFERENCE - ODA-102, Section 6.3)

With B and C, will change to have CS flow vice number of pumps. Will add a note in about sub-cooling margin to clarify. Two by two format.

13 F 2 B S C, CR, 41.7 14 H 2 B S A, CR, 41.7 15 H 3 X M S A, CR, 41.7 - ABN-709 Section 3.2 says that a steam header pressure high condition Page 2 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only will cause all steam dumps to open in steam pressure mode. In the question, it says that some will open. Verify whether this is an issue with the question content or the procedure text.

Tave below 553, P-12 interlock is in play. The question is correct. The procedure could be enhanced.

The original bank question used for the modification appears to have at least one error (circulating water pumps designated for Unit 1 instead of Unit 2). See if it needs to be addressed.

Will be fixed. No changes to question provided are necessary.

C, CR, 41.7 - NUREG-1021, Appendix A, Section C.2.m discourages the use of distracters that do not follow grammatically from the question stem. In this case, distracters A and B do not follow grammatically in answering the question what is the reason for ensuring an operable battery charger is aligned? They say loss of load XYZ, not really explaining why. Since they look different for this reason, and dont E really answer the question, they can be screened out as incorrect answers based on 16 H 3 B format, not on applicant knowledge.

S Will change.

On answers C and D, they say something may inadvertently open. The reason for ensuring a battery charger is aligned is to ensure that they do not inadvertently open.

The wording could use some adjustment.

Revised text.

B, CR, 41.7 - The K/A statement deals with the ability to manually operate AND monitor MFW turbine trip indications in the control room. The question involves monitoring the automatic operation of the MFW pump trip only, which is associated E

17 H 2 X N with the A3 K/A category. This question does not fully address the K/A statement.

S Rewritten to a) give conditions to need manual trip (operations), and b) based on that what trip indications the operator would expect. Keep separate from Scenario 2, Event 4.

D, CR, 41.7 - If the answers to the question all state a valve position based on the situation, and the reason for the alignment, the stated question should ask for both E

18 H 3 N parts of the answer. Also, provide reference that shows that 1-HS-2211/12 is S interlocked with the HS-2484 or -2485 limit switches.

Reworded question to address and added reference material.

C, CR, 41.7 - The initial conditions refer to inverter 1PC4. The inverter is IV1PC4, E and there is a distribution panel 1PC4. To avoid confusion, ensure that the 19 F 3 X B S equipment labeling is correct. (Will fix labeling)

The question also says the plant wishes to continue to operate. At what power Page 3 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only and/or Mode? It should be more specific so that it can be tied to the Tech Specs.

This avoids any potential on a correct answer A. (Will add Mode 1 in cue).

The portion of the answers to 118 VAC Vital Distribution Panel 1PC4 should be moved up into the question statement since there are no choice options associated with it. (Will address)

Revised.

D, CR, 41.7 - If the minimum DC voltage for equipment operation is 105 V, then isnt this too the minimum voltage needed for plant parameter instrumentation in the Control Room? Unless the procedures say different, answer A is correct as well.

E 20 F 3 X N Attachment 17, page 13/30, Step 13 basis talks about basis for voltage is to maintain S

4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> supply and ability to start the EDG, not maintaining instrumentation. Research has shown that PAM instrumentation needs a minimum of 100 V, so answer A is incorrect. Changes made.

D, CR, 41.8 - References are provided that show why the distracters are incorrect.

However, we dont see a reference that proves why the correct answer is correct.

21 F 3 X B S Provide justification that the low starting air condition doesnt adversely affect the normal starting circuit.

Reviewed again. No changes necessary.

B, CR, 41.11 - The trend presented for the Main Steam Line N-16 Radiation Monitor provided is decreasing. The question asks whether this trend is expected/unexpected, and why. For distracter A, the explanation for the trend says that because of decreased steam flow past the detector, the radiation monitor indication decreases. Therefore, it justifies the reading as expected, vice unexpected E as stated in distracter A. This is conflicting and implausible.

22 H 2 X B S Fixed A to make plausible.

If there is a basis for a SIAS, there is going to be need for radiation monitors throughout the plant. It isnt plausible that a radiation monitor is isolated from detection on a SIAS. This is directed at distracter C.

Some of the rad monitors isolate on a SIAS. Explanation supports this.

B, CR, 41.7 - Distracter A is plausible if there are other loads being sequenced on from the Blackout Sequencer at times equal to or greater than 45 seconds. Verify.

E 23 H 3 X X N It runs for 110 seconds, and additional loads will shed. Will add reference.

S On the second cue point, specify reactor trip vice trip.

Revised.

24 H 3 B E D, CR, 41.7, .10 - Repeat from 2010 Exam - It would enhance the plausibility of distracters B and C if the header flow indication matched one provided in the ABN Page 4 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S referenced. There are actions associated with 18000 and 16456 gpm, but none are auto starts.

Changed distracters B and C to use 16,456 gpm.

A, CR, 41.7 - You only have to know the status of IA Compressor X-01 to be able to E answer the question. If you know that this compressor is running and loaded then 25 H 3 B knowledge of IA Compressor 1-02 is not needed.

S Adjusted distracter C to address this.

26 H 3 N S D, CR, 41.10 A, CR, 41.5 - The K/A is about a loss of containment integrity, which implies that containment integrity was in place. The question starts from the point that containment is not set. K/A mismatch.

In shutdown conditions, in most cases, containment is not set. If it was written to say E set containment, there would be challenges in coming up with plausible distracters.

27 F 3 X N The question addresses contingencies for what needs to be done to set containment, S

which addresses the K/A.

For distracters B and C, the Explanation explains why they are plausible, but doesnt explain why they are incorrect. Clarify.

Explanations revised.

D, CR, 41.9 - The provided references dont justify why answer A is a plausible distracter. Information on when the Containment Pressure Relief System is to be E

28 H 3 X M placed in service is needed. In addition, the explanation for answer A has an S incorrect statement about temperature (legacy from the bank question).

Revised.

29 H 3 N S B, OR, 41.1 30 H 3 B S B, CR, 41.6 31 H 2 N S A, CR, 41.5 32 H 4 N S B, CR, 41.5 B, CR, 41.13 - Knowing/performing the actions described look like SRO level of knowledge. Not sure that an RO would be expected to know this.

E The second and third answers are trained on for all fuel handling operators prior to 33 H 3 X M fuel handling, including RO. The first answer can be answered correctly by systems S

knowledge of correct location of fuel transfer cart and where it needs to be with the transfer tube gate valve open/closed. Satisfactory.

The applicant only has to know two of the three parts given for each possible answer Page 5 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only in order to come up with the correct answer. Three part answers are very difficult.

Recommend changing this to two part answer.

Revised.

For the applicant to take actions per the referenced ABN procedure, he/she has to determine if the refueling cavity level and the spent fuel pool level are equal or diverging. Provide some indication of spent fuel pool level in the initial conditions.

Not necessary.

A, CR, 41.5 - NEED TO REPLACE QUESTION - A failure of the of the controlling 34 H 3 B S S/G level controller high is also the cause for Event 2 in Scenario 2.

Revised question to make different than Scenario 2 Event 2.

B, OR, 41.14 - With the graduations on the graph at 20 psig/division and 0.2 pot setting, how can the proper pot setting be determined to 3 significant figures (2.07 for example)? It could be argued that the answer is about 3.8. In this case, it could be 35 H 2 M S argued that there is no correct answer for this question.

The applicant can come up with 3 significant figures if he/she determines the linear equation from the graph. Also, actual pot settings are 3 significant figures. No issues.

36 F 2 M S C, CR, 41.1 A, CR, 41.13 (Editorial) The discussion of a couple of the distracters involves WGS083, which doesnt appear to be one of the choices. See the Explanation 37 F 2 B S section.

Fixed.

38 F 2 B S D, CR, 41.8 B, CR, 41. - The question involves a situation where the control room does not intentionally progress to, meaning they dont start system in error. Therefore, the EOP doesnt directly state, If started in error, do this With that, the applicant has to come up with a means to address this that is covered in EOP-0.0B. Action has to be taken to address the loss of CCW to the RCPs. This can be done by tripping the RCPs (Step 7), or by resetting Containment Spray and realigning CCW to the RCPs U per Attachment 9. An applicant could argue that distracter C is correct as well, 39 H 3 X B S because it is a means to address this situation in the procedure, and it is for the right reason.

Revised cue so that distracter C is incorrect based on procedure usage rules.

For distracter D, justify why spraying water in containment would be the basis for securing RCPs. If this is plausible, it seems that a lot of other equipment may be affected/secured when Containment Spray is used. Page 6 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only RCPs are not environmentally qualified, unlike some equipment in containment.

They wouldnt stop spray on them to save the pumps. There may be a misconception that they need to save the pumps, but they are not safety-related.

Acceptable.

40 H 3 M S C, OR, 41.14 C, CR, 41.10- It is difficult to see how a purpose for checking S/G level at a certain level would be to ensure level instrumentation is working correctly. The operator could read the indication, and it could be correct or not. Simply reading the gauges doesnt ensure that they are working correctly. This part of distracter D is E implausible.

41 F 3 B Recognized it wasnt plausible. Fixed it.

S As a format comment, the answers should be in a form that matches grammatically with the question? For example, Which is the purpose for checking S/G levels at XX? To ensure that S/G water levels are sufficient to cover a ruptured S/G tube.

Revised.

42 F 3 M S B, CR, 41.10 D, CR, 41.10 - Since the RCP trip criteria in ABN-101 are applicable in the EOPs per EOP-0.0A, Attachment 1.A, is there any conflict with this and a Note in FRC-0.2A that says that RCPs should not be tripped if normal conditions do not exist? One says trip if you meet certain criteria, the other says you shouldnt do it. This appears E

43 H 3 N to be an area where operators could be in conflict. Running a RCP that is degrading S with no seal cooling and high vibration can lead to another LOCA on top of the one that already exists in the question. This needs to be reviewed.

Added bases page which supports the fact that there is one answer: they cannot trip the RCPs in FRC-0.2A unless directly told to do so.

A, CR, 41.10 - Per ABN-105, Section 6.3, Step 1, if reactor makeup is not in a Normal Operating Mode, it says to establish an Alternate Operating Mode per SOP-104A/B. If an Alternate Operating Mode cannot be established, continue with the ABN. Two of the Alternate Operating Mode options available (Section 5.2) in SOP-U 44 H 3 X M 104A are Manual Dilution and Manual Alternate Dilution. There is no prohibition S stated in ABN-105 for Alternate Dilution usage. If the applicant says that either Manual Dilution (answer A) or Manual Alternate Dilution (answer C) is correct, they are allowed by procedure. There appears to be two correct answers.

Recognizes that there are two correct answers. Revised answer C.

45 F 3 B S D, CR, 41.10 46 F 1 X B S C, CR, 41.7 - The K/A statement deals with knowledge of reasons for auto actions in Page 7 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only the CCW system resulting from an ESFAS actuation as they apply to the loss of CCW. There are no given conditions in the question that relate it to a loss of CCW system situation.

CIAS B is a subset of the ESFAS signal, which affects CCW flow, and causes a loss of CCW flow to components. Accepted.

The question provided is not very discriminating in itself. The reason for a containment isolation signal is to ensure that the CCW system is not a path to release from Containment? An applicant could tell the three distracters are wrong just by knowing the basis for containment isolation in general.

This is the only path to address this question. Accepted.

47 F 2 B S B, CR, 41.10 D, CR, 41.1 - This is repeat question from the 2010 exam. From review of the 2010 exam, the match is Question 54. However, they are not the exact same question.

Information was provided about containment pressure and trend in the cue that is not given in this version. Also the answers vary some in the text. Since one of the answers deals with evaluating containment pressure, justify why it is not provided.

E Changes were made on a bank question to improve the quality question. Classify as 48 H 3 X X B a Bank question.

S Discuss expectations for RO versus SRO questions on EOP procedure transitions/CSF tree usage.

The RO has level of knowledge to stay in the procedure in the correct answer. The RO has level of knowledge that they cannot change procedures with power greater than 5%. Therefore, it can answered with RO level of knowledge.

49 F 3 N S C, CR, 41.10 A, CR, 41.10 - If the applicant only knows that RCPs are not available because of E loss of offsite power, he/she can eliminate answers B and D as distracters. One of 50 H 3 X N them needs to test some other knowledge.

S Changed D to another plausible answer - using steam dumps.

51 F 3 B S B, CR, 41.7 C, CR, 41.10 -For answer B, the Explanation says that it is plausible if the applicant incorrectly thought that the pump failed to minimum speed. There are no indications E in the question that talk about the TD AFW pump having any malfunctions, so there 52 H 2 N isnt the availability for it to be judged as malfunctioning.

S There is a level of knowledge on TDAFW steam admission valves and their failure position on SBO - loss of instrument air. They have to know what the system does, and they need this withstanding any TDAFW pump failure. Lesson notes added to Page 8 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only help with explanation. Acceptable.

For the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> time limitation, is there a 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> time limit on some other piece of equipment in this situation? Just trying to understand why the 4 hour4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> time limitation is plausible.

Explanation is provided. No issue.

D, CR, 41.10 - Since there are 3 reasons stated for setting AFW flow to 100 GPM E per S/G, recommend changing the question to say a reason vice the reason for 53 H 3 X M that action.

S Revised.

54 H 2 B S C, OR, 41.10 A, CR, 41.7 - The K/A statement asks for knowledge of reasons for actions contained in the EOP or abnormal procedure for loss of vital AC instrument bus. The question U

55 F 3 X N tests the applicants knowledge of cause/effect with the CVCS, possibly the ability to S monitor a transient, but not this K/A statement.

Question does not address the K/A. Provided another question.

A, CR, 41.10 - The K/A statement tests the knowledge of reasons for reactor/turbine trip criteria at the site during grid disturbances. This question tests knowledge of the trip criteria, but not the reasons for the trip criteria.

In the supporting material, the basis for the criteria to trip the reactor instead of the U

56 F 2 X N turbine above 50% power needs a reference.

S To write this to the basis, it is found in an IEEE standard that supports the ERCOT guidance. Far beyond the level of knowledge for RO/SRO. Resampled as AK1.02 and provide another question.

Need CR on training for AK3 on 077 topic to address topic in ILT.

57 H 3 N S B, CR, 41.5 58 H 3 B S B, CR, 41.7 A, CR, 41.9 - The K/A statement asks for knowledge of the reasons for different inputs that will cause a reactor building evacuation. As given, this question tests U

59 H 2 X N knowledge of the inputs causing a reactor building evacuation, not the reasons for the S inputs.

Replaced question to address K/A.

D, CR, 41.7 - (Editorial) The references for the question are Steps 2.3.3 and 2.3.4 of 60 F 3 B S ABN-304. That section deals with a Circulating Water pump trip. The steps for lowering vacuum are Steps 3.3.3 and 3.3.4. Page 9 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Revised.

A, CR, 41.8 - This question can be answered correctly by knowing only the first part of the answer. The applicant doesnt have to have correct knowledge of the second part to get this right.

Revised.

The explanation, nor the referenced procedure (SOP-802), does not explain what the basis is for placing control room ventilation in Isolation Mode. The Precaution in the E procedure says that in this case, manual operator action is needed with a smoke 61 F 2 X M alarm to make any changes. Based on the initial conditions, a) why is alignment to S

Emergency Recirculation needed, and b) why is shifting to Isolation Mode necessary? The criteria driving these changes arent provided.

Addressed.

Provide a copy of the original bank question so it can be compared to the proposed Modified question.

Provided. No issues.

C, CR, 41.11 - Stem states monitor number is FFL-160. ALM gives this indication, 62 F 2 N S FFLu-60. Ensure correct number is used in stem to alleviate any potential confusion.

Question is adequate as is. No changes.

C, CR, 41.10 - Based on reading the RNO for Step 12a of EOS-1.1A, the fact that no steam generator is faulted is enough to allow transition to EOS-1.2A. The other reason is if SI pumps cannot be stopped after a faulted S/G depressurizes due to continuing decrease of RCS pressure below 1700 psig. It reads as if to say. It says to transition if there is no S/G faulted OR still depressurizing AND some indication of U RCS condition. This does not clearly tie to the procedure, so it looks like there is 63 H 3 X N S presently no correct answer given. Resolve.

Resolved.

Second, use of negatively stated questions is discouraged per NUREG-1021, Appendix B, Section C.2.e.

Revised.

64 H 3 N S A, CR, 41.14 A, CR, 41.1 - Is it procedurally allowed, or is it consistent with Tech Specs, to stay at E power with the plants means of rod control tagged out for repairs? Auto rod control 65 H 3 X N is not allowed in this situation. It seems that the initiating conditions are implausible.

S Revised to address concerns.

66 F 2 B S C, CR, 41.10- Justify how distracter A is plausible. Would an applicant think that it is Page 10 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only plausible to leave/abandon watch when his time is up?

It deals with their knowledge of the 2 hour2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br /> rule and its exceptions. Leave as is.

C, CR, 41.10 - This question can be answered by only knowing the first OR second E part. It can be answered correctly without knowing half of the answer is 67 F 3 N correct/incorrect.

S Revised to focus on second part of the question.

B, CR, 41.10 - Provide the basis documentation for 2 SROs being able to approve E marking a procedure step N/A if a clearance is installed on a component to be 68 F 2 X B operated. It is not evident in STA-201.

S Reference added to support distracter A.

D, CR, 41.10 - Upon review of ODA-403 and OWI-103-3, it isnt clear that a Shift E Manager can suspend the Locked Component Deviation Log. Show where the Shift 69 F 3 X M Manager has this authority, and when it applies. For distracter B.

S Supporting reference added.

70 F 2 B S A, CR, 41.10 B, CR, 41.13 - The applicable procedure step to this situation (STA-603, Step 6.2.10, not Step 6.3.11 for gaseous effluent releases) says a new release permit is needed if a release was terminated due to a valid high radiation alarm. Therefore, if it is found that the high radiation alarm is invalid, it implies that the discharge can be started with the current permit. Based on this, distracter A is also a correct answer.

Revised.

It is not stated in STA-603, but it is expected that a high radiation alarm received, valid or invalid, would drive a Condition Report to be created to find the cause of the U issue. It is stated in STA-603 that all liquid radioactive releases have to be approved 71 F 3 X X B by the Shift Manager, so both parts of distracter C could be argued as being correct S

as well.

Agree it is a second correct answer. Revised to fix issue.

The only approvals spoken of in STA-603 are from the Shift Manager and the Chemistry Manager. Revision 20 of the procedure removed approvals for the RP Manager, so it is unclear if that is plausible.

Revised.

A Condition Report is needed for STA-603 to enhance language addressing how to handle alarms that are not valid.

72 F 2 N S B, CR, 41.11 Page 11 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 12 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only C, CR, 41.10 - Use of negatively stated questions is discouraged per NUREG-1021, 73 H 3 B S Appendix B, Section C.2.e. (inappropriate = not appropriate).

Based on further review, the question is acceptable.

74 F 3 B S D, CR, 41.10 75 F 3 B S A, CR, 41.7 B, CR, 43.5 - The question provided deals with EOP usage during a Loss of All Feed event at Unit 1. Per the K/A statement, the question is supposed to test the ability of the applicant to explain variations between controls, systems, instrumentation and 76 H 4 X N S procedural actions on a LOAF comparing Units 1 and 2.

There is a difference in the S/G level based on Unit 1 or 2, so the applicant needs to have the ability to account for variances in this between the units. Accept.

77 H 3 B S B, CR, 43.5 78 F 3 N S C, CR, 43.1 79 H 3 N S A, CR, 43.2 B, CR, 43.2 - The first required action is to immediately initiate action to restore the affected RHR loop to OPERABLE status. The set up of the question says to immediately restore BOTH trains of RHR to OPERABLE status These are not the same things. It can be argued since that works in tandem with the answer choices, that there is no correct answer for this question. Resolve.

U 80 H 3 X N Resolved.

S Will make the wording consistent with the Tech Specs so there are correct and incorrect answers.

Recommend changing the verbs in the question stem from past to present tense.

Revised part of the bullets to address this.

A, CR, 43.5 - Use of negatively stated questions is discouraged per NUREG-1021, Appendix B, Section C.2.e.

Second, the intent of the if Auxiliary Feedwater flow was previously throttled statement is meant to associate the answer with other procedures that drive a U change in AFW flow (based on review of the bank question). However, with the 81 H 4 X M S structure of the sentence, the applicant could interpret that to mean which procedures would NOT require entry into FRH-0.1A if AFW flow was throttled could mean assume that the AFW throttling that is stated didnt take place; which of these procedure entries wouldnt require entry into FRH-0.1A?

The wording of the question could be interpreted in more than one manner, and this Page 12 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 13 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only is complicated with the negatively stated portion of the question.

Also, the applicant can make the correct choice by knowing only one part of the answer.

Revised.

82 H 4 N S D, CR, 43.5 D, CR OR, 43.5 - Per ABN-106, Section 3.3, Step 1a. RNO, it says to initiate a power reduction to less than or equal to 50% in 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. Answer D says to reduce power to less than 50% in 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />. These two items are different, so the correct answer is incorrect.

Revised reference and question to address. A CR will be written for the procedure enhancement.

U 83 H 1 X M If the reference cited is given with the question, the applicant can direct lookup the S first part of the answer response and find the correct answer without displaying any knowledge on the second part.

Reference removed.

For distracter A, procedure Step 4a. RNO says to have the unit in Mode 3 in 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.

With the discovery at 1200, the time would be 1800 vice 1600.

Revised.

A, CR, 43.5 - On 2010 exam - (Editorial) The explanation for answer B says that containment pressure is 16 psig and rising. The question says it is 14 psig and 84 H 3 B S lowering.

Fixed.

85 H 3 B S B, CR, 43.5 86 F 3 N S D, CR, 43.5 A, CR, 43.5 - This question could be answered correctly with knowledge of RHR operational range limitations (systems) and the Tech Spec LCO and Applicability Statements. This is typically RO level of knowledge. Verify what level of knowledge 87 F 3 X N S expectations are for ROs with respect to this Tech Spec onsite.

Reviewed by facility staff, and the Exception Note in LCO 3.4.12, is not considered RO level of knowledge. No changes needed.

B, CR OR, 43.5 -This question is marked needing a reference for the applicant E (Steam Tables). Based on the information given, it doesnt appear necessary. The 88 H 3 X X M original bank question was justified in providing for a copy to answer.

S Changed from OR to CR. Page 13 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 14 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B, CR, 43.5 - The K/A statement deals with the applicants ability to predict impacts of malfunctions or operations involving unloading the EDG prior to securing it. The question deals with impacts of running an EDG unloaded for an extended time, past the design time limit. This may be addressed under the A2.06 K/A, but it is different than the selected K/A A2.04.

A SRO question cannot be written to address A2.04. Reselected to A2.06.

U 89 F 2 X X N For the given question, it asks what the required action with the EDG is and what S

procedure to use to accomplish it. Reading ABN-602, Section 8.3, Step 8 RNO, it is the intent to stop the EDG. In the case where it was run unloaded for greater than 30 minutes, it has to be run loaded at 3.5 MW for 60 minutes per SOP-609A. However, after the loaded run, the intent would be to secure the EDG using SOP-609A.

Therefore, distracter D could be argued as being a correct answer as well.

Changed to say immediate stop vice stop, 90 H 3 N S C, CR, 43.5 C, CR, 43.5 (On 2011 Exam) - Question can be answered by RO level knowledge in the first part without having to have knowledge of the procedure required in the second part.

91 H 3 X X B S Based on discussion, the SROs are the one that coordinate how to do purging activities. The RO may know some of this information, but they are not required to having this level of knowledge. Leave as is.

B, CR, 43.5 - What level of knowledge is required for RO/SRO applicants for Reactor Trip criteria other than in the EOPs?

The ROs are not expected to know this reactor trip criteria for this.

92 F 3 N S (Procedure Comment) Procedure ALM-0072A does not refer the control room to procedure ABN-304 for the alarm given in the question (Window 3.5). However, ABN-304, Section 4.0 says an entry condition is receiving this Window 3.5 alarm.

The procedure should be revised.

Will review for procedure revision.

C, CR, 43.5 - The K/A statement addresses the applicants ability to identify post-accident instrumentation associated with the Area Radiation Monitoring System in use of emergency procedures or the Emergency Plan. The given question ability to determine Tech Spec implications for a failed radiation monitor during plant startup.

E 93 H 4 X X B This does not match the K/A.

S To answer this question, the applicant has to know that the failed rad monitor is a PAM instrument, and then they have to select the correct Tech Spec action for this.

Question does address the K/A statement.

In the question given (bank question), it says that answer C is correct. In the cue, it Page 14 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 15 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only says the RCS temperature is 340F (Mode 4). The given correct answer says that an LCO action statement is entered after entering Mode 3. Per Tech Spec 3.0.4, this is not allowed unless justified by a plant risk assessment, which isnt stated. Therefore, answer C is incorrect.

Tech Spec LCO 3.0.4 will allow entry into a mode if the LCO Condition Actions allow the plant to operate indefinitely at that mode with the condition (3.0.4a). With one PAM instrument inoperable, they would take action to restore it to operability in 30 days (Condition A). If they dont meet this, Required Action in Condition B drives them to file a report outlining the repair plan in 14 days, but does not drive the plant to shut down/reduce mode. Therefore, this mode change is allowed. No change to the question necessary, but basis information revised to explain this.

(Tech Spec comment) In Tech Spec 3.3.3, Condition D, it says to enter a condition referenced in Table 3.3.3-1. When you refer to this table, it has a list of conditions to enter, but the title says it is based on entry from Required Action E.1. It based on entry from Required Action D.1. The Tech Specs need to be corrected. LCO Action Statements Conditions E and F correctly refer to Condition D.

Will put in a CR for this.

D, CR, 43.2 - Since there are no actions in the TRM that drive the plant to be in Mode 2 by a certain time in action statements, justify why saying placing the unit in Mode 2 within 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> is plausible.

E 94 F 3 X B Revised to add procedure reference to justify Mode 2 plausibility.

S Review the Explanations for distracters B and C. There are errors in the justifications for them being incorrect that need to be fixed.

Revised.

B, CR, 43.5 - This question can be answered solely on knowledge of conditions with instrumentation that necessitate the entry into an abnormal operating procedure U (ABN). This is RO level of knowledge.

95 H 3 X B S Can answer with RO level of knowledge. Changed to make it focus on which procedure to enter. SRO level of knowledge is which ABN to enter. ROs not taught this.

A, OR, 43.1 - If the reference provided for this question is TRM 13.10.31 only, the applicant can immediately see that the action in distracter D is not one of the choices provided. It not being in the TRM section makes this implausible.

U The reference is given, but in a packet that doesnt correlate it directly to this 96 H 3 X X M S question. Therefore, the applicant has to find it in the packet. OK.

For distracter C, it is not recommended to include an answer that says no impact or nothing happens. The applicant can immediately screen this out as a distracter not on level of knowledge but merely by the fact that something has to be impacted for thi Page 15 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 16 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only question to be asked. Second, saying that there are no impacts on hydrogen recombiner operations when there is a 30 day action to restore related instrumentation to operable status means there is some impact on its operation. The answers contradict each other. The distracter is not plausible.

Also, the applicant only has to know the first part of A to answer the question.

Knowledge of restoring the channel to operability would not be needed.

Revised to address concerns with question.

C, CR, 43.5 - Are there any credible cases where no work may be performed with no impairments allowed? I didnt see any in IPO-010A.

E Are there any instances where there are no limits on the penetration impairments put 97 F 2 X B S in place, but there is no requirement to have pre-approved temporary seals at the work site?

Clarified by adding statement about status of fuel in the vessel.

98 H 3 N S C, CR, 43.5 99 F 2 N S D, CR, 43.4 100 F 2 N S A, CR, 43.5 B= 33 F= 33 E= 26 RO TOTALS: M= 12 H= 42 U= 7 Additional Notes: Percentage unsatisfactory questions submitted is 9.3% (7/75)

N= 30 B= 8 F= 9 E= 4 SRO TOTALS: M= 4 H= 16 U= 6 Additional Notes: Percentage unsatisfactory questions submitted is 24% (6/25)

N= 13

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N

2. Chief Examiner comments are indicated in blue.

3. Average difficulty is __ 3___ on the RO exam and _3__ on the SRO exam.

4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 4 43.1 = 2 41.2 = 43.2 = 3 Page 16 of 17 OBDI 202 - IOLE Process

ES-401 CP-2012-06 17 Form ES-401-9 41.3 = 2 43.3 =

41.4 = 43.4 = 1 41.5 = 7 43.5 = 20 41.6 = 1 43.6 =

41.7 = 21 43.7 =

41.8 = 2 41.9 = 2 41.10 = 25 41.11 = 3 41.12 =

41.13 = 2 41.14 = 3

5. The answer distribution is: RO / SRO A = 19 (25%) / 6 (24%)

B = 20 (27%) / 8 (32%)

C = 19 (25%) / 6 (24%)

D = 17 (23%) / 5 (20%)

6. There are 6 questions with attachments provided. Page 17 of 17 OBDI 202 - IOLE Process