ML13024A364

2012 Point Beach Nuclear Plant Initial License Examination Written Examination Review Form ES-401-9

ML13024A364
Person / Time
Site:	Point Beach
Issue date:	12/17/2012
From:	Division of Nuclear Materials Safety IV
To:	Florida Power & Light Energy Point Beach
Zoia, Charles
Shared Package
ML11354A208	List: ML13024A360 ML13024A364 ML13028A063 ML13035A242
References
Download: ML13024A364 (14)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1+ H 2 B S NRC: 1/2 twice style noted - satisfactory as-is.

2 H 2 N E NRC: For consistency, lower should be bolded and in all caps.

Licensee: Bolded and capitalized both instances of lower in the choices so that they are consistent.

3 F 2 B S 4 F 2 B S NRC: (2003 NRC Exam) 5 F 2 B U NRC: A) As written, there is no correct answer because it takes 2 things to satisfy the ATWS procedure requirements. B) There are 2 E

implausible distractors (A and D). After discussion with facility, changed to enhancement (E).

Licensee: The background for Step 19 of CSP-S.1 stated that two things were required to verify the reactor is subcritical. This question asked for one of those two things, so there was only one correct answer - C. In addition, D is plausible because it was a common misconception that a lowering startup rate indicated subcriticality.

To further clarify the question and address a concern with the A distractor, we made minor changes to the stem and constructed four choices using combinations of two true and two false statements.

Also added the CSP-S.1 background to the references.

6 F 3  ?  ? N  ? NRC: Form-401-5 cites an objective number but the attached question does not show it. Is this information RO knowledge E

required from memory?

Licensee: Operator Time Critical actions are necessary to mitigate an event/accident and ensure safe continuous unit shutdown. They include manual actions that are credited in the analysis, UFSAR, and Technical Specifications. The SGTR time critical operator action to prevent steam release after 30 minutes is important to safety, covered in operator training, and required from memory. We have also added the objective and the Operations Manual references.

7 H 3  ?  ? B  ? NRC: (2005 NRC Exam) Verify that this question is RO knowledge required from memory.

E Licensee: This concept is important for the Reactor Operator to understand and expected to known from memory. PBNP objectives such as from LP0465 support this knowledge for RO. The training material is clear and emphasized in simulator training. We added the lesson plan as a reference.

Page 1 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

Page 2 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 8+ H 3 B S NRC: (2005 NRC Exam) 9 H 3 B E NRC: Form-401-5 cites an objective number but the attached question does not show it.

Licensee: Updated the submitted question with the objective text.

10 H 2 N S NRC: 1/2 twice style noted - satisfactory as-is.

11 H 2 N E NRC: LOK=F. If the applicant recalls the indications of loss of yellow bus, the correct procedure becomes apparent.

F Licensee: Changed question to low cog.

12 F 2 N S 13+ H 3 B S 14 H 3 N U NRC: Why would this question be required from memory? (EOP-1, Step 10.c RNO).

S Licensee: Satisfactory as-is: the question can be answered with system knowledge that we require the RO to know from memory.

15+ H 2 B S 16 F 2 B S NRC: (2005 NRC Exam) 17+ H 2  ? B  ? NRC: Please clarify what Distractor C and its explanation are saying, and why Distractor D is incorrect (See Step 40 RNO).

E Licensee: Feed and Bleed was an ineffective alternative strategy from the WOG ERGs. The difference was confused by some operators. D distractor is not correct, because the step 40 RNO does not secure the Bleed and Feed alignment, but rather takes additional actions to establish SG injection flow. We addressed the concern with C by removing feed and bleed. We also revised the B distractor for symmetry with C and enhanced the explanation for the D distractor to make it clearer.

Page 3 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 18 F 2 N U NRC: Either A or B must be true the way these distractors were written - the 13.8 KV breakers either have a trip function or they E

dont, the other choices are then meaningless. Also, the distractors need to address the breakers ONLY per the stem.

Licensee: The choices were in the pattern of All open, All remain as-is, Some open, One closes. The correct answer was All remain as-is. Took breaker out of the stem to emphasize system operation.

Also, to make D distractor match the others more closely, it was re-worded.

19 F 2  ? N  ? NRC: Verify all aspects of the correct answer are RO required knowledge (TS Bases/LCO Actions).

E Licensee: One hour or less TSACs are required RO knowledge. In addition, this is an AOP procedure action the RO should know. The reason given in the second part of the answer (to meet the safety analysis) is covered in reactor operator training in LP3339 RCS Technical Specifications (Objective 057.01.LP3336.017). We also removed the second part of the answers to simplify the question construction.

20 H 2 N S 21 H 2 N S 22+ H 2 B U NRC: The BYPASS annunciator is too obvious and should be included in all lists. Other combinations should be used instead.

Licensee: Modified the question to place NIS Trip Bypassed in each distractor and adjusted the responses and explanations for balance.

Also corrected a typo on ES 401-5 in K/A #.

23 H 3 N E NRC: Using the given reference should be omitted from the stem of the question. The applicant should be able to determine if any supplied reference is needed. (GENERIC).

Licensee: Deleted using the question reference from the question.

24+ F 3 B S NRC: (2005 NRC Exam) 25 F 2 B S Page 4 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 26 F 2 B E NRC: A) MFW, CCW and RMUW are also listed in procedure CSP-Z.2, so all could be considered correct responses (given the non-specific way the stem is written). Modify the stem to seek the MOST LIKELY cause of flooding. B) The objective doesnt seem to match.

Licensee: Added most likely to the stem and changed the objective reference to one that more explicitly matches the question.

27 H 3 N E NRC: Distractor C is not plausible because adverse conditions previously existed by BOTH parameters. Distractor A proposed the incorrect understanding of setpoints with the correct RCP action.

Change to address CURRENT non-adverse conditions (correct) not requiring RCP trip (incorrect).

Licensee: Modified the D distractor for question symmetry and used partly true/partly false distractors. The explanations were also enhanced and the peak containment pressure was limited to 4.5 psig to test knowledge of the radiation setpoint. Finally, we corrected the cutoff K/A statement on Form ES 401-5.

28 H 1 N U NRC: A) 1/2 twice style noted. B) it would not be plausible for either parameter to increase. It is too simple of a question as written -

F 2 B perhaps it could address a more indirect operational implication resulting from the changes to these parameters instead of directly addressing how they change.

Licensee: Unable to write a question to this K/A above GFE level.

Randomly and systematically drew a different K/A. Another question (Bank and LOD=F) was submitted and approved.

29+ H 3 B S NRC: (2005 NRC Exam) 30 F 2 N U NRC: It would not be plausible for RH-626 to close on a 1FT-626 failing low, which makes 3 distractors implausible.

H B Licensee: We submitted a different question for this K/A using more integrated plant knowledge, which the Chief Examiner approved for use. The replacement question (Bank) is High Cog.

31+ H 2 N S 32 F 2 B S NRC: (2007 NRC Exam) 33 H 3 N S 34+ H 2 B S Page 5 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 35 H 2 N U NRC: It would not be plausible for the MFRV Bypass Valves to be throttling or the MFIVs to be open on a SI actuation which makes 3 distractors implausible.

Licensee: Three of ten validators missed this question. The most common mistake was not remembering which channels feed which functions. They chose A because they thought the only result was a controller shift to single element operation. C was plausible if they believed that only a reactor trip occurred with the MFIV open. Also, D was plausible if they believed only a reactor trip occurred.

Modified the question stem and C distractor. C and D were now plausible choices if the student believed that reactor trip will occurred as a result of the failures (with and without AFW actuation). The question was approved with changes noted.

36 H 3 N E NRC: It would not be plausible for the valves to remain in their non-SI state on both valve pairs (not realizing an SI occurred?). Modify distractor A to a variation of the correct response.

Licensee: Modified distractor A to a variation of the correct response.

37 H 2 B S NRC: (2007 NRC Exam) 1/2 twice style noted - satisfactory as-is.

38+ F 1 B U NRC: LOD =1; involves simply recognizing one of two setpoints which are independent of containment pressure.

2 E Licensee: Initially felt the question was satisfactory because three of ten licensed operators missed this question. Distractors A, C and D were chosen once each. After the discussions, the question was considered an enhancement. Modified D and changed the setpoint.

39 H 3 B S NRC: (2005 NRC Exam) 40 F 2 B S NRC: 1/2 twice style noted - satisfactory as-is.

41+ H 2 B E NRC: Distractor A is not plausible - it is incorrect for 2 different reasons. Either reference the controlling channel or fail the non-controlling channel HIGH.

Licensee: Revised A distractor to make it the 1LI-471 yellow channel, which is a controlling channel, and failed it low.

42 F 2 B S NRC: (2007 NRC Exam) 43 H 3 N S Page 6 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 44 H 2 N E NRC: Part (b) of the K/A specifies using procedures to correct Procedures should also be referenced in the distractors for completeness.

Licensee: This question tested both parts of the listed K/A. The "use procedures" part of the K/A is implicitly tested by OM 3.7 provisions needed to arrive at the correct answer. We did not list OM 3.7 in the stem to avoid leading to the answer. The question explanation section has been updated with this information.

45+ F 2 N E NRC: Distractor D is not plausible because this is the normal speed. Also, the distractors provide more information than is sought by the question stem.

Licensee: Modified D to make it plausible for the given conditions.

Also removed extra information in the distractors.

46 H 2 N E NRC: LOK = F. Also, how fast will this happen? Distractor D, as written, contradicts no operator action.

F Licensee: Modified the stem to say a lockout occurred a minute ago. and removed no operator action from the stem. We changed the question to low cog.

47 F 2 B S NRC: (2002 NRC Exam) 48 H 2 B S NRC: (2007 NRC Exam) 1/2 twice style noted - satisfactory as-is.

49+ H 2 B S NRC: (2002 NRC Exam) 1/2 twice style noted - satisfactory as-is.

50 F 2 N S 51+ F 2 B S NRC: (2005 NRC Exam) 52 H 2 N E NRC: Part (b) of the K/A specifies using procedures to correct Procedures should also be referenced in the distractors for completeness.

Licensee: Added a reference to the RMSASRB in the question stem and also fixed the ES form - K/A was cut off.

53 H 3 N S 54+ H 2 B S NRC: (2007 NRC Exam)

Page 7 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 55 F 2 B S NRC: (2007 NRC Exam) 56 F 3 N E NRC: Change the time 1B01 was lost to 10 seconds to make distractor D more plausible.

Licensee: Changed the time that 1B01 was lost to 10 seconds.

57 H 2 B E NRC: Distractor D and distractor A would both be true if the CBD step counter had failed. Replace distractor A.

Licensee: Removed the embedded portion of D in distractor A 58 H 2 B S NRC: (2007 NRC Exam) 59+ F 2 B S 60 F 3 B S NRC: 1/2 twice style noted - satisfactory as-is.

61 H 3 B E NRC: Remove Inadvertent from distractors A and B to enhance plausibility of these choices.

Licensee: Removed inadvertent from choices A and B.

62+ F 2 B E NRC: 1) Part (b) of the K/A specifies using procedures to correct Procedures should also be referenced in the distractors for completeness. 2) The K/A statement on Form 401-5 was cut off.

3) Distractor B should be termed reverification instead of double verification.

Licensee: Fixed form ES-401 K/A statement and modified distractor B as requested. Also added a procedure reference to meet the (b) part of the K/A.

63 F 2 B E NRC: (2007 NRC Exam) Modify Distractor C to have the Exhaust fan trip on CVI to enhance distractor plausibility.

Licensee: Modified distractor C to say Exhaust fan trip on CVI and updated the explanation.

64 H 3 N S 65 F 2  ? N  ? NRC: Why wouldnt the Distractor D method work?

E Licensee: Added water deluge valves (not ANSUL) to the explanation.

Page 8 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 66+ F 1 B U NRC: Too easy for a group who has practiced this basic action repeatedly.

2 N Licensee: Replaced the question with one more suitable. Correct performance of EOP actions is important for reactor operators.

67 F 2 B E NRC: Replace implausible Distractor A.

Licensee: Replaced distractor A.

68 F 2  ? B  ? NRC: Explain why this activity is an RO responsibility required from memory.

E Licensee: Plant operators routinely walk down jobs, procedures, clearances, and scaffolding. Control operators also field questions from plant personnel and Auxiliary Operators. They need to know the requirements of the Guarded Equipment procedure. This K/A and learning objective is covered in licensed operator training. Also modified the stem and distractor C to clarify and eliminate ambiguity.

69 F 2 N S 70+ H 3 x B U NRC: The K/A states mechanical or electrical drawing - the referenced drawing is neither. Also, references provided should not E

be referred to in the question stem (GENERIC).

Licensee: Logic diagrams can be considered electrical drawings.

Fixed the listed reference to be given on the ES-401-5 to match the submitted question (typo). We deleted using the question reference from the question.

71 F 2 B S NRC: (2007 NRC Exam) 72+ H 3  ? B  ? NRC: (2005 NRC Exam) Explain why this activity is an RO responsibility required from memory.

S Licensee: At Point Beach, we have historically used licensed Reactor Operators to operate the manipulator crane. We include the Manipulator Crane in the RO Training Program and we include OJT/TPE for the manipulator crane in the TRQM 18.31 Reactor Operator Qualification Manual.

73 F 2 x B U NRC: The K/A knowledge is given - additional information is required to correctly respond.

Licensee: Revised the question to test the radiological hazard knowledge and add discriminatory value.

Page 9 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 74 F 3 B E NRC: (2007 NRC Exam) Distractor C (2) is implausible and should reference Unit 1/Unit 2 versus A and B for symmetry. In addition, H

LOK = H.

Licensee: Modified distractor C(1) and C(2) as requested. Also changed to high cog.

75+ H 2  ? B  ? NRC: Explain why this activity is an RO responsibility required from memory. 1/2 twice style noted E

Licensee: COs are not required to be able to classify, but are expected to be cognizant of RCS leakage thresholds for classification so they can provide SRO the information when they are exceeded.

COs are provided EAL classification training as a part of their curriculum. Modified the question stem to remove the SU5 RCS Leakage specific classification and modified explanations to emphasize intent of question.

For RO portion of Exam: N(10%,8)=28, M(11)=0, B(<75%,56)=47; LOK: F=35, H(50-60%,38-45)=40; E=30, U(20%,15)=6 Average LOD=(172/75)=2.29 76 H 2 N S NRC: Explain why the CCW surge tank level stabilized (Ref: AOP-1A, step 15b).

Licensee: The automatic thermal barrier isolation that took place, stopped the RCS to CCW system leakage, thus CCW surge tank level stopped going up. Updated the correct answer explanation for additional clarity.

77 H 2 N E NRC: Distractor C points unnecessarily to securing emergency boration - establishing normal boration is sufficient.

Licensee: Changed C distractor to remove the secure emergency boration statement.

78+ H 2 x B U NRC: Selection of procedures is required for SRO-only per 55.43.

E Licensee: After discussion with the facility and additional NRC reviews, this question was determined to be an enhancement with minor changes. Replaced one distractor and made minor enhancements to two other distractors.

Page 10 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 79 H 3 N E NRC: The question must be cleaned up to be grammatically correct and clear. For example, write the entire question in past tense and arrange the distractors into a table format.

Licensee: Cleaned up the tense mismatches and made it easier to read. Put answer choices in a table format and deleted using the question reference from the question.

80 H 3 N E NRC: The statement in Distractor C can be considered true, and must be rewritten to be an action, similar to the other distractors.

Also, aligning another power source in Distractor B is not needed.

Licensee: Reworded choice C to make it an action and removed aligning another power source from B as discussed.

81 H 2 N  ? NRC: Explain why the battery can be considered operable and fully charged if it has been disconnected from its charger for 2 hours2.314815e-5 days <br />5.555556e-4 hours <br />3.306878e-6 weeks <br />7.61e-7 months <br />.

S Licensee: This swing battery was not kept connected to plant loads, so with the charger removed there was no drain on the battery. Per P&L referenced, this battery can be disconnected from the charger for a period of time and still be considered operable.

82+ H 3  ? B  ? NRC: Explain why 3.0.3 is plausible and the basis for the 1545 time (Distractor D).

E Licensee: Removed LCO 3.0.3 from distractor D and replaced it with TSAC 3.1.4 A which was plausible if the candidate believed this made the rod inoperable. We also deleted using the question reference from the question.

83+ H 2 x N U NRC: PAB Ventilation remaining running is not plausible (Distractor A). Also, TS entries are not addressed in all answers (Distractors E

A & B).

Licensee: PAB ventilation running was plausible because the loss of power is to Unit 1 only and PAB ventilation power supplies were split between U1 and U2. If the other set of fans were running, they would have remained running. PAB ventilation was needed for a LBLOCA per AST analysis assumptions, and must be manually re-started within 30 minutes per the EOP-1.3 and TS 3.7.14 basis, (not system knowledge). Thus, the question could not be answered on the basis of system knowledge alone. Explanations and distractors were enhanced to reflect this discussion. Two choices pointed to the TSAC entries for their proposed inoperability, and two choices (including the correct answer) were OPERABLE with no TSAC entries.

Page 11 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 84 H 3 x B U NRC: Selection of procedures is required for SRO-only per 55.43.

Licensee: The question required the SRO to select the attachment to be used and demonstrate his knowledge of the content of the procedure attachment. We did not list the attachment in the question because the title would give away the answer. To make it work, as suggested, needed another Attachment choice. We replaced the question with one that matches the same K/A and allowed procedure selection. Also changed remotely to locally for clarity.

85 H 2 N U NRC: With the knowledge that yellow path CSPs are optional Distractors B & C are not plausible, and this seems to be the main concept needed to answer this question correctly.

Licensee: Added additional information to the stem and modified the choices as required for the distractors to be plausible. The changes restored the questions high cog level.

86 H 2 N E NRC: SI Termination is not plausible (Distractor A).

S Licensee: Discussed that A was plausible because several SI termination criteria are met if you do not recognize containment pressure is adverse or do not remember the criteria. After discussion with Chief Examiner, the question is sat as-is.

87+ H 2 B E NRC: The correct answer is also the longest one. Suggest modifying it or other distractors to avoid drawing attention to it.

Licensee: Rearranged the presentation of the procedure titles to make the correct answer closer in length to the other choices.

88 H 2 N U NRC: A) 1/2 twice question with >2 non-plausible distractors. The incorrect tailpipe temperature is way too high to be believable for an applicant who passed the GFES or heard about the accident at TMI.

B) Why is it plausible that isolating the PRT was successful?

Licensee: A) The 650 degree choices have been shown by industry operating experience to be highly plausible. After discussion with the Chief Examiner, we changed the choices to provide 2 different values for tailpipe temperature. B) The PRT level rising could plausibly be other primary relief valves lifting/leaking after the pressure transient.

Also, corrected K/A cutoff on ES ES-401-5 form.

Page 12 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 89 H 2 x N U NRC: A) Selection of procedures is required for SRO-only per 55.43. B) The question asks what NOT to do versus what would be the correct action to be taken.

Licensee: Randomly drew new generic K/A within the SW system.

(076.02.01.20) Service Water - Ability to interpret and execute procedure steps. We replaced the question with a SW procedure use scenario, meeting 10CFR55.43 (b)(5) use of procedures with a minor editorial change to the new distractor A per NRC request.

90+ H 2 B E NRC: Aligning Containment Spray to a degraded recirculation loop is not plausible (Distractor C).

S Licensee: Explained that because power was restored to that train, there was nothing that indicated it was degraded. The opposite train was degraded. Determined to be Sat with no changes needed.

91+ F 2 N E NRC: Distractor B did not select a procedure, as required for an SRO-only question.

Licensee: Modified distractor B to: generate an Expedited Work Order per WM-AA-200 Work Management Process Overview, and corrected K/A cutoff on ES-401-5 form.

92 H 3 N E NRC: The stem asks for a single basis, whereas the correct answer is two bases. Suggest refocusing the stem to per the TS Bases.

Licensee: Changed stem per comments to say bases versus basis, and corrected typo in K/A number on ES-401-5 form.

93 H 2 N E NRC: Nothing in this question suggests urgency - remove immediately from the action to trip the reactor (Distractor A).

Licensee: Changed distractor A to remove immediately, per comments.

94+ F 2 N E NRC: The Core Loading Supervisor (CLS) would be more plausible than the OATC (Distractor A) for giving approval.

Licensee: Changed per comments to replace Operator at the Controls (OATC - Distractor A) with SFP Supervisor (formerly distractor B) and added Core Load Supervisor for B.

Page 13 of 14

ES-401 Written Examination Review Worksheet Form ES-401-9 Point Beach December 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 95+ F 2 B E NRC: Pre-Screening would be implausible (Distractor A) for the given situation.

Licensee: Removed 'pre-screening' out of distractor 'A' and replaced it with the 'Onsite Review Group approval' previously known as PORC. Corrected learning objective number on ES-401-5 form.

96 H 2 B E NRC: (2007 NRC Exam) Given that the head bolts are detensioned; requires only Mode 6 definition recall (LOK = F). Also, Mode 3 is F not plausible.

Licensee: Added discovered to the stem and replaced MODE 3 with Undefined. Also changed question to low cog.

97 H+ 3 B S NRC: (2007 NRC Exam) 98 H 3 B S NRC: (2005 NRC Exam) 99 F 2 B S NRC: (2003 NRC Exam) 100 H 3 B S For SRO portion of Exam: N(10%,2)=14, M(4)=0, B(<75%,19)=11; LOK: F=5, H(50%,13)=20; E=13, U(20%,5)=4 Average LOD=58/25=2.32 Additional Comments:

1) Using a given reference should be omitted from the stem of any applicable question. The applicant must determine if a supplied reference is needed. Licensee: Ensured that using the given reference was removed from the stem for all questions where references were provided.

Page 14 of 14