ML15110A239

2015 Perry Power Station Initial License Exam Administrative Submittal Form ES-401-9

ML15110A239
Person / Time
Site:	Perry
Issue date:	04/03/2015
From:	Walton R NRC/RGN-III/DRS/OLB
To:	FirstEnergy Nuclear Operating Co
References
Download: ML15110A239 (7)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 Y B S 2 F 2 Y B S 3 F 3 Y B S 4 F 2 Y B S 5 H 3 Y B S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 F 2 Y N E Can a Temp Mod tag be hung on an OOS annunciator ever? If so, then distractor B may be construed as correct also! (Time S dimension?). RE: Temp Mod procedure DOES NOT state that can hang temp mod tags on annunciators. But PAP-1404 confirms answer.

7 F 2 Y B S 8 H 3 Y B S 9 H 3 Y B S 10 F 2 Y B S 11 H 3 Y N S 12 H 3 Y N S 13 F 3 Y N S 1/2X2 Q 14 H 3 Y N S 1/2X2 Q 15 H 3 Y N S 16 H 2 Y N S 17 H 4 Y M S Previous NRC exam 2010 18 H 3 Y N S 19+ H 3 Y B S 20 H 3 Y Y M U 1/2X2 Q SRO-only question for procedure usage and refueling.

Suggest replacing this question (and maybe repicking K/A) since this S

question is too close to Q37. RE: ROs are expected to know entry conditions to ONIs.

21 H 3 Y N S 1/2X2 Q 22 H 2 Y M S 23 F 2 Y B S 24 F 2 Y B E Before distractors, add Order of preference shown is from most preferred to least preferred. RE: Will add most preferred to least preferred in question stem.

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25 H 2 Y B S 1/2X2 Q I presume that the students will perform the calculations to determine average temperature for this exercise and not have it provided! RE: Will provide students with sheets 9 and 28 of 33, Rounds sheets. Students will perform average calculation.

26 F 3 X Y B U First sentence of stem gives condition for requiring ED. Second sentence says condition of first sentence is met, then answer must E

be to ED (distractors B or C). Suggest rewording in form of a question. Why does EOP-2 require ED if SP Level cant be maintained >2 above horizontal vents? Are ROs taught EOP basis? RE: (Previously approved by Clinton exam bank question, not unsat if changed). Question stem changed.

27 F 2 Y B S 28 H 3 Y B S 29 F 2 Y B S 30 H 3 Y M S 31 F 3 Y N S 32 F 2 Y B S 33 F 2 Y B S 34 F 2 Y B S 35 H 3 Y Y M U 1/2X2 Q. Previous NRC exam 2013. Looks like SRO-only question due to entering into ONI procedures (not EOPs). RE: Not an SRO-S only question since ROs are required to know entry conditions to ONIs.

36 H 3  ?? X Y  ? B U 1/2X2 Q. (SRO-only question since requires knowledge of TS Action steps?) Reference ONI-C-11-1 steps in lieu of TS steps! Series of True/False questions? Move common part of all distractors into stem. Distractors not credible since TS does not have 20 minute criteria. RE: 20 minute TS action statement DOES exist in TS -

therefore is CREDIBLE. Also, answer is below TS line in SRO guidance but it less than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, so knowledge must be known by RO. Common text in distractors moved to stem, CRD pressure attribute changed to rod position based on Q revalidation.

37 H 3 Y B S

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 38 H 3 Y B S Previous NRC exam 2010.

39 H 3 Y N S 1/2X2 Q. SRO Only? RE: ROs are required to know ONI entry conditions.

40 H 2 Y B S Previous NRC exam 2010.

41 H 2 Y B S 1/2X2 Q 42 F 2 Y N S 43 H 3 Y B S 44 H 3 Y B S 45 H 3 Y B S 46 F 3 Y N S 47 F 2 Y B S 48 F 3 Y N S 49 H 2 Y B S Previous NRC exam 2010 50 H 2 Y B S 51 H 2 Y B S 1/2X2 Q 52 H 3 Y M S 53 H 2 X Y B U Backwards Logic: Effect occurs, what caused it? Should read, effect occurs, what does operator do, or procedure response Suggest changing distractors to what response to operators take. RE:

Changed distractors such that must pick entry into Off Normal Instructions to address adverse condition vs.name that event.

54 H 2 X Y B U 1/2X2 Q. If DFWCS level channels fail, why shift DFWCS to AUTO?

Implausible distractors A & C. RE: Changed A & C distractors to read shift DFWCS to single element.

55 F 2 Y B E 1/2X2 Q. Previous NRC exam 2010. If normal Rx Level setpoint is 196 and question has setpoint selected to 200, then applicant must S

apply this new knowledge to answer question correctly. Not a memory question. RE: Operators are taught that setpoint setdown will control to level setpoint NOT to 196

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 56 F 2 Y N S 57 H 1 Y M S 1/2X2 Q. This is better done as a JPM rather than a written question.

58 H 2 Y N S 59 H 3 Y N S 60 H 2 Y B E 1/2X2 Q. Need to pull up common words from each distractor and put into stem. RE: Pulled up common phrases from distractors into stem of question.

61 H 3 Y M S 1/2X2 Q. Previous NRC 2010 Exam 62 H 2 Y B E 1/2X2 Q. Previous NRC exam 2013 IS this a modified question?

RE: Yes, question was changed since instrument air pressure alone S

was not enough. Temp air compressor added. Plant mod.

63 F 2 Y N S 1/2X2 Q 64 F 2 Y B S 65 F 2 Y B S 66 F 2 Y N S 1/2X2 Q 67 F 2 Y B S 68 H 3 Y B S 69 F 2 Y N E 1/2X2 Q. Missing blank in stem for 2nd answer. RE: Blank added for (2).

70 H 3 Y N S 1/2X2 Q 71 F 2 Y B S 72 H 3 Y B S 1/2X2 Q 73 F 2 Y N S 1/2X2 Q 74 F 2 Y N S 1/2X2 Q

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 75 H 3  ?? M E K/A mismatch? Heat Transfer mechanisms, submergence, spray cooling and steam cooling. RE: This is a good question that may miss the K/A. Had Perry change K/A for better match. Increased difficulty of question by changing answer from submergence to steam cooling 76 F 2 Y Y B S 77 F 2 Y Y N S 78 F 2 Y Y B S 79 F 2 Y Y B S 80 H 3 Y Y N S 81 F 2 X X Y Y N U 1/2X2 Q. 2 Person Line of Site Rule is in stem and also in 2 distractors. Also, DSO reference seems too high in chain of command to be a creditable distractor. RE: Question replaced.

82 F 2 Y Y N S 83 H 3 Y Y N S 1/2X2 Q 84 H 2 Y Y B S 85 H 2 Y Y N S 86 H 2 Y Y B S 87 H 2 Y Y B S Previous NRC exam 2010.

88 H 2 X Y NO N U 1/2X2 Q. Question can be answered without the stem! Not an SRO-only question. Reference TS in lieu of SOI precautions/limitations.

How is this not a memory question? Need to pull common phrase from distractors into stem of question. ALSO, need to delete reference to SOI-M14 and reference instead TS Bases to ensure SRO-only question. RE: Question stem and distractors changed.

89 H 3 Y Y M S 90 H 2 Y Y N S 1/2X2 Q 91 F 3 Y Y N S 1/2X2 Q 92 H 3 Y Y B S 1/2X2 Q.

ES-401 Written Examination Review Worksheet Form ES-401-9 PERRY 2015 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 93 H 3 Y Y N S 1/2X2 Q 94 H 3 Y Y B S (References Provided) Why are 4 similar PEIs provided? RE: Will remove Figures from partial EOP chart.

95 H 3 Y Y B S Previous NRC exam 2010.

96 H 3 Y Y N S 1/2X2 Q 97 H 3 Y Y N S 1/2X2 Q 98 H 4 Y Y N S 99 H 3 Y Y N S 1/2X2 Q 100 H 2 Y Y N S 1/2X2 Q 33% of questions in one of two taken twice format (1/2X2 Q)