ML13178A362

2013 Perry Nuclear Power Plant Initial License Examination Administrative Es 401-9-1

ML13178A362
Person / Time
Site:	Perry
Issue date:	03/14/2013
From:	NRC/RGN-III
To:	FirstEnergy Nuclear Operating Co
Bielby, M.
Shared Package
ML11354A269	List: ML13175A081 ML13178A362 ML13178A363 ML13178A365 ML13178A366
References
Download: ML13178A362 (12)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 F 2 X B E NRC: D. not plausible. None of the procedure table modes or staffing types contain 3 ROs, only 1 or 2. Revise the number of ROs to make D. plausible.

Licensee: Changed D to 1 RO since this is requirement for Modes 4 and 5.

2 F 2 B S 2009 PER RO Exam 3 F 2 B S 4 H 3 B S 2009 PER RO Exam; NRC: Question was modified.

M Licensee: Question was modified.

5 F 3 N E NRC: Replace should in last line of stem with something like are procedurally required Licensee: Changed you should to Plant procedures require you to.

6 H 3 N S 7 F 2 N E NRC: The referenced procedure, NOP-OP-4101, Step 4.7.7, states that the RPM and Shift Supervisors approval may only be granted. Is the term Shift Supervisor equivalent to Unit Supervisor or Shift Manager? If Unit Supervisor is equivalent, there may be no correct answer.

Licensee: The terms Shift Supervisor and Shift Manager are equivalent. Shift Supervisor is the older title and not changed in all Fleet procedures yet. Neither title is equivalent to Unit Supervisor.

8 F 2 N S 9 F 2 M S 10 H 2 N E 1 of 2 twice; NRC: LOK is F vice H. Memorization of two separate facts (declaration time and location).

F Licensee: Changed LOK to Fundamental.

11 H 2 B S 2005 PER RO Exam 12 H 2 B S 2007 PER RO Exam 1

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 13 H 2 N S 14 H 2 X B E 1 of 2 twice; NRC: Distracters A. and D. are not independent of one another, ie, if A. were correct, then D. would also be correct, and vice versa. Replace one of distracters A. or D. with plausible distracter.

Licensee: D is plausible per the following additional explanation:

The Applicability for TSV & TCV trips is 38% RTP. These are required to be operable at >40% RTP. Note: this was different than the original justification.

15 H 2 B S 1 of 2 twice 16 H 2 X M E 1 of 2 twice; NRC: 1) KA doesnt match. KA 295016, AK2.02 is Knowledge of the interrelations between control room abandonment B

and Local Control Stations, not AA2.02, Ability to determine and/or interpret Local Control Stations as they applied to control room abandonment.

2) Stem, last sentence, spelling of Emergency

3) B not M, same correct answer, no distracters significantly modified.

Licensee: 1) Originall wrote question to correct KA number, but had incorrect statement identified. Corrected KA statement to match KA number. 2) Corrected spelling of EMERGENCY. 3) Changed from M to B.

17 F 2 B S 18 H 2 M S 1 of 2 twice; 2009 PER RO Exam; NRC: Need original question.

Licensee: Submitted original question.

19 H 3 B S 1 of 2 twice 20 F 2 N S 21 F 2 M S 1 of 2 twice; NRC: B not M. Did not change stem condition and one distracter, or correct answer.

B Licensee: Changed from M to B.

22 F 2 B E 2003 PER RO Exam; NRC: LOK is H vice F. Need to recall and analyze non-linear curve containing multiple parameters (SP H

temperature and level, RPV pressure).

Licensee: Revised LOK to H.

2

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 23 H 2 N S 24 F 2 X M E 2009 PER RO Exam; NRC: 1) Does not meet KA. KA 295028 EA1.01 is for DW Spray not DW ventilation.

2) Need original question.

Licensee: 1) 295028 EA1.01 WAS ON es 401-4, Rejected KA list.

Left wrong KA number on Exam. Corrected Exam to show 295028 EA1.02. 2) Submitted original question.

25 H 2 B E 2007 PER RO Exam; NRC: LOK is F vice H. Recall of entry condition for ED.

F Licensee: Changed LOK to F.

26 H 1 X N U 1 of 2 twice; NRC: Low LOD makes two distracters implausible.

Second part of question lowers overall difficulty. Systems with auto-S initiation generally have a manual initiation that bypasses the auto-initiation interlocks, so it becomes easy to eliminate two of the distracters (B. and C. implausible). Rewrite second second part of choices.

Licensee: Choices B and D are plausible because the ADS and SPMU systems do not always initiate using the A&D p/bs. If RHR LOCA signal not present, back-panel switch manipulations must be made to allow the SPMU A&D p/bs to work. If LP pump not running ADS A&D p/bs wont work. Added to question justification.

27 H 3 M S 28 H 2 M S 1 of 2 twice; 2009 PER RO Exam; NRC: B not M. Did not change stem condition and one distracter.

B Licensee: Changed from M to B.

29 F 2 B S 1 of 2 twice 3

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 30 F 2 X N U NRC: 1) C. and D. are not plausible (doesnt make sense), simple math indicates that raising voltage in a voltage to hertz ratio will increase the ratio (ie, make it worse) without even knowing anything about the expected system response or operator action (ie, common sense: why would system be designed that way or procedure written as such?).

2) D. conflicts with last stem sentence, because one says if this alarm does not clear within a minute and other says ...to clear the alarm.

Licensee: Added info to stem: gen. freq. at 58 hz, terminal voltage at 24000 volts, Div 1 DG running for monthly surveillance, and Buses EH12 & EH13 lined up to preferred source.

Then changed C to breakers feeding buses EH12&13 will trip on underfreq, and changed D to Div 1 DG output breaker will trip on underfreq.

31 H 3 B E 1 of 2 twice; NRC: 1) Stem, first sentence need to be rephrased.

2) Re-format stem to separate and reflect the different time elements in which the events happen, ie, currently all are grouped together in bullets under sentence that says plant is operatingfollowing conditions:.

Licensee: 1) and 2) changed stem to reveal timeline of events.

32 H 3 N S 1 of 2 twice 33 H 2 B S 34 F 2 B E NRC: Distracters B.,C. and D. all give actions and parameter results; however, correct answer A. only gives action with no parameter result. Need to make answer and distracters the same format.

Licensee: Changed A format to action and result to resemble B,C and D.

35 H 3 X X B U 1 of 2 twice; 2007 PER SRO Exam; NRC: 1) Second part of question is SRO only (LCO).

S

2) Doesnt meet KA. Supposed to be for Inadvertent Containment Isolation.

Licensee: 1) Per Clarification Guidance for SRO-Only Questions, LCOs, Applicabilitys and Actions 1 hr are RO knowledge. 2) The loss of RPS bus causes an inadvertent containment isolation which causes a loss of SDC.

4

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 36 F 2 X N E NRC: Make order of equipment qualification and personnel access the same in Distracters C. and D.

U Licensee: This was KA miss, could not rewrite acceptable question and choices, replaced with new question.

37 H 3 M S 38 H 3 X M U 1 of 2 twice; 2003 PER RO Exam; NRC: This is a K5 KA, applied to ability A2.12. A2 is Ability to (a) predict the impacts.of Pump S

Runout on RHR/LPCI: INJECTION MODE; and b) based on those predictions, use procedures Licensee: Wrote question to correct KA number, but had incorrect KA statement on exam. Corrected KA statement to match KA number, changed guidance to procedural guidance and added additional info to justification.

39 F 2 M S 40 H 2 M U NRC: Either A or B may be correct. Applicant could argue that there was no indication of why LPCS inadvertently initiated. As a result, how could the applicant be sure LPCS wouldnt auto start after restoration of the power supply even if LPCS was procedurally disabled from auto start?

Licensee: Added to stem due to failure of DW pressure trip units and added and were successful to immediate action statement in the stem.

NOTE: Post exam comments changed answer from A to B.

41 F 2 B S 42 F 2 B S 2010 PER RO Exam 43 F 2 B S 44 F 3 N E 1 of 2 twice; NRC: LOK is H vice F. Requires comprehension and analysis to determine correct answer.

H Licensee: Changed LOK to H.

45 H 3 B S 46 F 3 B S 5

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 47 F 2 B S 1 of 2 twice 48 H 2 N S Ref. Provided: LPRM to APRM Channel Assignment Attachment 2 of SOI-C51 (APRM) 49 H 3 B S 50 H 2 M S 2010 PER RO Exam; NRC: Need original question.

Licensee: Submitted original question.

51 H 2 M E 1 of 2 twice; NRC: Make Part 2 format of distracters C and D same as A and answer B, ie, either the action only, or both the action and reason for the action.

Licensee: Change C and D to make Part 2 action only.

52 H 2 B E 2001 PER RO Exam; NRC: LOK is F vice H. Memorization of separate valve trips (one doesnt depend on the other). Either know, F

or dont know the trips, cannot use comprehension or analysis to determine answer.

Licensee: Made LOK F.

53 H 2 M S 2007 PER RO Exam: Need original question.

Licensee: Submitted original question.

54 H 3 M S 2010 PER RO Exam; NRC: B not M. Did not significantly change one stem condition or distracter. Same correct answer.

B Licensee: Change to B.

55 F 3 N S 1 of 2 twice 56 H 2 N E 1 of 2 twice; NRC: LOK is F vice H. Memorization of monitor trip and location (one doesnt depend on the other). Either know, or F

dont know, cannot use comprehension or analysis to determine answer.

Licensee: Made LOK F.

57 F 1 X N U NRC: 1) Low LOD. Applicant only has to know that Hanna not connected and can eliminate three distracters.

H

2) Hanna distracter is not believable (plausible) because it would never provide power in/out of the Perry Transmission Yard (all other lines are connected).

Licensee: Replaced question with LOK=H.

6

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 58 F 3 B S 59 F 2 N S 1 of 2 twice 60 F 2 M E NRC: 1) In order to balance choices with correct answers, suggest:

a) RHR C= immediately in distracter A.; b) ESW B=18 seconds in B

distracter B.

2) B not M, no significant change to condition in stem.

Licensee: 1) Changed A to RHR-immediately and changed B to ESW-18 seconds. 2) Changed M to B.

61 F 2 N S 1 of 2 twice 1 of 2 twice 62 H 3 N S 63 F 2 B S 64 H 3 B S 65 H 3 M E NRC: Change the tense of the choices to reflect that you are asking the applicant to predict the outcome:

a) insert will after Pump and replace tripped with trip.

b) insert will after Pump.

c) insert will after ISOL and replace isolated with isolate.

d) insert will after ISOL and replace isolated with isolate.

Licensee: Changed tense of choices to predict outcome.

7

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 66 H 2 X X N U Ref. Provided: ABB Steam Tables 1 of 2 twice; NRC: 1) Does not meet KA. How does determining bulk RPV temperature and comparing to BHD or Vessel Head Flange temperature apply to operational implications?

2) Explain why A.,B., and D. are plausible (believable).

Licensee: 1) Without RRPs running, monitoring of bulk coolant temperature by instrumentation is not accurate. The operational implications are determining bulk coolant temperature in order to avoid exceeding cooldown rate. With the scram now reset, cold CRD water is no longer entering the RPV. 2) Revised format into 1 of 2 twice by adding determination of cooldown rate exceeding 100 deg F, and replaced the implausible determination of bulk RPV water temperature with plausible number.

NOTE: Post exam comment changed answer from C to A.

67 H 1 M U Ref. Provided: (Modified) EOP-SPI Supplement Figure #7 HDOL; 2 S 2009 PER RO Exam; NRC: Low LOD. Applicant only has to plot the given Containment parameters for H2 and pressure to determine that Containment HDOL has not been exceeded.

Licensee: Provided reference drawing to be used (deleted notes) and required plotting [H2].

68 F 1 X M U NRC: Low LOD and C. and D. not plausible: a) C. and D. are same; b) why would moving empty hoist be prevented? Also, if no automatic interlocks, why are B., C. and D. distracters plausible?

Licensee: Replaced question.

69 H 2 N S 70 H 3 N S 71 F 2 M E NRC: 1) Paralleling in this fashion always requires SS to rotate in clockwise direction which eliminates need to know other distracters.

B Eliminate part (3) to increase LOD and plausibility.

2) B or N, but not M. Questions are not related.

Licensee: 1) Eliminated part 3. 2) Changed to B.

1 of 2 twice 72 F 3 N S 73 F 3 N S 8

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 74 H 2 M E NRC: 1) Knowing damper status or fan status eliminates need to know status of the other. Eliminate either the damper positions or the fans status to increase LOD.

2) B not M, no significant condition change in stem.

Licensee: 1) Eliminated damper positions from answer/distracters.

2) By eliminating damper positions, made question M.

75 H 2 X M U 1 of 2 twice; NRC: A.(1) and B.(1) not plausible. It doesnt make sense that operating pressure within design of the SRV spring set pressure would allow the pressure safety limit to be exceeded. In order to exceed the pressure safety limit, nothing would work.

Licensee: Revised question choices A and B.

LOK (F/H) 35/40=47%/53% (H=38-45)

B/M/N 33/17/25=44%/23%/33% (B56; M11, N@H=10, N@H8)

UNSAT for RO Only 7/75=9%

E for RO Only 20/75=27%

1 of 2 twice RO=25 2009 2010 B 2 2 M 4 1 1 H 3 N S 1 of 2 twice 2 H 3 N S Ref. Provided: TS 3.8.3 (Partial) 3 H 3 N S 4 F 3 N S 5 H 3 N E 1 of 2 twice; NRC: 1) Errata: Last bullet of stem, T and N in VENT have a space.

2) Revise choices to a two column format, one column for procedure and other for direction.

Licensee: 1) Corrected VEN TPIPE. 2) Revised choices and put into two column format.

6 H 3 B E NRC: Revise last sentence of stem to eliminate the term should (ie, replace should with would).

Licensee: Replaced should with would.

9

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Ref. Provided: EPI-A1 7 H 3 N S 8 H 2 X M U Ref. Provided: PDB-A06, Power Flow Map (modified);

NRC: Determining that loss of 1 RRP would cause flow to lower less than 50% and interpreting the power to flow map is RO knowledge.

Licensee: Directing the procedure is SRO duty. This is beyond Immediate Action of procedure.

Note: Post exam comments deleted this question because no correct answer, and on RRP trip plant ends up to right of the Controlled Entry/Immediate Exit region, not as indicated in the question justification.

9 H 2 N S 10 H 2 X X X N U NRC: 1) Stem contains cues and not SRO only. Knowledge of procedure contents is RO knowledge. The term Nuclear Closed Cooling is used in both the stem and choices, and taking into account the event, that knowledge gives away choices 1 and 4.

The only decision is whether Choice 2 or 3 is correct, and since 3 is used in three distracters, it obviously makes them wrong, and an astute test taker would choose the correct answer B.

2) Does not meet KA. How does question test knowledge based on system pressure?

3) Replace Item 3, or rewrite, one of the distracters. As written, applicant only would have to know what procedure did not contain direction to address mitigation of the event (ie, 3), not select the procedures that mitigate the event.

Licensee: Replaced question.

11 H 2 N S Ref. Provided: PDB-C0011, PSA Pre-Solved Configurations for On-Line Risk(partial); 1 of 2 twice; NRC: Determining risk is RO job? Notification of duty team can be done by RO?

Licensee: Determining Risk is SRO job. NOP-OP-1007 specifically states Shift Manager responsibilities.

12 H 2 N S Ref. Provided: TS 3.6.2.1 (partial)

Ref. Provided: PAP-1604, Reports Management; EPI-A1, 13 H 2 B S Emergency Action Levels 10

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 14 H 3 N S 15 H 2 M S 1 of 2 twice 16 F 2 B S 17 F 2 M S 1 of 2 twice 18 H 2 M S 19 H 3 M S Ref. Provided: TS 3.5.1 (partial); 1 of 2 twice 20 H 2 B S Ref. Provided: PMT Data; 2010 PER SRO Exam; 1 of 2 twice 21 H 3 M S Ref. Provided: TS 3.3.6.1 (partial); PDB-I-5 (partial); 1 of 2 twice; 2010 PER SRO Exam 22 H 3 N E NRC: 1) What is OAI-1701?

2) No, references, SROs expected to memorize LCOs?

Licensee: 1) OAI-1701 is procedure for tracking LCOs. It discusses Support-Supported Systems and describes the LCO 3.0.6 exception to LCO 3.0.2. 2) Since the temperature limit for ECCW is not covered in TS LCO 3.7.10, it is important for the SRO to know other factors that might cause a system to become inoperable.

Ref. Provided: TS 3.1.3 (partial); TS 3.1.4 23 H 2 B S 24 H 3 N S 1 of 2 twice 25 H 2 M S LOK (F/H) 3/22=12%/88% (H=38-45)

B/M/N 5/7/13=20%/28%/52% (B19; N2@H, M4)

UNSAT for SRO Only 2/25=8%

E for SRO Only 3/25=12%

1 of 2 twice SRO=9 UNSAT Overall 12/100=12%

2009 2010 B 0 2 M 0 0 11

ES-401 Written Examination Review Worksheet Form ES-401-9 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F) fundamental or (H) higher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distracters contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distracters are not credible; single implausible distracters should be repaired, more than one is unacceptable.

• One or more distracters is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B) bank, (M) modified, or (N) new. Check that (M) modified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U) unsatisfactory (requiring repair or replacement), in need of (E) editorial enhancement, or (S) satisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

12