ML12074A232

2012 Byron Initial License Examination Administrative ES-401-9

ML12074A232
Person / Time
Site:	Byron
Issue date:	02/10/2012
From:	Bielby M Operations Branch III
To:	Exelon Generation Co
References
Download: ML12074A232 (9)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 N S 2 F 2 N S 3 H 2 N E NRC: Fundamental memory of seal outlet temperature trip criteria with simple math, not higher cognitive level.

F Licensee: Revised LOK to F.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

• The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

• The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

• The answer choices are a collection of unrelated true/false statements.

• The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

• One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

• The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

• The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

• The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

• The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

Page 1 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 4 H 2 B U NRC: Same is not a plausible distractor for charging flow and pressurizer level when no operator action with leakage. Therefore, A S

and B arent plausible.

Licensee: Modified choices such that charging flow and pzr level are compatible, ie, same direction.

5+ H 3 N S 6 H 3 B S 7 H 2 N E NRC: a) Fundamental memory vice Higher cognitive level question of how to trip Rx for ATWS.

F b) If applicant knows both RDMGs are normally running, then can eliminate both A and B without reading further of how they are tripped. Replace/rewrite A or B.

Licensee: a) LOK=F. b) Replaced B. Opening the BYB breaker is plausible because during surveillance testing opening this breaker will result in a reactor trip.

8+ H 2 N E NRC: Clarify HFP and HZP. Errata: misspelled transitioned.

Licensee: Clarified HFP and HZP, corrected misspelling.

9 H 2 B E 2011 BWD crt exam U NRC: a) Replace A. Closing MSIVs is also a correct action.

b) B is an SRO action. The other choices are actions taken by the ROs. Either modify, or replace B.

Licensee: a) Replaced A with an incorrect action. b) ROs are required to know info such as SG levels below the 2BFR-H.1 transition level, but AF flow is met making this incorrect.

10 F 2 N S 1/2 twice 11 H 2 N S 1/2 twice 12+ H 3 N U NRC: B and D implausible. B, loss of one battery charger would not cause trip. D, loss of both DC buses would not allow unit to stay on line. Rewrite or replace B and D.

Licensee: Incorporated comment by incorporating common term into stem, deleting the second column of choices (ie, Unit Status) and replacing B and D.

13 F 3 B S Page 2 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 14 H 3 N S 15 F 2 B E NRC: A needs clarification. Need to add leak between likely and location.

Licensee: Incorporated comment.

16 F 2 B U NRC: B and C implausible. B, dumping steam at max rate would result in much larger cooldown than 100 deg F/hr. C, reduction to one train SI (RHR, CV, SI) would be about 4000 gpm from RWST; however, max makeup to RWST is only about 150 gpm. Replace, or rewrite B and C.

Licensee: Modified stem to include currently at Step 12, ESTABLISH ONE TRAIN OF ECCS FLOW. Replaced distracters B and C.

17+ H 2 B S 2009 BWD ILE 18 H 3 N S 19+ F 2 B S 2011 BWD ILE 20 H 2 M E NRC: Delete on PZR Hi Level from heading in 3rd column of choices. Cuing applicants to consider a trip based on PZR level, let applicants determine if there is a trip without narrowing possibility of what to consider.

Licensee: Comment incorporated.

21 H 2 B S 22+ H 2 N S 23 H 3 N S 24+ F 2 B E 2010 BYR crt exam; 1/2 twice NRC: a) Add ONLY to end of D distractor.

b) Add a reason for Excess Letdown in distractor A to make it resemble format of other choices.

c) Are operators required to memorize the number 75 gpm from RNO Step 1b of 1BOA SEC-8? If not, then make reasons wrong for distracters.

Licensee: Comments incorporated. Operators are expected to know 75gpm from RNO Step 1b of 1BOA SEC-8.

Page 3 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25 H 2 N E 1/2 twice F NRC: Fundamental memory of generator trip which causes Main Turbine and reactor trip and Fire Protection actuation due to UAT sudden pressure trip above P-8 vice Higher cognitive level question.

Licensee: LOK=F 26 F 2 N E 1/2 twice NRC: C and D: insert indicating lights after pump in C and after Pump in D.

Licensee: Comments incorporated.

27 F 3 B S 28 F 2 N S 29+ F 3 N S 30 H 3 N S 1/2 twice 31+ F 3 B S 2009 BWD crt exam 32 H 3 N S 33 F 3 N S 34 H 3 B S 35+ H 3 N S 1/2 twice 36 F 2 N S 1/2 twice 37 F 2 N S 38+ F 2 N E NRC: Remove status, not required to meet KA, only lowers difficulty of question.

Licensee: Comment incorporated.

39 H 2 N S 40 H 3 N S 41 F 2 N S Page 4 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 42+ H 2 B S 43 H 2 B S 44 F 2 B S 45+ F 2 B S 2011 BWD NRC exam 46 F 2 N S 1/2 twice 47 F 2 N E NRC: Is MAXIMUM flexibilitya technical term understood by operators? If not, rephrase stem.

Licensee: Incorporated comment.

48 H 2 B S 49+ F 2 B S 2011 BWD crt exam; 1/2 twice 50 H 2 N U NRC: Double jeopardy with Q12, replace question.

Licensee: Replaced question.

51+ H 3 N U NRC: a) Stem, last sentence, replace is most likely to with will.

b) Will reduction in frequency cause DG output breaker to open on high current?

Licensee: a) Incorporated comment. B) Breaker is rated for 3000 amps, so answer is No.

Page 5 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 52 H 2 N E NRC: a) Stem, third sentence, delete the GREATEST probability of, and in last sentence, delete MOST likely to be . These imply more than one correct answer, and if there is more than one correct answer, then either rewrite or replace question.

b) If exposed for same time in all Cases, then for same point source, the further away, the less dose. But doesnt appear valid in all Cases if time and distance is varied. Explain justification for answer.

Note: The reference for distance states that DR decreases by a factor of d1/(d2)2 , ie, Distance1 divided by square of Distance2 .

Reference does not support answer where the exposure time and distance vary.

Mathematically, the numerator could change less, or more, than the denominator, and the final exposure could be less, or more. Just depends on the relative time and distance changes.

D=(DR x t) x d1/(d2)2 Licensee: a) Comments incorporated. b) OPRO2J measures dose rate, not dose; therefore DR2=DR1XD1/D22 53 F 1 N U NRC: a) Low LOD. Asking straight forward backup water supply question. b) B could also be correct answer, SX could be used as backup supply for SFP. c) KA asking for physical connections the reference, System Description, pg 35, provides the physical connection, ie, backup water supply to ring header in AB.

Licensee: Incorporated comments, rewrote question to meet KA, LOD=2, and only 1 correct answer.

54+ H 2 N S 55 H 3 B S 2011 BWD NRC exam; 1/2 twice 56 H 3 B S Reference; 57 H 2 N S 58 H 3 N S 1/2 twice 59+ F 2 N S 60 F 3 B S 61 H 3 B S Page 6 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 62+ H 3 B S 63 F 2 N S 64 F 2 N S NRC: Replace/rewrite B and C, not plausible.

Licensee: Replaced question.

65 H 3 B S 2011 BWD NRC exam 66+ H 3 N S 1/2 twice 67 H 3 N S 68 H 2 B U 1/2 twice NRC: B and C implausible, ie, boration equivalent to withdrawing rods, or dilution equivalent to inserting rods.

Licensee: Rewrote question.

69 F 2 N S 1/2 twice 70+ H 2 B S 2011 BWD NRC exam 71 F 2 B S 72+ F 2 B S 2011 BWD NRC exam 73 F 1 B U NRC: a) Low LOD. B, C and D inform many personnel directly, while A only informs an individual. Without knowing correct answer, it is evident that A answers the question.

b) A not plausible because he is not notified directly by anyone.

Replace, or rewrite question.

Licensee: Could not identify additional replacement plausible distracters, replaced question.

74 F 2 B S 1/2 twice 75+ F 3 B S For RO portion of Exam: N(10%,8)=42/56%, M(11)=1, B(<75%,56)=32/43/%; LOK: F=36, H(50-60%,38)=39/52%; E=11, U(20%,15)=8 76 H 3 N S 77 H 3 N S Page 7 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 78+ F 2 B S 79 H 3 N S 80+ H 3 N S Reference 81 H 2 B S Reference 82 H 2 B S 83+ F 3 B E NRC: Eliminate possible cue in last line of stem. Replace when they all are concerned with SHUTDOWN MARGIN with for initiating boration.

Licensee: Incorporated comment.

84 H 3 N S 85 H 3 N S 86+ H 2 N S 87 H 2 X B U 1/2 twice NRC: a) Not SRO only. Requires recall of whether valve is MOV or AOV (memory and RO knowledge). The procedure titles are self explanatory of the malfunction, and ROs would be expected to know the effects of the malfunction on AOVs or MOVs and the RH HX outlet temperature.

b) Doesnt seem clear that the question is addressing the plant as a whole (ie, the overall effect that increases RH HX temperature) or just the individual item listed.

Licensee: Replaced question.

88+ F 2 N U 1/2 twice 89 H 3 N S 90 H 3 B S 2011 BWD crt exam 91+ F 2 B S 2009 BWD crt exam 92 H 3 B S 93 H 3 N S Page 8 of 9

ES-401 Written Examination Review Worksheet Form ES-401-9 Byron January 2012 Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 94 F 2 B S 2011 BWD crt exam 95+ F 2 N U NRC: a) B and C have cues to correct answer. Item 6 is obviously correct, who else other than Operations would ensure plant conditions?

b) D is implausible. Item 1, not believable that Operations would write an Electrical Engineering test procedure.

Licensee: Rewrote choices to revise stem items and reordered items for choices.

96+ F 2 N S 1/2 twice 97 F 3 N S 98 F 2 B S 2011 BWD NRC exam 99+ F 2 B S 100 F 2 B S 2011 BWD NRC exam For SRO portion of Exam: N(10%,3)=13/52%, M(4)=0, B(<75%,19)=12/48%; LOK: F=11, H(50%,13)=14; E=1, U(20%,5)=3/12%

Comments:

Arrange choices based on length of sentence, ie, from shortest to longest.

Are questions requiring references identified to the applicant? Put together in book, let applicant pick one he wants.

For SRO only questions, make sure you are in an SRO position in the question stem.

Could increase Us based on post-exam comments.

Page 9 of 9