ML22083A062

Wat 2022-02 Draft Written Comments

ML22083A062
Person / Time
Site:	Waterford
Issue date:	03/09/2022
From:	Heather Gepford NRC/RGN-IV/DORS/OB
To:	Entergy Operations
References
Download: ML22083A062 (10)

Text

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9

1. LOK 2. LOD 6

7 Ans Letter CFR 8

Q#

(F/H)

(1-5) stem focus cueing T/F cred dist partial job link minutia

1. /

units back-ward K/A SRO-only B/M/N U/E/S A/B/C/

D 55.41 55.43 Explanation rev0 of questions (used for first validation)

Rev1 of questions for second validation Rev2 of questions after final changes made.

1 H

2 N

S D

7 2

F 3

N S

B 10 3

H 3

N S

C 7

4 F

3 N

S B

7 5

H 3

N S

D 5

6 H

3 N

S A

7 Need to improve the stem to ask "what item receives a signal to open or close as a result of the CURRENT stem conditions?"

Stem corrected with rev 2 question and is now Sat.

7 F

2 N

S C

7 8

H 3

N S

B 5

9 H

3 N

S A

7 10 F

3 N

S D

7 11 H

3 N

S A

7 12 F

3 N

S B

10 13 H

3 B

S C

7 2009 NRC Q57 14 F

3 N

S C

14 15 F

3 N

S B

8 16 F

3 N

S A

10 17 F

2 B

S C

10 2017 NRC

3. Psychometric Flaws

4. Job Content Flaws

5. Other a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 18 H

3 N

S D

7 19 F

3 N

S B

7 20 F

3 N

S D

10 21 H

3 N

S B

5 22 H

2 N

S C

5 2006 NRC-after validation determined that two answers could be correct (this was a bank question with rev1) so wrote new question. With rev 2 of the question it is Sat.

23 H

3 N

S B

7 24 H

2 N

S D

7 Licensee requested a change to stem values and answer because of poor validation. NRC agreed to accept the changes and question is Sat with the rev 2 file.

25 H

3 N

S A

7 26 H

3 B

S C

7 2015 NRC 27 F

3 N

S D

5 didn't validate well.Licensee stated that there is confusion on 5gpd setpoint, and the IR is low at 2.5. Licensee made two changes to the question and NRC agreed with these changes and question is sat with rev2 question.

28 H

3 N

S A

7 29 F

2 N

S D

7 Licensee noted that Q29 and Q54 ovelap on SA valve, need to change one. Changed Q54 so this question is Sat with Rev 2.

30 F

3 N

S B

7 31 H

3 N

S B

5 a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 32 F

3 B

S B

7 2018 NRC Q41 stem edit for step in EOP. Complete in rev2 and is now Sat.

33 F

3 N

S A

10 34 H

3 N

S A

7 35 H

3 B

S D

5 36 H

3 N

S B

5 37 H

2 N

S D

7 38 H

3 N

S D

7 Trip path is incorrect. NRC corrected and is now Sat with rev 2.

39 F

3 N

S A

7 changed stem of question after validation (two correct answers) because these valves have both N and air accumulators. Question is Sat with rev 2.

40 H

3 N

S A

10 2012 NRC 41 H

2 B

S B

7 2010 NRC 42 H

3 N

S D

10 43 H

3 N

S A

10 44 H

3 N

S A

7 45 H

2 N

S C

10 changed stem after validation because part 2 of B and D could be argued correct at all times.

With rev 2 of question it is now Sat.

46 F

3 N

S C

10 47 F

3 N

S C

7 48 H

3 B

S C

7 2008 NRC a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 49 F

3 N

S C

14 50 F

2 N

S A

7 51 F

3 N

S C

7 52 F

2 N

S A

7 53 H

3 N

S C

5 54 H

2 N

S B

4 must change due to overlap with Q29. New question written after first validation. With rev 2 of the question file it is now Sat.

55 H

3 M

S D

7 mod from 2008 NRC Q27 56 F

3 B

S D

10 2010 NRC Q67 57 H

3 N

S A

10 stem focus was poor, licensee repaired stem and with Rev 2 of question it is now Sat.

58 F

2 N

S D

10 59 F

3 B

S A

5 2009 NRC Q9 60 H

3 N

S C

5 Poor validation (licensee states LOD = 5). Licensee wrote new question. With Rev 2 question it is now Sat.

61 H

2 M

S A

7 Mod from 2017 NRC Q21 62 H

3 B

S A

5 2012 NRC Q58 a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 63 F

2 N

S B

10 need to change distracter D because "Discipline Supervisor" is a fleet term for Entergy but is not used at Wat-3 and confused validators. Licensee used a different distracter and with Rev 2 file it is now Sat.

64 F

2 B

S A

2 2010 NRC Q69 65 F

2 N

S B

10 66 F

3 N

S B

7 67 F

3 N

S A

11 68 F

2 N

S B

7 69 H

2 B

S C

12 2012 NRC Q73 70 H

2 N

S C

12 71 F

2 N

S C

10 Licensee requested stem improvement after validation. NRC agreed to stem mod and with rev2 the question is now Sat.

72 F

2 N

S B

10 73 H

3 M

S C

5 mod from 2009 NRC Q61 74 H

3 N

S D

10 Licensee stated LOD = 5 for complexity of pump start times and memorization values. Wrote new question after first validation.

Question is Sat with rev 2 question.

75 H

3 N

S D

7 76 H

3 B

S A

5 2012 NRC Q11 SRO 77 F

3 M

S D

5 2006 NRC Q87 a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 78 H

3 B

S C

5 2015 NRC SROQ13 79 F

3 N

S A

1 80 H

3 N

S C

5 Didn't validate well.Licensee stated that LOD is 5 on this question. Modified stem and choices to reduce LOD and with rev2 question is now Sat.

81 H

3 B

S D

5 2017 NRC Q76 82 F

3 N

S B

2 Rev 1 question didn't validate well so modified stem slightly and with rev2 question is now Sat.

83 H

3 N

S B

2 Licensee didn't like question on basis of operational validity to take out all heater banks during a single maintenance activity.

Proposed modified stem and and with rev2 of question it is now Sat.

84 H

3 B

S C

5 2011 NRC Q77 85 H

2 N

S B

5 Didn't validate well due to LOD 5 contention by licensee. Modified question and choices with rev2 and it is now Sat.

86 H

2 B

S A

2 2015 NRC SRO Q8 87 H

3 B

S B

5 2006 NRC Q91 88 H

3 B

S A

5 2014 NRC Q86 89 H

3 M

S A

5 2014 SRO Q86 a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 90 H

3 N

S C

2 91 H

3 M

S D

5 2018 NRC Exam Wrote new question and it didn't validate well due to LOD 5 by licensee. KA was too obscure for NI's and core voiding. Changed KA and wrote modified question for second validation and with this change (Rev2) it is now Sat.

92 H

2 M

S B

4 2014 Q8 SRO NRC Wrote new question but licensee stated that valves no longer leak due to repairs and procedure had been updated during exam dev.

Licensee sent updated procedure.

Modified a 2014 question to replace this question and with rev2 it is now Sat.

93 H

3 B

S D

6 2008 NRC After first validation agreed that there were too many Inst Air topics on this exam so changed KA to refueling topic to meet sample requirements of 55.43.

Wrote new question on Refueling, didn't validate well so selected a bank question with rev 2 and question is now Sat.

94 F

2 N

S C

5 question didn't validate well and licensee proposed new question on Fluorides. It didn't validate well.

New question on Clorides and it validated better and with Rev 2 this question is now Sat. a Rev 3

ES-401 GG-2021-10 Written Examination Review Worksheet Form ES-401-9 95 H

2 N

S B

2 96 F

2 N

S C

3 97 H

3 N

S D

5 98 F

3 B

S A

4 2017 NRC SRO Q23 99 F

3 N

S D

5 100 H

3 N

S A

5 Results Table RO LOK -H 41 54.67 Avg RO LOD 2.71 Flaws 10 CFR Distribution RO LOK-F 34 AVG SRO LOD 2.76 Stem focus 0

41(b)1 0

43(b)1 1 SRO LOK - H 18 Overall LOD 2.72 Cues 0

41(b)2 1

43(b)2 5 SRO LOK - F 7

T/F 0

41(b)3 0

43(b)3 1 Cred Dist 0

41(b)4 1

43(b)4 2 RO Bank 12 16 SRO Bank 9

36 Partial 0

41(b)5 13 43(b)5 15 RO Mod 3

4 SRO Mod 4

16 job link 0

41(b)6 0

43(b)6 1 RO New 60 80 SRO New 12 48 units 0

41(b)7 35 43(b)7 0 minutia 0

41(b)8 1

Total Bank 21 21 backward 0

41(b)9 0

Total Mod 7

7 KA 0

41(b)10 19 Total New 72 72 SRO-only 0

41(b)11 1

LOD = 1 0

41(b)12 2

RO Sat 75 100 SRO Sat 25 100 41(b)13 0

RO Unsat 0

0 SRO Unsat 0

0 Answer Dist ( in %)

41(b)14 2

RO Edit 0

0 SRO Edit 0

0 RO-A 20 SRO-A 7

RO-B 19 SRO-B 6

Total Sat 100 100 Total Unsat 0

0 RO-C 19 SRO-C 6

Total Edit 0

0 RO-D 17 SRO-D 6 a Rev 3

Question Overlap with the Previous Two NRC Exams QA form ES-401-6 item 4 requires 4 or less questions on RO exam from previous two NRC exams, SRO portion two or less questions from previous two NRC exams*

• if this is exceeded, the region shall call NRR to explain why/reasons.

RO overlap from previous two NRC exams (2018-11 AND 2020-08 EXAMS)

Q32 2018-11 Q41 SRO overlap from the previous two NRC exams (2018-11 AND 2020-08 EXAMS)

Q91 modified from 2018-11

Questions Requiring References to Answer 6-10 questions rule of thumb on SRO portion of exam are allowed to be open reference, while the RO exam is generally none RO Questions that require hand-outs none SRO Questions that require hand-outs Q100 EAL wall chart