ML113330734: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
 
(6 intermediate revisions by the same user not shown)
Line 2: Line 2:
| number = ML113330734
| number = ML113330734
| issue date = 09/22/2011
| issue date = 09/22/2011
| title = Salem, Units 1 & 2 - NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)
| title = NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)
| author name =  
| author name =  
| author affiliation = NRC/RGN-I
| author affiliation = NRC/RGN-I
Line 9: Line 9:
| docket = 05000272, 05000311
| docket = 05000272, 05000311
| license number = DPR-070, DPR-075
| license number = DPR-070, DPR-075
| contact person = JACKSON D E RGN-I/DRS/OB/610-337-5306
| contact person = JACKSON D  RGN-I/DRS/OB/610-337-5306
| case reference number = TAC U01830
| case reference number = TAC U01830
| package number = ML110030673
| package number = ML110030673
| document type = License-Operator, Part 55 Examination Related Material
| document type = License-Operator, Part 55 Examination Related Material
| page count = 8
| page count = 8
| project = TAC:U01830
| stage = Draft Other
}}
}}
=Text=
{{#Wiki_filter:ES-401                                                    Written Examination Review Worksheet                                                        Form ES-401-9
: 1. 2.      3. Psychometric Flaws            4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#    LOK  LOD (F/H) (1-5) Stem Cues T/F        Creel. Partial Job- Minutia #1      Back- Q= SRO B/M/N U/E/S                                Explanation Focus                Dist.          Link          units ward KIA Only 1      F                                                                                  N    U  Overlap with SRO #16. replaced question.
Modified Choices C and D to enhance discriminatory value.
2      H                                                                                  N    E Added additional information to justification section.
Recategorized as memory level, not higher order.
3      F                                X                                                N    U Modified stem and distracters to make distracters credible.
Modified stem, added information relating to A TWS condition to 4      H                                                                                  B    E  enhance credibility of the 'two-pump' choices. Modified choices to improve discriminatory value.
(Draft 0#18) 5      M                                                                                  N    E  Question replaced because of overlap with tested knowledge in Questions 29 and 56.
(Draft Q#5) 6      F                                                                                  B    E  Minor distractor wording change. Changed value in distractor to improve plausibility. Added to justification to explain why motor temp may not be increasing with 10 flow alarm.
(Draft Q#6) 7      F                                                                                  B    E Replaced one implausible distractor, reordered choices.
(Draft 0#7) 8      H                                                                                  B    E  Removed "references provided" statement from the stem. Modified choice values to prevent overlap between choices.
(Draft 0#8)
Reworded several choices to improve plausibility and eliminate overlap with Q#20.
9      F                                                                                  N    E This question matches KIA for applying system limits because the heat up of seal injection to 225°F in 5.3 minutes is a system limitation as documented in a loss of CCW calculation and in the Technical Basis Document for S2.0P-AB.CC-0001.
Page 1
ES-401                                                Written Examination Review Worksheet                                                Form ES-401-9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                    8.
0#  LOK  LOD (F/H) (1-5) Stem Cues TIF      Credo Partial Job- Minutia #1 Back 0= SRO B/M/N utE/S                              Explanation Focus              Dist.        Link          units ward KIA Only (Draft 0#9) 10    F                                                                            B  E  Removed "hotter" and "cooler" from Choices BID. Unnecessary, could be confusing and could cue the correct answer.
(Draft 0#10) 11    F                                                                            N  E  Changed running pump in the stem to enhance distractor credibility.
Reordered answer choices.
(Draft 0#11) 12    H                                                                            B  E  Removed unnecessary, leading information ("since it is not being discriminated out") from both Choice C and Choice D.
13    H                                                                            B  S  (Draft 0#12)
(Draft 0#13) 14    M                                                                            N  E  Fixed typo in Choice D. Modified wording in Choices A and B for symmetry. Corrected reference page number.
15    H                                                                            N  S  (Draft 0#14)
(Draft 0#15) 16    H                                                                            N  E Replaced O. Too many GFE level questions on test.
(Draft 0#16) 17    F                              X                                              N  U  Modified choices to ensure each choice is mutually exclusive of the corresponding choice.
(Draft 0#17) 18    H    3.0                                                                      N  E Minor editorial changes in distracters.
Page 2
ES-401                                                                        2                                                              Form ES-401-9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                    8.
LOK  LOD Q#  (FIH) (1-5) Stem Cues TlF      Credo Partial Job- Minutia #1 Back- Q= SRO BIMtN utEtS                                Explanation Focus              Dist.        Link          units ward KIA Only 19    H                                                                            B  S 20    F                                                                            N  S  Modified Choice C to prevent overlap with Q#9 (Draft Q#8).
21    F                                                                            B  E  Replaced one non-credible distractor. Reordered choices.
22    H                                                                            B  S  Reclassified as higher cognitive level question.
23    M                                                                            B  U  Low LOD. Replaced question.
24    H                                                                            B  S H                                                                            B  S  Added to justification to explain that lockout switch is plausible 25 because it is a Unit difference.
26    F                                                                            N  S 27    F                                                                            N  S  Reclassified as a lower cognitive level question.
Corrected minor typo in stam.
28    H                                                                            N  E  Added "initial" to stem question.
Modified wording of generator MW choice to make clearly wrong.
Minor stem wording change.
29    H                                                                            N  S Added to justification to elaborate on why Choice A is wrong.
30    H                                                                            B  S 31    H                                          X                        X        B  U  Weak KIA match and limited operational validity. Replaced Q.
32    H                                                                            N  S  Added to question justification to explain the KIA match.
33    H                                                                            N  S  Fixed typos in justification section.
34    H                                                                            N  S 35    F                                                                            M  S 36    H                                                                            B  S Page 3
ES-401                                                                        2                                                              Form ES*401*9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                    8.
LOK  LOD Q#  (F/H) (1-5) Stem Cues T/F      Credo Partial Job- Minutia #1 Back-    Q= SRO B/M/N UlE/S                              Explanation Focus              Dist.        Link          units ward KIA Only Modified conditions in stem to remove the leak and replace with 37    H                                                                            B    E  different fault. Changed to address the overlap between 0#37 and 0#3.
H                                    X                                        B    E  Added transmitter O/S in stem to improve distractor plausibility and 38 increase LOD.
39    F                                                                            B    S 40    F                                                                            N    S
* 41    H                                                                            B    S  Minor distractor wording change, rearranged choices.
* 42    H                                                                            B    S 43    H                                                                            N    S 44    H                                                                            N    S 45    H                                                                            B    S 46    F                                                                            B    S 47    F                                                                            N    S 48    F                      X                                                    B    U  Collection of TruelFalse statements. Rewrote Q to fix psychometric flaw.
49    H                                                                            N    S  Too many GFE level Qs on RO exam. Replaced with new Q.
50    F                                                                            N    E  Modified stem to make more operationally valid.
51    H                                                                            N    S 52    F                                                                            N    U  Low LOD based on initial reactor trip EOP immediate action, practiced many times in simulator scenarios. Question replaced.
53    F                                                                            N    E  Added "lAW ... " to stem.
* 54    H                                                                            N    E  Rewrote question and distracters to improve plausibility of choices.
55    H                                                                            N    S Page 4
ES-401                                                                        2                                                                  Form ES-401-9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                      8.
LOK  LOD 0#  (FtH) (1-5) Stem Cues TtF      Credo Partial Job- Minutia #t Back-    0= SRO B/M/N utEtS                                Explanation Focus              Dist.        Link          units ward KIA Only 56    H                                                                            B    S 57    F                                                                            B    S 58    H                                                                            N    S                                                                        I I
F                                                                            B    S  Corrected typo in justification section. Modified justification for 59 correct answer to explain TS.
F                                                                            N    E  Replaced distractor containing 2R1A because discriminatory value 60 reduced by 0#61. Reordered choices.
61    H                                                                            N    S 62    H                                                                            M    S 63    H                                                                            B    S 64    H                                                                            B    E  Replaced one implausible distractor. Reordered choices.
65    F                                                                            B    E  Modified stem and one distractor to improve distractor plausibility.
66    F                                                                            N    E  Modified stem to enhance distractor plausibility.
67    F                                                                            N    S 68    H                                                                            N    S  Facility confirmed this is expected RO level knowledge.
H                                                                            B    E  Modified justification to align with key answer. Reduced total pri to 69 sec leakage in stem to ensure per-generator value is within limits.
70    H                                                                            B    S                                                                        .
SRO 0#22 helps to answer this question. However, RO exam is          !
71    H                                                                            B    S  collected before SRO exam distributed to applicants. Therefore, no    i need for change to SRO 0#22.                                          I Removed "the" from Choice C. Added to "Unit 1 trips" choice, to 72    H                                                                            B    E  provide information for assessment that emergency not in progress.
Reordered choices.
73    F                                                                            B    S Page 5
ES-401                                                                        2                                                                    Form ES-401-9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other  6. 7.                                      8.
LOK  LOD Q#  (F/H) (1-5) Stem Cues T/F      Credo Partial Job- Minutia #1 Back-    Q=    SRO B/M/N utEtS                                Explanation Focus              Dist.        Link          units ward KIA Only F                                                                              B    E  Modified a distractor to remove leading information. Replaced one 74 distractor with a plausible incorrect choice.
75    F                              X                                                N    U  Modified stem. Replaced ChOices Band D with plausible distracters.
(SRO#1) 76    H                                                                              M    E Deleted mode information from the stem.
(SRO #2) 77    H                                                                              N    E  Provided RCS temperature and pressure parameters, vice plant mode, in the stem.
(SRO #3) 78    H                                                                              N    E  Modified the choice wording to ensure question requires procedure knowledge to answer correctly.
(SRO #4) 79    H                                                                              N    E  Changed Choice C from "2FR" to "1 FR" to align with the designated unit Improved stem clarity by changing to " ... and which would require ._."_
80    H                                                                              N    S  (SRO#5)
(SRO#6)
If "O/S" means out of service, which could be interpreted as inoperable, then Choice C is not credible since it wouldn't change 81    H                                                                              N    E  conditions. Perhaps better to state 22 RHR pump is in standby.
Choice D could be argued as a second correct answer, based on assumption that the channel is inoperable because of its audible function.
82    H                                                                              B    S  (SRO#7) 83    H                                                                              N    s  (SRO#8) 84    H                                                                              N    S  (SRO#9)
(SRO #10) 85    H                                                                              B    E Rewrote stem and distrac!ers to better test procedure knowledge.
Page 6
ES-401                                                                            2                                                              Form ES-401-9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws        5. Other 6. 7.                                    8.
LOK  LOD Q#  (F/H) (1-5) Stem Cues T/F      Credo Partial Job- Minutia #1 Back-          Q= SRO B/M/N U/E/S                            Explanation Focus              Dist.        Link          units ward      KIA Only (SRO #11) (DRAFT SRO #12) 86    H                              X                                                  N    E Replaced Choice A distractor. Added explanation of KIA match.
87    H                                                                                  N    S  (SRO #12) (DRAFT SRO #13)
                                                                          ._-. ~.---~
(SRO #13) (DRAFT SRO #14) 88    H                                    X                                            N    U  Minor stem wording change. Reworded Choice A to eliminate the 2 00 correct answer.
(SRO #14) (DRAFT SRO #15) 89    H                                                                        X        N    U  KA mismatch. KA is for ESF AS. Question tests RPS. Wrote new question to same KA.
(SRO #15) (DRAFT SRO #16) 90    H                                                                                  B    E  Fixed typos. Minor stem rewording. Reworded distracters for clarity and to improve discriminatory value.
(SRO #16) (DRAFT SRO #11) 91    H                                                                              X  N    U Not SRO only. Replaced Q.
(SRO #17) 92    H                                                                                  N    E Minor wording changes in Choices C and D.
93    H                                                                                  M    S  (SRO #18)
(SRO#19) 94    H                                                                              X  B    U Could be answered with system knowledge only. Replaced question. i (SRO #20) 95    H                                                                              X  B    U  Not SRO-only. Can be answered with basic GFE knowledge.
Replaced question.
96    H                                                                                  B    S  (SRO #21)
Page 7
ES-401                                                                          2                                                                Form ES-401-9
: 1. 2.        3. Psychometric Flaws          4. Job Content Flaws    5. Other 6. 7.                                    8.
LOK  LOD Q#  (F/H) (1-5) Stem Cues TIF      Credo Partial Job- Minutia #1      Back- Q= SRO B/M/N utE/S                              Explanation Focus              Dist.        Link          units ward  KIA Only (SRO#22)
Minor stem wording change to ensure all choices are supported by the question.
97    H                                                                              B    E This question contains information that could help to eliminate two distracters on RO 0#71 by narrowing the annual PSE&G limit to between 1900 and 2350 mR. However. RO exam is collected before SRO exam distributed to applicants. Therefore, no need for change.
(SRO#23) 98    H                                                                              B    E Rewarded stem and choices to improve distractor plausibility.
(SRO#24) 99    H                              X                                                B    E  Rewrote stem conditions and choices to enhance discriminatory value of distracters.
100  H                                                                              N    S  (SRO #25)
Page 8}}

Latest revision as of 20:02, 6 February 2020

NRC Comments on Draft Licensee-Developed Exam (Written) (Folder 2)
ML113330734
Person / Time
Site: Salem  PSEG icon.png
Issue date: 09/22/2011
From:
NRC Region 1
To:
Public Service Enterprise Group
JACKSON D RGN-I/DRS/OB/610-337-5306
Shared Package
ML110030673 List:
References
TAC U01830
Download: ML113330734 (8)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Creel. Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 F N U Overlap with SRO #16. replaced question.

Modified Choices C and D to enhance discriminatory value.

2 H N E Added additional information to justification section.

Recategorized as memory level, not higher order.

3 F X N U Modified stem and distracters to make distracters credible.

Modified stem, added information relating to A TWS condition to 4 H B E enhance credibility of the 'two-pump' choices. Modified choices to improve discriminatory value.

(Draft 0#18) 5 M N E Question replaced because of overlap with tested knowledge in Questions 29 and 56.

(Draft Q#5) 6 F B E Minor distractor wording change. Changed value in distractor to improve plausibility. Added to justification to explain why motor temp may not be increasing with 10 flow alarm.

(Draft Q#6) 7 F B E Replaced one implausible distractor, reordered choices.

(Draft 0#7) 8 H B E Removed "references provided" statement from the stem. Modified choice values to prevent overlap between choices.

(Draft 0#8)

Reworded several choices to improve plausibility and eliminate overlap with Q#20.

9 F N E This question matches KIA for applying system limits because the heat up of seal injection to 225°F in 5.3 minutes is a system limitation as documented in a loss of CCW calculation and in the Technical Basis Document for S2.0P-AB.CC-0001.

Page 1

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

0# LOK LOD (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back 0= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only (Draft 0#9) 10 F B E Removed "hotter" and "cooler" from Choices BID. Unnecessary, could be confusing and could cue the correct answer.

(Draft 0#10) 11 F N E Changed running pump in the stem to enhance distractor credibility.

Reordered answer choices.

(Draft 0#11) 12 H B E Removed unnecessary, leading information ("since it is not being discriminated out") from both Choice C and Choice D.

13 H B S (Draft 0#12)

(Draft 0#13) 14 M N E Fixed typo in Choice D. Modified wording in Choices A and B for symmetry. Corrected reference page number.

15 H N S (Draft 0#14)

(Draft 0#15) 16 H N E Replaced O. Too many GFE level questions on test.

(Draft 0#16) 17 F X N U Modified choices to ensure each choice is mutually exclusive of the corresponding choice.

(Draft 0#17) 18 H 3.0 N E Minor editorial changes in distracters.

Page 2

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (FIH) (1-5) Stem Cues TlF Credo Partial Job- Minutia #1 Back- Q= SRO BIMtN utEtS Explanation Focus Dist. Link units ward KIA Only 19 H B S 20 F N S Modified Choice C to prevent overlap with Q#9 (Draft Q#8).

21 F B E Replaced one non-credible distractor. Reordered choices.

22 H B S Reclassified as higher cognitive level question.

23 M B U Low LOD. Replaced question.

24 H B S H B S Added to justification to explain that lockout switch is plausible 25 because it is a Unit difference.

26 F N S 27 F N S Reclassified as a lower cognitive level question.

Corrected minor typo in stam.

28 H N E Added "initial" to stem question.

Modified wording of generator MW choice to make clearly wrong.

Minor stem wording change.

29 H N S Added to justification to elaborate on why Choice A is wrong.

30 H B S 31 H X X B U Weak KIA match and limited operational validity. Replaced Q.

32 H N S Added to question justification to explain the KIA match.

33 H N S Fixed typos in justification section.

34 H N S 35 F M S 36 H B S Page 3

ES-401 2 Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N UlE/S Explanation Focus Dist. Link units ward KIA Only Modified conditions in stem to remove the leak and replace with 37 H B E different fault. Changed to address the overlap between 0#37 and 0#3.

H X B E Added transmitter O/S in stem to improve distractor plausibility and 38 increase LOD.

39 F B S 40 F N S

  • 41 H B S Minor distractor wording change, rearranged choices.
  • 42 H B S 43 H N S 44 H N S 45 H B S 46 F B S 47 F N S 48 F X B U Collection of TruelFalse statements. Rewrote Q to fix psychometric flaw.

49 H N S Too many GFE level Qs on RO exam. Replaced with new Q.

50 F N E Modified stem to make more operationally valid.

51 H N S 52 F N U Low LOD based on initial reactor trip EOP immediate action, practiced many times in simulator scenarios. Question replaced.

53 F N E Added "lAW ... " to stem.

  • 54 H N E Rewrote question and distracters to improve plausibility of choices.

55 H N S Page 4

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD 0# (FtH) (1-5) Stem Cues TtF Credo Partial Job- Minutia #t Back- 0= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only 56 H B S 57 F B S 58 H N S I I

F B S Corrected typo in justification section. Modified justification for 59 correct answer to explain TS.

F N E Replaced distractor containing 2R1A because discriminatory value 60 reduced by 0#61. Reordered choices.

61 H N S 62 H M S 63 H B S 64 H B E Replaced one implausible distractor. Reordered choices.

65 F B E Modified stem and one distractor to improve distractor plausibility.

66 F N E Modified stem to enhance distractor plausibility.

67 F N S 68 H N S Facility confirmed this is expected RO level knowledge.

H B E Modified justification to align with key answer. Reduced total pri to 69 sec leakage in stem to ensure per-generator value is within limits.

70 H B S .

SRO 0#22 helps to answer this question. However, RO exam is  !

71 H B S collected before SRO exam distributed to applicants. Therefore, no i need for change to SRO 0#22. I Removed "the" from Choice C. Added to "Unit 1 trips" choice, to 72 H B E provide information for assessment that emergency not in progress.

Reordered choices.

73 F B S Page 5

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only F B E Modified a distractor to remove leading information. Replaced one 74 distractor with a plausible incorrect choice.

75 F X N U Modified stem. Replaced ChOices Band D with plausible distracters.

(SRO#1) 76 H M E Deleted mode information from the stem.

(SRO #2) 77 H N E Provided RCS temperature and pressure parameters, vice plant mode, in the stem.

(SRO #3) 78 H N E Modified the choice wording to ensure question requires procedure knowledge to answer correctly.

(SRO #4) 79 H N E Changed Choice C from "2FR" to "1 FR" to align with the designated unit Improved stem clarity by changing to " ... and which would require ._."_

80 H N S (SRO#5)

(SRO#6)

If "O/S" means out of service, which could be interpreted as inoperable, then Choice C is not credible since it wouldn't change 81 H N E conditions. Perhaps better to state 22 RHR pump is in standby.

Choice D could be argued as a second correct answer, based on assumption that the channel is inoperable because of its audible function.

82 H B S (SRO#7) 83 H N s (SRO#8) 84 H N S (SRO#9)

(SRO #10) 85 H B E Rewrote stem and distrac!ers to better test procedure knowledge.

Page 6

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only (SRO #11) (DRAFT SRO #12) 86 H X N E Replaced Choice A distractor. Added explanation of KIA match.

87 H N S (SRO #12) (DRAFT SRO #13)

._-. ~.---~

(SRO #13) (DRAFT SRO #14) 88 H X N U Minor stem wording change. Reworded Choice A to eliminate the 2 00 correct answer.

(SRO #14) (DRAFT SRO #15) 89 H X N U KA mismatch. KA is for ESF AS. Question tests RPS. Wrote new question to same KA.

(SRO #15) (DRAFT SRO #16) 90 H B E Fixed typos. Minor stem rewording. Reworded distracters for clarity and to improve discriminatory value.

(SRO #16) (DRAFT SRO #11) 91 H X N U Not SRO only. Replaced Q.

(SRO #17) 92 H N E Minor wording changes in Choices C and D.

93 H M S (SRO #18)

(SRO#19) 94 H X B U Could be answered with system knowledge only. Replaced question. i (SRO #20) 95 H X B U Not SRO-only. Can be answered with basic GFE knowledge.

Replaced question.

96 H B S (SRO #21)

Page 7

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

LOK LOD Q# (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only (SRO#22)

Minor stem wording change to ensure all choices are supported by the question.

97 H B E This question contains information that could help to eliminate two distracters on RO 0#71 by narrowing the annual PSE&G limit to between 1900 and 2350 mR. However. RO exam is collected before SRO exam distributed to applicants. Therefore, no need for change.

(SRO#23) 98 H B E Rewarded stem and choices to improve distractor plausibility.

(SRO#24) 99 H X B E Rewrote stem conditions and choices to enhance discriminatory value of distracters.

100 H N S (SRO #25)

Page 8

Retrieved from "https://kanterella.com/w/index.php?title=ML113330734&oldid=2465158"