ML102220153: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
 
Line 15: Line 15:


=Text=
=Text=
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
{{#Wiki_filter:ES-401                                                           Written Examination Review Worksheet                                                                     Form ES-401-9 DC COOK 2010 Initial License Exam
: 4. Job Content Flaws
: 1.       2.         3. Psychometric Flaws               4. Job Content Flaws         5. Other   6.     7.                                     8.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 1 H 2         Y N S 2 H 3         Y B S 3 F 2         Y B S 1/2 4 H+ 3   X     Y B S 1/2 Not plausible that Thot would rise post reactor trip/turbine trip with RCS flow in other loops. For "D" distractor, suggest Tcold and Thot remain stable. RE: 2 validators chose distractor "D" as a correct answer. Possible misconception - acceptable.
Q#  LOK      LOD (F/H)    (1-5)  Stem Cues         T/F   Cred. Partial Job- Minutia #/ Back-             Q= SRO B/M/N U/E/S                                      Explanation Focus                      Dist.        Link            units ward      K/A Only 1     H       2                                                                             Y           N     S 2     H       3                                                                             Y           B     S 3     F       2                                                                             Y           B     S     1/2 4   H+       3                               X                                             Y           B     S     1/2 Not plausible that Thot would rise post reactor trip/turbine trip with RCS flow in other loops. For D distractor, suggest Tcold and Thot remain stable. RE: 2 validators chose distractor D as a correct answer. Possible misconception - acceptable.
5 F 2         Y B S   Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
5     F       2                                                                             Y           B     S Instructions
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
: 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
: 1.       Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 3. Check the appropriate box if a psychometric flaw is identified:
: 2.       Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
: 3.       Check the appropriate box if a psychometric flaw is identified:
* The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
* The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
* The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
* The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
* The answer choices are a collection of unrelated true/false statements.
* The answer choices are a collection of unrelated true/false statements.
* The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
* The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
* One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:
* One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 4.       Check the appropriate box if a job content error is identified:
* The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
* The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
* The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
* The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
* The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with que stion in gallons).
* The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
* The question requires reverse logic or application compared to the job requirements.
* The question requires reverse logic or application compared to the job requirements.
: 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
: 5.       Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
A "+" in the "Q#" column indicates that question was reviewed as part of the representative sample of 30 questions.
: 6.       Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
: 7.       Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8.       At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.


ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
ES-401                                               Written Examination Review Worksheet                                                     Form ES-401-9 DC COOK 2010 Initial License Exam
: 4. Job Content Flaws
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                     8.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6 H 2         Y N S 1/2 7+ H 3         Y N S 8 H 3         Y N S 1/2 9 H 3         Y B S 1/2 Are RO's taught basis for AOP'S? RE: Yes same as SRO objectives.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link          units ward K/A Only 6   H     2                                                             Y       N   S   1/2 7+   H     3                                                             Y       N   S 8   H     3                                                             Y       N   S   1/2 9   H     3                                                             Y       B   S   1/2 Are ROs taught basis for AOPS? RE: Yes same as SRO objectives.
10 F 3         Y B S 11+ H 3         Y B S 12 H 4         Y M S 13 F 2         Y N S 14+ F 2         Y B S 15 H 1         Y X N U This is a direct look up. Do RO's need to know how to navigate TRMs? RE: Modified question such that RO's do not need to know time req'ts of TRM. Deleted reference material.
10   F     3                                                             Y       B   S 11+   H     3                                                             Y       B   S 12   H     4                                                             Y       M   S 13   F     2                                                             Y       N   S 14+   F     2                                                             Y       B   S 15   H     1                                                             Y     X N   U   This is a direct look up. Do ROs need to know how to navigate TRMs? RE: Modified question such that ROs do not need to know time reqts of TRM. Deleted reference material.
16 F 2         Y B S 17 H 2         Y B S 18+ F 3         Y N E Nothing in reference provided provides basis for correct answer! RE: Will provide references during OV week. RE: Have references provided via CD! RO's expected to know this information.
16   F     2                                                             Y       B   S 17   H     2                                                             Y       B   S 18+   F     3                                                             Y       N   E   Nothing in reference provided provides basis for correct answer! RE:
19 H 4         Y B S 20 H 3         Y B S 21+ H 3         Y B S I didn't understand the reasons provided for the incorrect answers.
Will provide references during OV week. RE: Have references provided via CD! ROs expected to know this information.
Wouldn't distracters A & C cause Regen Hxer outlet temp (chg flow) to rise, but would not account for rise in CCP flow rates?! RE: Changed wording of explanation for A & C distracters being incorrect.
19   H     4                                                             Y       B   S 20   H     3                                                             Y       B   S 21+   H     3                                                             Y       B   S   I didnt understand the reasons provided for the incorrect answers.
22 H 3         Y M S 23 H 3         Y B S 24 F 3         Y B S ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
Wouldnt distracters A & C cause Regen Hxer outlet temp (chg flow) to rise, but would not account for rise in CCP flow rates?! RE:
: 4. Job Content Flaws
Changed wording of explanation for A & C distracters being incorrect.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 25+ F 2          Y  N S  26 H 4          Y  M S  27 H 3          Y  N S 1/2 28+ H 3          Y  M S  29  H 2          Y  B S  30 F 2          Y  B S  31+ F 2          Y  B S  32 F 2          Y  B S  33 H 2          Y  B S  34+ H 3          Y  M S  35 F 3          Y  B S  36 H 3          Y  M S  37 H 4          Y  N S 2 gpm loss from letdown, but 8 gpm loss from RCPs?  That's 2 gpm per pump??!  RE:  Examiner reeducated!
22   H     3                                                             Y       M   S 23   H     3                                                             Y       B   S 24   F     3                                                             Y       B    S
38+ H 4          Y  B S  39 H 4   X      Y  B U S With Rx trip, P10 condition to deenergize SR instruments is not satisified. Therefore distracter "A" is correct. RE:  P10 fails in the tripped condition, thus deenergizing both SR NI's.
40 F 3          Y  B S  41 H 3          Y  B S  42+ F 3          Y  B S What is feedwater conservation, and why will it not occur?  RE: 


Means if there is a line break, don't put all water on floor. Throttle flow. 43 H 2          Y  B S  44 H 2          Y  N E Acceptable 'level'?  RE: Will use "Range" ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
ES-401                                               Written Examination Review Worksheet                                                       Form ES-401-9 DC COOK 2010 Initial License Exam
: 4. Job Content Flaws
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                                     8.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 45+ H 2         Y N S 46 H 2          Y N S 47 H 2          Y N U S Is there a difference between answers if there was a steam line isolation trip vs an OTDT trip?  Reference material provided does not support Distractor "C"!   RE:  No MSL Isolation trip for PWR design. 1085 won't lift per simulato r eval. Keep Q as is.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link          units ward K/A Only 25+   F    2                                                             Y       N   S 26  H     4                                                              Y       M    S 27  H     3                                                              Y       N   S  1/2 28+   H    3                                                             Y       M   S 29  H     2                                                             Y       B   S 30  F     2                                                             Y       B   S 31+   F    2                                                              Y       B   S 32  F    2                                                             Y       B   S 33  H     2                                                             Y       B   S 34+   H    3                                                              Y       M    S 35  F     3                                                              Y       B   S 36  H    3                                                             Y       M   S 37  H    4                                                              Y       N    S   2 gpm loss from letdown, but 8 gpm loss from RCPs? Thats 2 gpm per pump??! RE: Examiner reeducated!
48+ F 2          Y  B S  49 F 3         Y B S May want to capitalize "FAILURES."  RE:  Don't capitalize failures at DC Cook. 50 F 3          Y  M S 51+ H 2         Y B S 1/2 52 F 2         Y B S 53 H 3          Y  B S  54+ H 3          Y B S 55 H 2         Y B S 56 H 2         Y B S What design of EDG do they have?  (Not EMD!) Woodward EDGs spin at 514 rpm.
38+   H    4                                                              Y       B   S 39  H     4                       X                                    Y        B   U   With Rx trip, P10 condition to deenergize SR instruments is not satisified. Therefore distracter A is correct. RE: P10 fails in the S
57+ F 2          Y N S 58 F 2          Y B S 59 F 3         Y M S 60+ F 3          Y B S 61 F 2         Y  B S  62 H 2         N  B U What does this question have to do with RBCCW or TBCCW water temperatures? RE: Replaced Q with Q from Watts Bar.
tripped condition, thus deenergizing both SR NIs.
63+ F 2          Y B S 64 F 2          Y  B S ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
40  F     3                                                              Y       B   S 41  H     3                                                             Y       B   S 42+   F     3                                                             Y       B   S   What is feedwater conservation, and why will it not occur? RE:
: 4. Job Content Flaws
Means if there is a line break, dont put all water on floor. Throttle flow.
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 65 H 2          Y  B S  66 F 3          Y  B S  67+ F 2          Y  B S  68 F 2          Y  N S  69 F 2          Y  B E "D" distracter does not appear to be plausible. How can equipt be in an LCO but be operable?  RE: Changed D distractor.
43  H     2                                                             Y       B   S 44  H     2                                                             Y       N   E   Acceptable level? RE: Will use Range
70+ F 2          Y B S 71 F 2          Y  B S  72 H 3         Y B S 73 H 2          Y  B S  74+ F 3         Y B S Are RO applicants expected to memorize fold out page conditions in Emergency procedures? RE: Yes.
75 F 3          Y  B E Typo in "D" distracter. RE:  Corrected.
76+ H 3          Y Y M S TS 3.1.4.B would be more restricted. Is it possible SRO applicant would say "B" is correct answer?  RE:  SRO training includes that BOTH TS's are to be entered.
77 H 2          Y Y N E Not "F" question since must analyze given conditions and determine ATWS exists and procedural references must be known. RE:  Changed to H.
78 H 3          Y N M E Not "F" question since must analyze given conditions and determine procedural useage. Not a memory question. RE:  Changed to H.
79+ H 3          Y N? M E Would SRO applicants alone know this?  RE:  Direct actions- and add procedure references to "B" and "C" distracters. RE: Modified question to two part answer, added references to distracters "B" &
"C". 80 H 2         Y Y B S 81 H 3          Y Y N S  82 H 2         Y Y B S ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 83+ H 1          Y Y N U 1/2, Part 2 of this question (ODCM) is a direct lookup. RE: Will review later! 84 F 3          Y N N S How is this a SRO only question?  RE:  The SRO only part is the application of the note through the implementation of the step. The Notes in the ECA are SRO only knowledge.
85 H 2  X        Y Y B U 1. Should not be providing applicant basis section of TS to answer


this question. TS 3.0.3 is not provided with reference for this question. 2. Direct look up from TS. RE: 1. Licensee changed the way references are provided such that references are in "book" order rather than in "test" order. 2. LsubA is not information known to operators. Deleted reference to LsubA and simply stated that ctmt leakage was sat and unsat.
ES-401                                                Written Examination Review Worksheet                                                      Form ES-401-9 DC COOK 2010 Initial License Exam
86+ F 3         Y Y B S 87 H 3         Y Y B S 88 H 3         Y Y B S 89+ F 2         Y Y M S 90 H 3         Y Y B S 91 F 2         Y Y N S 92 H 3         Y Y B S 93+ H 3         Y Y B E 94 H 3         Y Y B S 1/2 95 H 3         Y Y B S 96 H 3         X Y B U S How does this meet K/A for TS? RE: The references are part of TS.
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                      8.
97+ H 3         Y Y M S 98 F 3         Y Y B S 99 H 2         Y Y B E Not an "F" rating, since applicants must review various system parameters and determine which CSF path is most important. RE:
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F      Cred. Partial Job- Minutia #/ Back-    Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link          units ward K/A Only 45+  H    2                                                              Y        N    S 46  H    2                                                              Y        N    S 47  H    2                                                              Y        N    U  Is there a difference between answers if there was a steam line isolation trip vs an OTDT trip? Reference material provided does not S
support Distractor C! RE: No MSL Isolation trip for PWR design.
1085 wont lift per simulator eval. Keep Q as is.
48+  F    2                                                              Y        B    S 49  F    3                                                              Y        B    S  May want to capitalize FAILURES. RE: Dont capitalize failures at DC Cook.
50  F    3                                                              Y        M    S 51+  H    2                                                              Y        B    S  1/2 52  F    2                                                              Y        B    S 53  H    3                                                              Y        B    S 54+  H    3                                                              Y        B    S 55  H    2                                                              Y        B    S 56  H    2                                                              Y        B    S  What design of EDG do they have? (Not EMD!) Woodward EDGs spin at 514 rpm.
57+  F    2                                                              Y        N    S 58  F    2                                                              Y        B    S 59  F    3                                                              Y        M    S 60+  F    3                                                              Y        B    S 61  F    2                                                              Y        B    S 62  H    2                                                              N        B    U  What does this question have to do with RBCCW or TBCCW water temperatures? RE: Replaced Q with Q from Watts Bar.
63+  F    2                                                              Y        B    S 64  F    2                                                              Y        B    S
 
ES-401                                                Written Examination Review Worksheet                                                    Form ES-401-9 DC COOK 2010 Initial License Exam
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                    8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F      Cred. Partial Job- Minutia #/ Back-    Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link          units ward K/A Only 65  H    2                                                              Y        B    S 66  F    3                                                              Y        B    S 67+  F    2                                                              Y        B    S 68  F    2                                                              Y        N    S 69  F    2                                                              Y        B    E  D distracter does not appear to be plausible. How can equipt be in an LCO but be operable? RE: Changed D distractor.
70+  F    2                                                              Y        B    S 71  F    2                                                              Y        B    S 72  H    3                                                              Y        B    S 73  H    2                                                              Y        B    S 74+  F    3                                                              Y        B    S  Are RO applicants expected to memorize fold out page conditions in Emergency procedures? RE: Yes.
75  F    3                                                              Y        B    E  Typo in D distracter. RE: Corrected.
76+  H    3                                                              Y    Y  M    S  TS 3.1.4.B would be more restricted. Is it possible SRO applicant would say B is correct answer? RE: SRO training includes that BOTH TSs are to be entered.
77  H    2                                                              Y    Y  N    E  Not F question since must analyze given conditions and determine ATWS exists and procedural references must be known. RE:
Changed to H.
78  H    3                                                              Y    N  M    E  Not F question since must analyze given conditions and determine procedural useage. Not a memory question. RE: Changed to H.
79+  H    3                                                              Y    N? M    E  Would SRO applicants alone know this? RE: Direct actions and add procedure references to B and C distracters. RE: Modified question to two part answer, added references to distracters B &
C.
80  H    2                                                              Y    Y  B    S 81  H    3                                                              Y    Y  N    S 82  H    2                                                              Y    Y  B    S
 
ES-401                                                Written Examination Review Worksheet                                                      Form ES-401-9 DC COOK 2010 Initial License Exam
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other 6. 7.                                    8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F      Cred. Partial Job- Minutia #/ Back-    Q= SRO B/M/N U/E/S                                Explanation Focus              Dist.        Link          units ward K/A Only 83+  H    1                                                              Y    Y  N    U  1/2, Part 2 of this question (ODCM) is a direct lookup. RE: Will review later!
84  F    3                                                              Y    N  N    S  How is this a SRO only question? RE: The SRO only part is the application of the note through the implementation of the step. The Notes in the ECA are SRO only knowledge.
85  H    2            X                                                Y    Y  B    U  1. Should not be providing applicant basis section of TS to answer this question. TS 3.0.3 is not provided with reference for this question. 2. Direct look up from TS. RE: 1. Licensee changed the way references are provided such that references are in book order rather than in test order. 2. LsubA is not information known to operators. Deleted reference to LsubA and simply stated that ctmt leakage was sat and unsat.
86+   F     3                                                             Y     Y B   S 87   H     3                                                             Y     Y B   S 88   H     3                                                             Y     Y B   S 89+   F     2                                                             Y     Y M   S 90   H     3                                                             Y     Y B   S 91   F     2                                                             Y     Y N   S 92   H     3                                                             Y     Y B   S 93+   H     3                                                             Y     Y B   E 94   H     3                                                             Y     Y B   S   1/2 95   H     3                                                             Y     Y B   S 96   H     3                                                             X     Y B   U   How does this meet K/A for TS? RE: The references are part of TS.
S 97+   H     3                                                             Y     Y M   S 98   F     3                                                             Y     Y B   S 99   H     2                                                             Y     Y B   E   Not an F rating, since applicants must review various system parameters and determine which CSF path is most important. RE:
Changed to H.
Changed to H.
ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
 
: 4. Job Content Flaws
ES-401                                                   Written Examination Review Worksheet                                               Form ES-401-9 DC COOK 2010 Initial License Exam
: 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 100 F 3         Y Y B S   Comment about references provided: 1. Each question using references should have "
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other 6. 7.                               8.
References Provided" at the end of the question but without telling the applicant what references he/she should look at. 2. All references to question numbers should be deleted from the references provided. Applicant should have enough knowledge to determine which documents from references provided are required to answer the question. Licensee response: The licensee agreed to remove direct references for each question and revise the examinee cue to, "References Provided."  Additionally, the references are now in "book" order rather than in "question" order.}}
Q#  LOK    LOD (F/H)  (1-5) Stem Cues T/F       Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                            Explanation Focus              Dist.        Link          units ward K/A Only 100   F     3                                                               Y     Y B     S Comment about references provided: 1. Each question using references should have References Provided at the end of the question but without telling the applicant what references he/she should look at. 2. All references to question numbers should be deleted from the references provided. Applicant should have enough knowledge to determine which documents from references provided are required to answer the question. Licensee response: The licensee agreed to remove direct references for each question and revise the examinee cue to, References Provided. Additionally, the references are now in book order rather than in question order.}}

Latest revision as of 15:52, 13 November 2019

DC Cook, 2010 Initial License Exam, Written Exam Comments, 401-9 Form
ML102220153
Person / Time
Site: Prairie Island  Xcel Energy icon.png
Issue date: 06/21/2010
From:
Division Reactor Projects III
To:
References
Download: ML102220153 (7)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 Y N S 2 H 3 Y B S 3 F 2 Y B S 1/2 4 H+ 3 X Y B S 1/2 Not plausible that Thot would rise post reactor trip/turbine trip with RCS flow in other loops. For D distractor, suggest Tcold and Thot remain stable. RE: 2 validators chose distractor D as a correct answer. Possible misconception - acceptable.

5 F 2 Y B S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:
  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
  • The answer choices are a collection of unrelated true/false statements.
  • The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:
  • The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  • The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
  • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 H 2 Y N S 1/2 7+ H 3 Y N S 8 H 3 Y N S 1/2 9 H 3 Y B S 1/2 Are ROs taught basis for AOPS? RE: Yes same as SRO objectives.

10 F 3 Y B S 11+ H 3 Y B S 12 H 4 Y M S 13 F 2 Y N S 14+ F 2 Y B S 15 H 1 Y X N U This is a direct look up. Do ROs need to know how to navigate TRMs? RE: Modified question such that ROs do not need to know time reqts of TRM. Deleted reference material.

16 F 2 Y B S 17 H 2 Y B S 18+ F 3 Y N E Nothing in reference provided provides basis for correct answer! RE:

Will provide references during OV week. RE: Have references provided via CD! ROs expected to know this information.

19 H 4 Y B S 20 H 3 Y B S 21+ H 3 Y B S I didnt understand the reasons provided for the incorrect answers.

Wouldnt distracters A & C cause Regen Hxer outlet temp (chg flow) to rise, but would not account for rise in CCP flow rates?! RE:

Changed wording of explanation for A & C distracters being incorrect.

22 H 3 Y M S 23 H 3 Y B S 24 F 3 Y B S

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 25+ F 2 Y N S 26 H 4 Y M S 27 H 3 Y N S 1/2 28+ H 3 Y M S 29 H 2 Y B S 30 F 2 Y B S 31+ F 2 Y B S 32 F 2 Y B S 33 H 2 Y B S 34+ H 3 Y M S 35 F 3 Y B S 36 H 3 Y M S 37 H 4 Y N S 2 gpm loss from letdown, but 8 gpm loss from RCPs? Thats 2 gpm per pump??! RE: Examiner reeducated!

38+ H 4 Y B S 39 H 4 X Y B U With Rx trip, P10 condition to deenergize SR instruments is not satisified. Therefore distracter A is correct. RE: P10 fails in the S

tripped condition, thus deenergizing both SR NIs.

40 F 3 Y B S 41 H 3 Y B S 42+ F 3 Y B S What is feedwater conservation, and why will it not occur? RE:

Means if there is a line break, dont put all water on floor. Throttle flow.

43 H 2 Y B S 44 H 2 Y N E Acceptable level? RE: Will use Range

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 45+ H 2 Y N S 46 H 2 Y N S 47 H 2 Y N U Is there a difference between answers if there was a steam line isolation trip vs an OTDT trip? Reference material provided does not S

support Distractor C! RE: No MSL Isolation trip for PWR design.

1085 wont lift per simulator eval. Keep Q as is.

48+ F 2 Y B S 49 F 3 Y B S May want to capitalize FAILURES. RE: Dont capitalize failures at DC Cook.

50 F 3 Y M S 51+ H 2 Y B S 1/2 52 F 2 Y B S 53 H 3 Y B S 54+ H 3 Y B S 55 H 2 Y B S 56 H 2 Y B S What design of EDG do they have? (Not EMD!) Woodward EDGs spin at 514 rpm.

57+ F 2 Y N S 58 F 2 Y B S 59 F 3 Y M S 60+ F 3 Y B S 61 F 2 Y B S 62 H 2 N B U What does this question have to do with RBCCW or TBCCW water temperatures? RE: Replaced Q with Q from Watts Bar.

63+ F 2 Y B S 64 F 2 Y B S

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 65 H 2 Y B S 66 F 3 Y B S 67+ F 2 Y B S 68 F 2 Y N S 69 F 2 Y B E D distracter does not appear to be plausible. How can equipt be in an LCO but be operable? RE: Changed D distractor.

70+ F 2 Y B S 71 F 2 Y B S 72 H 3 Y B S 73 H 2 Y B S 74+ F 3 Y B S Are RO applicants expected to memorize fold out page conditions in Emergency procedures? RE: Yes.

75 F 3 Y B E Typo in D distracter. RE: Corrected.

76+ H 3 Y Y M S TS 3.1.4.B would be more restricted. Is it possible SRO applicant would say B is correct answer? RE: SRO training includes that BOTH TSs are to be entered.

77 H 2 Y Y N E Not F question since must analyze given conditions and determine ATWS exists and procedural references must be known. RE:

Changed to H.

78 H 3 Y N M E Not F question since must analyze given conditions and determine procedural useage. Not a memory question. RE: Changed to H.

79+ H 3 Y N? M E Would SRO applicants alone know this? RE: Direct actions and add procedure references to B and C distracters. RE: Modified question to two part answer, added references to distracters B &

C.

80 H 2 Y Y B S 81 H 3 Y Y N S 82 H 2 Y Y B S

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 83+ H 1 Y Y N U 1/2, Part 2 of this question (ODCM) is a direct lookup. RE: Will review later!

84 F 3 Y N N S How is this a SRO only question? RE: The SRO only part is the application of the note through the implementation of the step. The Notes in the ECA are SRO only knowledge.

85 H 2 X Y Y B U 1. Should not be providing applicant basis section of TS to answer this question. TS 3.0.3 is not provided with reference for this question. 2. Direct look up from TS. RE: 1. Licensee changed the way references are provided such that references are in book order rather than in test order. 2. LsubA is not information known to operators. Deleted reference to LsubA and simply stated that ctmt leakage was sat and unsat.

86+ F 3 Y Y B S 87 H 3 Y Y B S 88 H 3 Y Y B S 89+ F 2 Y Y M S 90 H 3 Y Y B S 91 F 2 Y Y N S 92 H 3 Y Y B S 93+ H 3 Y Y B E 94 H 3 Y Y B S 1/2 95 H 3 Y Y B S 96 H 3 X Y B U How does this meet K/A for TS? RE: The references are part of TS.

S 97+ H 3 Y Y M S 98 F 3 Y Y B S 99 H 2 Y Y B E Not an F rating, since applicants must review various system parameters and determine which CSF path is most important. RE:

Changed to H.

ES-401 Written Examination Review Worksheet Form ES-401-9 DC COOK 2010 Initial License Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 100 F 3 Y Y B S Comment about references provided: 1. Each question using references should have References Provided at the end of the question but without telling the applicant what references he/she should look at. 2. All references to question numbers should be deleted from the references provided. Applicant should have enough knowledge to determine which documents from references provided are required to answer the question. Licensee response: The licensee agreed to remove direct references for each question and revise the examinee cue to, References Provided. Additionally, the references are now in book order rather than in question order.

Retrieved from "https://kanterella.com/w/index.php?title=ML102220153&oldid=2129441"