ML100640178: Difference between revisions
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
||
Line 17: | Line 17: | ||
=Text= | =Text= | ||
{{#Wiki_filter:}} | {{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9 | ||
: 3. Psychometric Flaws | |||
: 4. Job Content Flaws | |||
: 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 1 F 2 Y N S 2 H 3 Y M S Will EDG load a bus after an OC condition? Load shedding occur? RE: Yes, bus is deenergized and picked up by EDG. | |||
3 H 3 Y M S 4 F @ 1 X N N U OP Bkrs open after 30 seconds? Closed not a creditable distractor with question as is? K/A is ability question not a knowledge question. Question too easy. RE: Changed second bullet to when breaker opens -.. | |||
5 H 2 N S 6 H 3 Y N S 7 H 2 Y N S 8 H@ 2 Y B S 9 H 2 Y B S 10 F 2 Y M S 11 H 3 Y M S Licensee provided TS Bases as reference to RO question, RE: Licensee has included same information in lesson plans for RO's. TS Bases provided to NRC review for information only. Ref to TS basis (and TS basis) removed. | |||
12 F@ 2 Y N S 13 F 2 Y B S 14 F 3 Y B S 15 F 3 Y B S K/A asks for effect to NPSH not vortexing! RE: Same symptoms but different causes. RE: Keep question as is! | |||
16 H@ 3 Y B S 17 F 3 Y B S 18 F 3 Y N S 1 ES-401 Written Examination Review Worksheet Form ES-401-9 | |||
: 3. Psychometric Flaws | |||
: 4. Job Content Flaws | |||
: 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 19 H 2 Y B S 20 H@ 2 N M S K/A requires using EOPS in conjunction with AOPs. AOP's not included in stem or distractors. RE: Actions in distractors are from AOPs. Will keep question as is. | |||
21 H 3 Y N S Is this information that would be know by an RO. RE: Yes 22 H@ 2 Y N S 23 F 3 Y N S 24 H 2 Y B S 25 H@ 3 Y M S 26 F 2 Y N S 27 F 3 Y B S 28 H 3 Y B S 29 H@ 2 X Y M E Per the explanation, distractor "C" is correct, not distractor "A" as indicated. Why is "B" not correct? RE: Information provided to support correct answer is in error. Will change justification for correct answer. Since Q & A were correct, changed this from "U" to "E" 30 H 3 Y N S 31 H 2 Y B S 32 H@ 2 Y B S 33 F 2 Y M S Why is "A" incorrect. Chief believed that rods would indicate '00' post-scram? RE: If ind. Rod scram sw. | |||
held in position, rod would indicate '-' on full core display, not '00' 34 H 2 Y M S 35 F 2 Y M S 36 H@ 2 Y B S 37 H 3 Y M S 2 ES-401 Written Examination Review Worksheet Form ES-401-9 | |||
: 3. Psychometric Flaws | |||
: 4. Job Content Flaws | |||
: 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 38 F 2 Y B S 39 F 2 Y B S 40 H 2 Y B S 41 F@ 3 Y B S 42 H 2 Y M S 43 H 2 Y N S 44 F 2 Y N S 45 F@ 3 Y B S 46 H 3 Y M S 47 H 3 Y N S 48 F@ 2 Y B E Remove 'is' after DG. RE: Comment incorporated. | |||
49 H 3 Y M S 50 F 2 Y M S Need procedural guidance for 11 o'clock position. RE: provided in QCOP 6600-02, step F.2.j! | |||
51 F 2 Y M S 52 F 2 Y M S Is 1B SWP normally selected for standby? RE: Yes, it's the only SWP with an auto start feature. | |||
53 F@ 2 Y B S 54 F@ 2 Y B S 55 H @ 4 Y B S What value, "Blocks to Full" on RWM? K/A ask effect of loss of malfunction of RWM. RE: Malfunction is in DFWLC which inputs to RWM. "Blocks to Full disabled" = doesn't enforce rod blocks above 10% pwr. 56 H@ 2 Y B S 57 H@ 2 Y B S 3 ES-401 Written Examination Review Worksheet Form ES-401-9 | |||
: 3. Psychometric Flaws | |||
: 4. Job Content Flaws | |||
: 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 58 F 2 Y B S 59 F 2 Y B S 60 H@ 2 Y N S 61 H 2 Y N S 62 H 2 Y M S 63 F@ 3 Y B S 64 H 2 Y N S 65 F 2 Y B S 66 F 2 Y M S 67 F@ 2 Y B S 68 H 2 Y M S 69 F 3 Y B S 70 F 2 Y M S 71 F@ 2 Y N S 72 F 2 Y N S 73 F 2 Y B S 74 H 3 Y M S 75 F@ 2 Y B E Switch B and A distractors so that they are in order from least significant to most significant! RE: Comment incorporated. | |||
1 H @ 3 Y Y N S Per QCOA 0202-04, D.2 actions to insert CRAM rods appears to be correct action! Why is distractor "A" incorrect? RE: Insert ALL CRAM rods is in correct! Only insert enough to get out of Region II. | |||
4 ES-401 Written Examination Review Worksheet Form ES-401-9 | |||
: 3. Psychometric Flaws | |||
: 4. Job Content Flaws | |||
: 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 2 H 3 Y Y N S 3 M 4 Y Y N S 4 H 2 Y Y N S 5 H@ 2 Y Y N S 6 H 2 Y Y N S 7 H 3 Y Y N S 8 H 2 Y Y M S 9 H@ 2 Y Y N S 10 H 3 Y Y N S K/A asks for H concentrations and H monitoring availability. RE: Providing H concentration provi des no additional information to question. Stem already has SRO in QGA 200-5, Hydrogen Control. | |||
11 H 3 Y Y N S 12 F@ 3 Y Y N S If RCIC flow controller placed in Manual at 400 gpm, would that still render RCIC inoperable? "D" distractor, would RCIC eventually need to be declared inoperable for this condition? RE: Yes, RCIC is still inoperable. Yes, RCIC eventually would be declared inop. Stem asks for IMMEDIATE actions! | |||
13 H 3 N Y N S K/A A2 requires predicting impact and procedure to correct. TS is used as procedure but no predicting of impact in answer. RE: In order to deduce the correct TS, applicant would need to know what impact was. | |||
14 H 3 N Y N U K/A A2 requires predicting impact AND procedure to correct control or mitigate-. Missing procedure reference. RE: Procedure references were pulled out to make the question look neater. Reference procedures put back in distractors. | |||
15 H@ 3 Y Y N S 16 H 2 Y Y N S 17 H 2 Y Y M S 18 H@ 2 Y Y N S 5 ES-401 Written Examination Review Worksheet Form ES-401-9 | |||
: 3. Psychometric Flaws | |||
: 4. Job Content Flaws | |||
: 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 19 H 2 Y Y M S 20 F 3 Y Y B S 21 F@ 2 Y Y B E Remove "temporary" from stem since answer contains "temporary." RE: Will replace with 'portable." | |||
22 F 2 Y Y M S 23 F 2 Y Y B S 24 F 2 Y Y N S 25 H@ 3 X Y Y B U Last bullet in stem states, " .. ready to provide SAMG decision making. Correct distractor includes entering into SAMGs. Clue in stem (SAMG) needs to be deleted. RE: moved bullet from stem to distractor "A." | |||
Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] | |||
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. | |||
: 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). | |||
: 3. Check the appropriate box if a psychometric flaw is identified: | |||
* The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much n eedless information). | |||
* The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). | |||
* The answer choices are a collection of unrelated true/false statements. | |||
* The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. | |||
* One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: | |||
* The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). | |||
* The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). | |||
* The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with que stion in gallons). | |||
* The question requires reverse logic or application compared to the job requirements. | |||
: 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). | |||
6 ES-401 Written Examination Review Worksheet Form ES-401-9 7 @ - Denotes 30 initial review questions that verified references, K/A conformance, level of difficulty and level of knowledge. | |||
RKWalton 10/20/2009}} |
Revision as of 15:50, 19 September 2018
ML100640178 | |
Person / Time | |
---|---|
Site: | Quad Cities ![]() |
Issue date: | 11/30/2009 |
From: | NRC/RGN-III |
To: | Exelon Generation Co |
References | |
50-254/09-301, 50-265/09-301, ES-401, ES-401-9 50-254/09-301, 50-265/09-301 | |
Download: ML100640178 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 1 F 2 Y N S 2 H 3 Y M S Will EDG load a bus after an OC condition? Load shedding occur? RE: Yes, bus is deenergized and picked up by EDG.
3 H 3 Y M S 4 F @ 1 X N N U OP Bkrs open after 30 seconds? Closed not a creditable distractor with question as is? K/A is ability question not a knowledge question. Question too easy. RE: Changed second bullet to when breaker opens -..
5 H 2 N S 6 H 3 Y N S 7 H 2 Y N S 8 H@ 2 Y B S 9 H 2 Y B S 10 F 2 Y M S 11 H 3 Y M S Licensee provided TS Bases as reference to RO question, RE: Licensee has included same information in lesson plans for RO's. TS Bases provided to NRC review for information only. Ref to TS basis (and TS basis) removed.
12 F@ 2 Y N S 13 F 2 Y B S 14 F 3 Y B S 15 F 3 Y B S K/A asks for effect to NPSH not vortexing! RE: Same symptoms but different causes. RE: Keep question as is!
16 H@ 3 Y B S 17 F 3 Y B S 18 F 3 Y N S 1 ES-401 Written Examination Review Worksheet Form ES-401-9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 19 H 2 Y B S 20 H@ 2 N M S K/A requires using EOPS in conjunction with AOPs. AOP's not included in stem or distractors. RE: Actions in distractors are from AOPs. Will keep question as is.
21 H 3 Y N S Is this information that would be know by an RO. RE: Yes 22 H@ 2 Y N S 23 F 3 Y N S 24 H 2 Y B S 25 H@ 3 Y M S 26 F 2 Y N S 27 F 3 Y B S 28 H 3 Y B S 29 H@ 2 X Y M E Per the explanation, distractor "C" is correct, not distractor "A" as indicated. Why is "B" not correct? RE: Information provided to support correct answer is in error. Will change justification for correct answer. Since Q & A were correct, changed this from "U" to "E" 30 H 3 Y N S 31 H 2 Y B S 32 H@ 2 Y B S 33 F 2 Y M S Why is "A" incorrect. Chief believed that rods would indicate '00' post-scram? RE: If ind. Rod scram sw.
held in position, rod would indicate '-' on full core display, not '00' 34 H 2 Y M S 35 F 2 Y M S 36 H@ 2 Y B S 37 H 3 Y M S 2 ES-401 Written Examination Review Worksheet Form ES-401-9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 38 F 2 Y B S 39 F 2 Y B S 40 H 2 Y B S 41 F@ 3 Y B S 42 H 2 Y M S 43 H 2 Y N S 44 F 2 Y N S 45 F@ 3 Y B S 46 H 3 Y M S 47 H 3 Y N S 48 F@ 2 Y B E Remove 'is' after DG. RE: Comment incorporated.
49 H 3 Y M S 50 F 2 Y M S Need procedural guidance for 11 o'clock position. RE: provided in QCOP 6600-02, step F.2.j!
51 F 2 Y M S 52 F 2 Y M S Is 1B SWP normally selected for standby? RE: Yes, it's the only SWP with an auto start feature.
53 F@ 2 Y B S 54 F@ 2 Y B S 55 H @ 4 Y B S What value, "Blocks to Full" on RWM? K/A ask effect of loss of malfunction of RWM. RE: Malfunction is in DFWLC which inputs to RWM. "Blocks to Full disabled" = doesn't enforce rod blocks above 10% pwr. 56 H@ 2 Y B S 57 H@ 2 Y B S 3 ES-401 Written Examination Review Worksheet Form ES-401-9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 58 F 2 Y B S 59 F 2 Y B S 60 H@ 2 Y N S 61 H 2 Y N S 62 H 2 Y M S 63 F@ 3 Y B S 64 H 2 Y N S 65 F 2 Y B S 66 F 2 Y M S 67 F@ 2 Y B S 68 H 2 Y M S 69 F 3 Y B S 70 F 2 Y M S 71 F@ 2 Y N S 72 F 2 Y N S 73 F 2 Y B S 74 H 3 Y M S 75 F@ 2 Y B E Switch B and A distractors so that they are in order from least significant to most significant! RE: Comment incorporated.
1 H @ 3 Y Y N S Per QCOA 0202-04, D.2 actions to insert CRAM rods appears to be correct action! Why is distractor "A" incorrect? RE: Insert ALL CRAM rods is in correct! Only insert enough to get out of Region II.
4 ES-401 Written Examination Review Worksheet Form ES-401-9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 2 H 3 Y Y N S 3 M 4 Y Y N S 4 H 2 Y Y N S 5 H@ 2 Y Y N S 6 H 2 Y Y N S 7 H 3 Y Y N S 8 H 2 Y Y M S 9 H@ 2 Y Y N S 10 H 3 Y Y N S K/A asks for H concentrations and H monitoring availability. RE: Providing H concentration provi des no additional information to question. Stem already has SRO in QGA 200-5, Hydrogen Control.
11 H 3 Y Y N S 12 F@ 3 Y Y N S If RCIC flow controller placed in Manual at 400 gpm, would that still render RCIC inoperable? "D" distractor, would RCIC eventually need to be declared inoperable for this condition? RE: Yes, RCIC is still inoperable. Yes, RCIC eventually would be declared inop. Stem asks for IMMEDIATE actions!
13 H 3 N Y N S K/A A2 requires predicting impact and procedure to correct. TS is used as procedure but no predicting of impact in answer. RE: In order to deduce the correct TS, applicant would need to know what impact was.
14 H 3 N Y N U K/A A2 requires predicting impact AND procedure to correct control or mitigate-. Missing procedure reference. RE: Procedure references were pulled out to make the question look neater. Reference procedures put back in distractors.
15 H@ 3 Y Y N S 16 H 2 Y Y N S 17 H 2 Y Y M S 18 H@ 2 Y Y N S 5 ES-401 Written Examination Review Worksheet Form ES-401-9
- 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other Q# 1. LOK (F/H) 2. LOD (1-5) Stem Focus Cues T/F Cred.Dist. Partial Job-Link Minutia#/ units Back-ward Q= K/A SROOnly 6. B/M/N 7. U/E/S 8. Explanation 19 H 2 Y Y M S 20 F 3 Y Y B S 21 F@ 2 Y Y B E Remove "temporary" from stem since answer contains "temporary." RE: Will replace with 'portable."
22 F 2 Y Y M S 23 F 2 Y Y B S 24 F 2 Y Y N S 25 H@ 3 X Y Y B U Last bullet in stem states, " .. ready to provide SAMG decision making. Correct distractor includes entering into SAMGs. Clue in stem (SAMG) needs to be deleted. RE: moved bullet from stem to distractor "A."
Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much n eedless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with que stion in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
6 ES-401 Written Examination Review Worksheet Form ES-401-9 7 @ - Denotes 30 initial review questions that verified references, K/A conformance, level of difficulty and level of knowledge.
RKWalton 10/20/2009