ML16116A362: Difference between revisions
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
StriderTol (talk | contribs) (Created page by program invented by StriderTol) |
||
| Line 15: | Line 15: | ||
=Text= | =Text= | ||
{{#Wiki_filter:}} | {{#Wiki_filter:Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 3 X B S 1 of 2 twice 2 H 3 X M S 3 H 3 X B S 4+ H 3 X N S 5 H 3 X B E NRC: Clarification of last bullet in stem, ie, "-makeup paths are OPERATING". LIC: Revised last bullet to clarify that "-BOTH SX Tower Makeup Pumps are RUNNING." 6 F 3 X N S Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: "Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). "Cues": The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). "T/F": The answer choices are a collection of unrelated true/false statements. "Cred. Dist": The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. "Partial": One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: "Job Link": The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). "Minutia": The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). "#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). "Backward": The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). A "+" in the "Q#" column indicates that question was reviewed as part of the representative sample of 30 questions. # 2 RO and 3 SRO Only questions submitted for NRC pre-review Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 7 F 3 X X B E NRC: Implausible distractor (A or B). B. is procedural (1BOA, RCP-2, Step5) sub-action of A. If A not true, then B would not be true. Rewrite/replace A or B. LIC: Changed Distractor B to "Check RCP thermal barrier return valve 1CC9438 - OPEN." This is a step in 1BOA PRI-6 for a CC malfunction with 1CC685 closed. It is an incomplete step in that the step has closed bullets listing 1CC685 AND 1CC9438 which makes it incorrect, but it is a plausible choice; however, it will not provide restoration of CC to the thermal barriers. 8+ H 3 X X M E 1 of 2 twice NRC: Why is "0" current plausible? Better distractor would be 20% indication for A and C (ie, wrong use of math). LIC: Changed A and C to 20% current. 9 F 3 2 X B E NRC: LOD=2, memorization of power supply. LIC: Changed to LOD 2. 10 H F H 3 2 X B E NRC: LOD=2, LOK=F, memorization of trip logic, fundamental knowledge (memory). LIC: Changed to LOD 2. Also changed stem of question to include reading and interpreting a gauge, given bistable trips and bypasses, and candidate must decide on how many further failures are required for 2 different ESF actuations. This is now a high cognitive level question. 11 H 3 X B S 12+ F 2 X B S 13 H 3 X B S 1 of 2 twice 14 H 3 X B S 15 H 3 X M S 16+ H 3 X M S 17 F 3 2 X B E NRC: LOD=2, knowledge of trip setpoint. LIC: Changed to LOD 2. 18 H 3 X B S 1 of 2 twice 19 H 3 X M B E NRC: Q M, changed condition in stem but did not change any of the distractors, or change the answer. LIC: Changed to Bank, not Modified. 20 H 3 X B S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 21+ F 3 X B S NRC: Answer key indicates both A. and B. are correct, but justification indicates A. is incorrect. LIC: Fixed the answer key, B is correct. 22 H F 3 2 X N E 1 of 2 twice NRC: 1) LOK=F, memory knowledge (like Q23); 2) LOD=2, memory knowledge, either know or don't know answer. LIC: Changed LOD to 2 and LOK to F. 23 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 24+ H 2 X B 25 H 3 X B E NRC: 1) Identify 0PRO9J and 1/2PR09J in stem. LIC: Added names of monitors. 26 F 3 X B S 2014 NRC ILE 27 H F H 3 2 X B E 2014 NRC ILE NRC: 1) LOD=2, reading simple straightforward gauge and recalling IA relief valve set-point is low level of difficulty. 2) LOK=F, straight forward reading only gauge provided, knowing trip set-point of IA relief valve is fundamental. 3) Clarify distractor D that lead SAC is running "-LOADED." LIC: 1) Changed to LOD 2. 2) Discussed LOK with Examiners, agreed it is H because of interpretation required of meter. 3) Clarified that lead SAC is running "-LOADED." 28+ F 3 2 X B E NRC: LOD=2; very straight forward question, picking out valve with specified isolation. LIC: Changed to LOD 2. 29 F 3 2 X X N E NRC: 1) LOD=2, straight forward question of power supply to reactor trip and bypass breakers. 2) Why are DC 113 and 114 plausible? LIC: 1) Changed LOD to 2. 2) Added to explanation for plausibility of DC 113 and 114. In short, because of the identical numbering of DC busses and AC instrument busses, and that AC 114 is ESF AS power supply, this is a common point of confusion for the ILT student. | ||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 30 H F H 3 2 X N E NRC: LOK=F, LOD=2; knowledge of Adverse Containment parameters is fundamental. Providing basically two different gauges to read each of the adverse parameters does not make the question challenging or greater than fundamental. LIC: Changed to LOD 2. Discussed LOK with Examiners, decided it IS H because of interpretation required of meter. 31 F 3 X B S 32+ F 3 X N S 33 F 3 2 X B E NRC: LOD=2. Question is borderline discriminating. LIC: Changed to LOD 2. 34 H 3 X N S 35 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 36+ H 3 X B S 37 H 3 2 X B E NRC: LOD=2, simple deduction that set-point not low enough to trip interlock and needs to be set lower. LIC: Changed to LOD 2. 38 H F H 3 2 X N E NRC: 1) Distractor A. action not plausible, ie, no switch or indication exists on MCR panel so why is starting 2A SAC from MCR panel believable? 2) LOK=F, similar to Q21. Fundamental knowledge (memory) of action to take and reading the provided gauge. 3) LOD=2, know no auto start and must start locally, LIC: (1) Changed Answer and 1 distractor to remove implausible distractor and to make the question more operationally correct. (2) Discussed LOK with Examiners, decided it is H because of interpretation required of meter. (3) Changed to LOD 2. 39 F 3 X B S 40+ H 3 X B S Handout: Steam Table 41 H 3 X N S 1 of 2 twice 42 H 3 X M S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 43 H 3 X X B U NRC: 1) A(1) and B(1) not plausible, ie, clogged SI line affects both charging and SI flow. 2) A(2) and B(2) could be same answer as written, ie, "throttle open" and "throttle fully open" could be same position. LIC: Changed A(1) and B(1) to a condition that ordinarily would lower Pzr level - Tave failing low. Added that Master level control is in AUTO to make those plausible. Changed B(2) to maintain 1CV182 at 52% demand. 44+ H F H 3 2 X B E Handout: Fig's: 1BOA Pri 10-3; 1BOA Pri 10-4 NRC: LOD=2, key words and phrases "after refueling" and "subcooled" words appear in stem and on the respective figures (curves), applicant only has to plot and locate the appropriate flowrate number. LOK=F, Knowing the associated meter is not necessary and only contributes to selecting the correct answer. LIC: 1) Changed LOD to 2. 2) Changed initial conditions in stem to eliminate statement "-after refueling" to "-RV head has NOT been detensioned." This changed the correct answer. 3) Also changed stem statement asking flowrate to maintain "the RCS in a subcooled condition" to "maintain the RCS temperature below its saturation temperature". This means the same as subcooled but is not a direct pointer to the correct curve. 4) Discussed LOK with Examiners, decided it is H because of interpretation required of the curves with the change in the wording. 5) Removed the flow meters stated in the answer and distractors. 45 H 3 X B S 46# H 3 X N S 47 H 3 X N S 1 of 2 twice 48+ H 3 X N S 49 F 3 2 X B E NRC: LOD=2, memorizing a number from CAUTION. LIC: Changed LOD to 2. 50 H 3 X N E NRC: 1) Wording for C. and D. doesn't make sense, ie, add "-to be fed-"? 2) D. identified as CORRECT in Answer Explanation. LIC: 1) Changed wording to make the meaning clearer. 2) Changed D to incorrect in explanation. 51 F 3 2 X N E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2. | |||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 52+ F 3 2 X B E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2. 53 F 3 X M S 54 H 3 X N S 55+ H 3 X B S 56# H 3 X B S Handout: BCB Fig 20a 57 F 3 X N S 58 F 3 X B E NRC: Add "Immediately" to A. to clarify when boration is to be stopped. LIC: Added IMMEDIATELY to choice A 59 H 3 X B S 60+ H F H 3 2 3 X B E NRC: LOK=F, LOD=2, simple knowledge (memory) of interlock.. LIC: Significantly revised question, such that candidate must determine what mode of operation steam dumps are in, whether it will have a temperature offset, whether the reference temperature is coming from no load or Tref. Added a likely distractor setpoint for condenser pressure based on a BOA. Changed to LOD 2 as noted (but it is now probably LOD 3. 61 F 3 X B S 62 H F H 3 X N E 1 of 2 twice NRC: 1) LOK=F, basic (fundamental memory) knowledge of valve control location and boration flowrate. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of meter. 63 F 3 X B S 64+ F 3 X X B S 65 F 3 X B S 66 F 2 X N S 1 of 2 twice 67 H F H 3 X N E NRC: LOK=F, basic (fundamental memory) knowledge of normal 100% RH system valve position. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of status lights. | |||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 68+ H 3 B U 1 of 2 twice NRC: Does not meet KA: How does question demonstrate ability to use procedure? LIC: Changed question into a 1 of 2 twice with the second part of the question procedurally directed actions. 69 F 3 X B S 70 F 3 X B S 71 H 3 B S 72+ F 3 1 2 X N B U NRC: LOD=1, this is required everyday knowledge for workers and operators signing into the plant each day. Not discriminating knowledge for licensed operator. LIC: Replaced with G2.4.1: BCA 0.0 immediate actions; LOD=2, LOK=F, Bank question. Updated ES-401-3, the Rejected K/As list, the Outline Change Form, and the Exam. 73 F 3 X N E 1 of 2 twice NRC: As written, there would be nothing wrong with notifying the RP before monitoring a second time. Modify stem to reflect the "-procedure requirement-". LIC: Added "In accordance with RP-AA-350-" 74 F 3 X X B S 75 F 3 X B S R RO Exam: LOK: H(50-60%)=42/75(56.0%); F=33/75(44%) N=20/75(26.7%), B(56 max)=49/75=65.3%, M=6/75(8%): U=3/75(4%); E=28/75(37.3%); LOD=2.72 76 H 3 X X N S 77 H 3 X X N S 78 F 3 X X N S 79 H 3 X X B S 80 F 3 X X N S 81 H 3 X X N S 82 H 3 X X N S 1 of 2 twice 83 F 3 X X M S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 84 H 3 X X B S 85 H 3 X X B S 1 of 2 twice 86 F 3 X X N S 1 of 2 twice 87# H 3 X X B S 88 H 3 X X N S Handout: EALs, EP-AA-1002, Add. 3 89 H 3 X X B S 90# F 3 X X N S 91 H 3 X X B S 92 H 3 2 X X B E Handout: TS 3.4.16 NRC: Is handout of TS provided? If so, why not a direct lookup? LOD=2, read TS, little discrimination. LIC: : Changed to LOD 2. Yes, TS is provided, but the candidate does not just read the TS. They must compare the different actions for the different activity levels, and must get the limit for I-133 from the Surveillance requirement and decide what constitutes the most limiting action. It is actions "below the line" removing it from RO knowledge, but has actions > 1 hour, so it's not required from memory. 93 H 3 X X B S 94 H 3 X X B S 95 F 3 X X N S 96 F 3 2 X X B E NRC: LOD=2, simple memory. LIC: Changed to LOD 2. 97 H F 3 2 X X M U NRC: 1) LOK=F, memory of cooldown limit; 2) LOD=D, simple math and memory of cooldown limit; 3) Not SRO Only, numbers above the TS line, RO required knowledge. LIC: Replaced question with a "basis" TS question. Rejected K/A and replaced. LOK=F, LOD=2 98# H 3 X X B S 99 F 3 X X B S 100 F 3 - X X N S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only For SRO portion of Exam: LOK: H(50%min)=15/25(60%); F=10/25(40%); N=11/25(44%), B(19 max)=12/25(48%), M=2/25(8%): U=1/25(4%); LOD=72/25=2.8 Were any Bank questions from previous NRC exams? Identified during validation. Remove handout reference from questions. Make sure question numbers aren't included on the Handout list to applicants during exam. Submit original questions for Modified questions to verify they were modified IAW ES 1021. Questions submitted. | |||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 3 X B S 1 of 2 twice 2 H 3 X M S 3 H 3 X B S 4+ H 3 X N S 5 H 3 X B E NRC: Clarification of last bullet in stem, ie, "-makeup paths are OPERATING". LIC: Revised last bullet to clarify that "-BOTH SX Tower Makeup Pumps are RUNNING." 6 F 3 X N S Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: "Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). "Cues": The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). "T/F": The answer choices are a collection of unrelated true/false statements. "Cred. Dist": The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. "Partial": One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: "Job Link": The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). "Minutia": The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). "#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). "Backward": The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). A "+" in the "Q#" column indicates that question was reviewed as part of the representative sample of 30 questions. # 2 RO and 3 SRO Only questions submitted for NRC pre-review Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 7 F 3 X X B E NRC: Implausible distractor (A or B). B. is procedural (1BOA, RCP-2, Step5) sub-action of A. If A not true, then B would not be true. Rewrite/replace A or B. LIC: Changed Distractor B to "Check RCP thermal barrier return valve 1CC9438 - OPEN." This is a step in 1BOA PRI-6 for a CC malfunction with 1CC685 closed. It is an incomplete step in that the step has closed bullets listing 1CC685 AND 1CC9438 which makes it incorrect, but it is a plausible choice; however, it will not provide restoration of CC to the thermal barriers. 8+ H 3 X X M E 1 of 2 twice NRC: Why is "0" current plausible? Better distractor would be 20% indication for A and C (ie, wrong use of math). LIC: Changed A and C to 20% current. 9 F 3 2 X B E NRC: LOD=2, memorization of power supply. LIC: Changed to LOD 2. 10 H F H 3 2 X B E NRC: LOD=2, LOK=F, memorization of trip logic, fundamental knowledge (memory). LIC: Changed to LOD 2. Also changed stem of question to include reading and interpreting a gauge, given bistable trips and bypasses, and candidate must decide on how many further failures are required for 2 different ESF actuations. This is now a high cognitive level question. 11 H 3 X B S 12+ F 2 X B S 13 H 3 X B S 1 of 2 twice 14 H 3 X B S 15 H 3 X M S 16+ H 3 X M S 17 F 3 2 X B E NRC: LOD=2, knowledge of trip setpoint. LIC: Changed to LOD 2. 18 H 3 X B S 1 of 2 twice 19 H 3 X M B E NRC: Q M, changed condition in stem but did not change any of the distractors, or change the answer. LIC: Changed to Bank, not Modified. 20 H 3 X B S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 21+ F 3 X B S NRC: Answer key indicates both A. and B. are correct, but justification indicates A. is incorrect. LIC: Fixed the answer key, B is correct. 22 H F 3 2 X N E 1 of 2 twice NRC: 1) LOK=F, memory knowledge (like Q23); 2) LOD=2, memory knowledge, either know or don't know answer. LIC: Changed LOD to 2 and LOK to F. 23 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 24+ H 2 X B 25 H 3 X B E NRC: 1) Identify 0PRO9J and 1/2PR09J in stem. LIC: Added names of monitors. 26 F 3 X B S 2014 NRC ILE 27 H F H 3 2 X B E 2014 NRC ILE NRC: 1) LOD=2, reading simple straightforward gauge and recalling IA relief valve set-point is low level of difficulty. 2) LOK=F, straight forward reading only gauge provided, knowing trip set-point of IA relief valve is fundamental. 3) Clarify distractor D that lead SAC is running "-LOADED." LIC: 1) Changed to LOD 2. 2) Discussed LOK with Examiners, agreed it is H because of interpretation required of meter. 3) Clarified that lead SAC is running "-LOADED." 28+ F 3 2 X B E NRC: LOD=2; very straight forward question, picking out valve with specified isolation. LIC: Changed to LOD 2. 29 F 3 2 X X N E NRC: 1) LOD=2, straight forward question of power supply to reactor trip and bypass breakers. 2) Why are DC 113 and 114 plausible? LIC: 1) Changed LOD to 2. 2) Added to explanation for plausibility of DC 113 and 114. In short, because of the identical numbering of DC busses and AC instrument busses, and that AC 114 is ESF AS power supply, this is a common point of confusion for the ILT student. | |||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 30 H F H 3 2 X N E NRC: LOK=F, LOD=2; knowledge of Adverse Containment parameters is fundamental. Providing basically two different gauges to read each of the adverse parameters does not make the question challenging or greater than fundamental. LIC: Changed to LOD 2. Discussed LOK with Examiners, decided it IS H because of interpretation required of meter. 31 F 3 X B S 32+ F 3 X N S 33 F 3 2 X B E NRC: LOD=2. Question is borderline discriminating. LIC: Changed to LOD 2. 34 H 3 X N S 35 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 36+ H 3 X B S 37 H 3 2 X B E NRC: LOD=2, simple deduction that set-point not low enough to trip interlock and needs to be set lower. LIC: Changed to LOD 2. 38 H F H 3 2 X N E NRC: 1) Distractor A. action not plausible, ie, no switch or indication exists on MCR panel so why is starting 2A SAC from MCR panel believable? 2) LOK=F, similar to Q21. Fundamental knowledge (memory) of action to take and reading the provided gauge. 3) LOD=2, know no auto start and must start locally, LIC: (1) Changed Answer and 1 distractor to remove implausible distractor and to make the question more operationally correct. (2) Discussed LOK with Examiners, decided it is H because of interpretation required of meter. (3) Changed to LOD 2. 39 F 3 X B S 40+ H 3 X B S Handout: Steam Table 41 H 3 X N S 1 of 2 twice 42 H 3 X M S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 43 H 3 X X B U NRC: 1) A(1) and B(1) not plausible, ie, clogged SI line affects both charging and SI flow. 2) A(2) and B(2) could be same answer as written, ie, "throttle open" and "throttle fully open" could be same position. LIC: Changed A(1) and B(1) to a condition that ordinarily would lower Pzr level - Tave failing low. Added that Master level control is in AUTO to make those plausible. Changed B(2) to maintain 1CV182 at 52% demand. 44+ H F H 3 2 X B E Handout: Fig's: 1BOA Pri 10-3; 1BOA Pri 10-4 NRC: LOD=2, key words and phrases "after refueling" and "subcooled" words appear in stem and on the respective figures (curves), applicant only has to plot and locate the appropriate flowrate number. LOK=F, Knowing the associated meter is not necessary and only contributes to selecting the correct answer. LIC: 1) Changed LOD to 2. 2) Changed initial conditions in stem to eliminate statement "-after refueling" to "-RV head has NOT been detensioned." This changed the correct answer. 3) Also changed stem statement asking flowrate to maintain "the RCS in a subcooled condition" to "maintain the RCS temperature below its saturation temperature". This means the same as subcooled but is not a direct pointer to the correct curve. 4) Discussed LOK with Examiners, decided it is H because of interpretation required of the curves with the change in the wording. 5) Removed the flow meters stated in the answer and distractors. 45 H 3 X B S 46# H 3 X N S 47 H 3 X N S 1 of 2 twice 48+ H 3 X N S 49 F 3 2 X B E NRC: LOD=2, memorizing a number from CAUTION. LIC: Changed LOD to 2. 50 H 3 X N E NRC: 1) Wording for C. and D. doesn't make sense, ie, add "-to be fed-"? 2) D. identified as CORRECT in Answer Explanation. LIC: 1) Changed wording to make the meaning clearer. 2) Changed D to incorrect in explanation. 51 F 3 2 X N E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2. | |||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 52+ F 3 2 X B E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2. 53 F 3 X M S 54 H 3 X N S 55+ H 3 X B S 56# H 3 X B S Handout: BCB Fig 20a 57 F 3 X N S 58 F 3 X B E NRC: Add "Immediately" to A. to clarify when boration is to be stopped. LIC: Added IMMEDIATELY to choice A 59 H 3 X B S 60+ H F H 3 2 3 X B E NRC: LOK=F, LOD=2, simple knowledge (memory) of interlock.. LIC: Significantly revised question, such that candidate must determine what mode of operation steam dumps are in, whether it will have a temperature offset, whether the reference temperature is coming from no load or Tref. Added a likely distractor setpoint for condenser pressure based on a BOA. Changed to LOD 2 as noted (but it is now probably LOD 3. 61 F 3 X B S 62 H F H 3 X N E 1 of 2 twice NRC: 1) LOK=F, basic (fundamental memory) knowledge of valve control location and boration flowrate. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of meter. 63 F 3 X B S 64+ F 3 X X B S 65 F 3 X B S 66 F 2 X N S 1 of 2 twice 67 H F H 3 X N E NRC: LOK=F, basic (fundamental memory) knowledge of normal 100% RH system valve position. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of status lights. | |||
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 68+ H 3 B U 1 of 2 twice NRC: Does not meet KA: How does question demonstrate ability to use procedure? LIC: Changed question into a 1 of 2 twice with the second part of the question procedurally directed actions. 69 F 3 X B S 70 F 3 X B S 71 H 3 B S 72+ F 3 1 2 X N B U NRC: LOD=1, this is required everyday knowledge for workers and operators signing into the plant each day. Not discriminating knowledge for licensed operator. LIC: Replaced with G2.4.1: BCA 0.0 immediate actions; LOD=2, LOK=F, Bank question. Updated ES-401-3, the Rejected K/As list, the Outline Change Form, and the Exam. 73 F 3 X N E 1 of 2 twice NRC: As written, there would be nothing wrong with notifying the RP before monitoring a second time. Modify stem to reflect the "-procedure requirement-". LIC: Added "In accordance with RP-AA-350-" 74 F 3 X X B S 75 F 3 X B S R RO Exam: LOK: H(50-60%)=42/75(56.0%); F=33/75(44%) N=20/75(26.7%), B(56 max)=49/75=65.3%, M=6/75(8%): U=3/75(4%); E=28/75(37.3%); LOD=2.72 76 H 3 X X N S 77 H 3 X X N S 78 F 3 X X N S 79 H 3 X X B S 80 F 3 X X N S 81 H 3 X X N S 82 H 3 X X N S 1 of 2 twice 83 F 3 X X M S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 84 H 3 X X B S 85 H 3 X X B S 1 of 2 twice 86 F 3 X X N S 1 of 2 twice 87# H 3 X X B S 88 H 3 X X N S Handout: EALs, EP-AA-1002, Add. 3 89 H 3 X X B S 90# F 3 X X N S 91 H 3 X X B S 92 H 3 2 X X B E Handout: TS 3.4.16 NRC: Is handout of TS provided? If so, why not a direct lookup? LOD=2, read TS, little discrimination. LIC: : Changed to LOD 2. Yes, TS is provided, but the candidate does not just read the TS. They must compare the different actions for the different activity levels, and must get the limit for I-133 from the Surveillance requirement and decide what constitutes the most limiting action. It is actions "below the line" removing it from RO knowledge, but has actions > 1 hour, so it's not required from memory. 93 H 3 X X B S 94 H 3 X X B S 95 F 3 X X N S 96 F 3 2 X X B E NRC: LOD=2, simple memory. LIC: Changed to LOD 2. 97 H F 3 2 X X M U NRC: 1) LOK=F, memory of cooldown limit; 2) LOD=D, simple math and memory of cooldown limit; 3) Not SRO Only, numbers above the TS line, RO required knowledge. LIC: Replaced question with a "basis" TS question. Rejected K/A and replaced. LOK=F, LOD=2 98# H 3 X X B S 99 F 3 X X B S 100 F 3 - X X N S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only For SRO portion of Exam: LOK: H(50%min)=15/25(60%); F=10/25(40%); N=11/25(44%), B(19 max)=12/25(48%), M=2/25(8%): U=1/25(4%); LOD=72/25=2.8 Were any Bank questions from previous NRC exams? Identified during validation. Remove handout reference from questions. Make sure question numbers aren't included on the Handout list to applicants during exam. Submit original questions for Modified questions to verify they were modified IAW ES 1021. Questions submitted.}} | |||
Revision as of 18:30, 28 May 2018
| ML16116A362 | |
| Person / Time | |
|---|---|
| Site: | Byron  |
| Issue date: | 03/23/2016 |
| From: | Bielby M E Operations Branch III |
| To: | Exelon Generation Co |
| References | |
| Download: ML16116A362 (9) | |
Text
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 3 X B S 1 of 2 twice 2 H 3 X M S 3 H 3 X B S 4+ H 3 X N S 5 H 3 X B E NRC: Clarification of last bullet in stem, ie, "-makeup paths are OPERATING". LIC: Revised last bullet to clarify that "-BOTH SX Tower Makeup Pumps are RUNNING." 6 F 3 X N S Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: "Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). "Cues": The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). "T/F": The answer choices are a collection of unrelated true/false statements. "Cred. Dist": The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. "Partial": One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: "Job Link": The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). "Minutia": The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). "#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). "Backward": The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). A "+" in the "Q#" column indicates that question was reviewed as part of the representative sample of 30 questions. # 2 RO and 3 SRO Only questions submitted for NRC pre-review Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 7 F 3 X X B E NRC: Implausible distractor (A or B). B. is procedural (1BOA, RCP-2, Step5) sub-action of A. If A not true, then B would not be true. Rewrite/replace A or B. LIC: Changed Distractor B to "Check RCP thermal barrier return valve 1CC9438 - OPEN." This is a step in 1BOA PRI-6 for a CC malfunction with 1CC685 closed. It is an incomplete step in that the step has closed bullets listing 1CC685 AND 1CC9438 which makes it incorrect, but it is a plausible choice; however, it will not provide restoration of CC to the thermal barriers. 8+ H 3 X X M E 1 of 2 twice NRC: Why is "0" current plausible? Better distractor would be 20% indication for A and C (ie, wrong use of math). LIC: Changed A and C to 20% current. 9 F 3 2 X B E NRC: LOD=2, memorization of power supply. LIC: Changed to LOD 2. 10 H F H 3 2 X B E NRC: LOD=2, LOK=F, memorization of trip logic, fundamental knowledge (memory). LIC: Changed to LOD 2. Also changed stem of question to include reading and interpreting a gauge, given bistable trips and bypasses, and candidate must decide on how many further failures are required for 2 different ESF actuations. This is now a high cognitive level question. 11 H 3 X B S 12+ F 2 X B S 13 H 3 X B S 1 of 2 twice 14 H 3 X B S 15 H 3 X M S 16+ H 3 X M S 17 F 3 2 X B E NRC: LOD=2, knowledge of trip setpoint. LIC: Changed to LOD 2. 18 H 3 X B S 1 of 2 twice 19 H 3 X M B E NRC: Q M, changed condition in stem but did not change any of the distractors, or change the answer. LIC: Changed to Bank, not Modified. 20 H 3 X B S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 21+ F 3 X B S NRC: Answer key indicates both A. and B. are correct, but justification indicates A. is incorrect. LIC: Fixed the answer key, B is correct. 22 H F 3 2 X N E 1 of 2 twice NRC: 1) LOK=F, memory knowledge (like Q23); 2) LOD=2, memory knowledge, either know or don't know answer. LIC: Changed LOD to 2 and LOK to F. 23 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 24+ H 2 X B 25 H 3 X B E NRC: 1) Identify 0PRO9J and 1/2PR09J in stem. LIC: Added names of monitors. 26 F 3 X B S 2014 NRC ILE 27 H F H 3 2 X B E 2014 NRC ILE NRC: 1) LOD=2, reading simple straightforward gauge and recalling IA relief valve set-point is low level of difficulty. 2) LOK=F, straight forward reading only gauge provided, knowing trip set-point of IA relief valve is fundamental. 3) Clarify distractor D that lead SAC is running "-LOADED." LIC: 1) Changed to LOD 2. 2) Discussed LOK with Examiners, agreed it is H because of interpretation required of meter. 3) Clarified that lead SAC is running "-LOADED." 28+ F 3 2 X B E NRC: LOD=2; very straight forward question, picking out valve with specified isolation. LIC: Changed to LOD 2. 29 F 3 2 X X N E NRC: 1) LOD=2, straight forward question of power supply to reactor trip and bypass breakers. 2) Why are DC 113 and 114 plausible? LIC: 1) Changed LOD to 2. 2) Added to explanation for plausibility of DC 113 and 114. In short, because of the identical numbering of DC busses and AC instrument busses, and that AC 114 is ESF AS power supply, this is a common point of confusion for the ILT student.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 30 H F H 3 2 X N E NRC: LOK=F, LOD=2; knowledge of Adverse Containment parameters is fundamental. Providing basically two different gauges to read each of the adverse parameters does not make the question challenging or greater than fundamental. LIC: Changed to LOD 2. Discussed LOK with Examiners, decided it IS H because of interpretation required of meter. 31 F 3 X B S 32+ F 3 X N S 33 F 3 2 X B E NRC: LOD=2. Question is borderline discriminating. LIC: Changed to LOD 2. 34 H 3 X N S 35 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 36+ H 3 X B S 37 H 3 2 X B E NRC: LOD=2, simple deduction that set-point not low enough to trip interlock and needs to be set lower. LIC: Changed to LOD 2. 38 H F H 3 2 X N E NRC: 1) Distractor A. action not plausible, ie, no switch or indication exists on MCR panel so why is starting 2A SAC from MCR panel believable? 2) LOK=F, similar to Q21. Fundamental knowledge (memory) of action to take and reading the provided gauge. 3) LOD=2, know no auto start and must start locally, LIC: (1) Changed Answer and 1 distractor to remove implausible distractor and to make the question more operationally correct. (2) Discussed LOK with Examiners, decided it is H because of interpretation required of meter. (3) Changed to LOD 2. 39 F 3 X B S 40+ H 3 X B S Handout: Steam Table 41 H 3 X N S 1 of 2 twice 42 H 3 X M S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 43 H 3 X X B U NRC: 1) A(1) and B(1) not plausible, ie, clogged SI line affects both charging and SI flow. 2) A(2) and B(2) could be same answer as written, ie, "throttle open" and "throttle fully open" could be same position. LIC: Changed A(1) and B(1) to a condition that ordinarily would lower Pzr level - Tave failing low. Added that Master level control is in AUTO to make those plausible. Changed B(2) to maintain 1CV182 at 52% demand. 44+ H F H 3 2 X B E Handout: Fig's: 1BOA Pri 10-3; 1BOA Pri 10-4 NRC: LOD=2, key words and phrases "after refueling" and "subcooled" words appear in stem and on the respective figures (curves), applicant only has to plot and locate the appropriate flowrate number. LOK=F, Knowing the associated meter is not necessary and only contributes to selecting the correct answer. LIC: 1) Changed LOD to 2. 2) Changed initial conditions in stem to eliminate statement "-after refueling" to "-RV head has NOT been detensioned." This changed the correct answer. 3) Also changed stem statement asking flowrate to maintain "the RCS in a subcooled condition" to "maintain the RCS temperature below its saturation temperature". This means the same as subcooled but is not a direct pointer to the correct curve. 4) Discussed LOK with Examiners, decided it is H because of interpretation required of the curves with the change in the wording. 5) Removed the flow meters stated in the answer and distractors. 45 H 3 X B S 46# H 3 X N S 47 H 3 X N S 1 of 2 twice 48+ H 3 X N S 49 F 3 2 X B E NRC: LOD=2, memorizing a number from CAUTION. LIC: Changed LOD to 2. 50 H 3 X N E NRC: 1) Wording for C. and D. doesn't make sense, ie, add "-to be fed-"? 2) D. identified as CORRECT in Answer Explanation. LIC: 1) Changed wording to make the meaning clearer. 2) Changed D to incorrect in explanation. 51 F 3 2 X N E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 52+ F 3 2 X B E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2. 53 F 3 X M S 54 H 3 X N S 55+ H 3 X B S 56# H 3 X B S Handout: BCB Fig 20a 57 F 3 X N S 58 F 3 X B E NRC: Add "Immediately" to A. to clarify when boration is to be stopped. LIC: Added IMMEDIATELY to choice A 59 H 3 X B S 60+ H F H 3 2 3 X B E NRC: LOK=F, LOD=2, simple knowledge (memory) of interlock.. LIC: Significantly revised question, such that candidate must determine what mode of operation steam dumps are in, whether it will have a temperature offset, whether the reference temperature is coming from no load or Tref. Added a likely distractor setpoint for condenser pressure based on a BOA. Changed to LOD 2 as noted (but it is now probably LOD 3. 61 F 3 X B S 62 H F H 3 X N E 1 of 2 twice NRC: 1) LOK=F, basic (fundamental memory) knowledge of valve control location and boration flowrate. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of meter. 63 F 3 X B S 64+ F 3 X X B S 65 F 3 X B S 66 F 2 X N S 1 of 2 twice 67 H F H 3 X N E NRC: LOK=F, basic (fundamental memory) knowledge of normal 100% RH system valve position. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of status lights.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 68+ H 3 B U 1 of 2 twice NRC: Does not meet KA: How does question demonstrate ability to use procedure? LIC: Changed question into a 1 of 2 twice with the second part of the question procedurally directed actions. 69 F 3 X B S 70 F 3 X B S 71 H 3 B S 72+ F 3 1 2 X N B U NRC: LOD=1, this is required everyday knowledge for workers and operators signing into the plant each day. Not discriminating knowledge for licensed operator. LIC: Replaced with G2.4.1: BCA 0.0 immediate actions; LOD=2, LOK=F, Bank question. Updated ES-401-3, the Rejected K/As list, the Outline Change Form, and the Exam. 73 F 3 X N E 1 of 2 twice NRC: As written, there would be nothing wrong with notifying the RP before monitoring a second time. Modify stem to reflect the "-procedure requirement-". LIC: Added "In accordance with RP-AA-350-" 74 F 3 X X B S 75 F 3 X B S R RO Exam: LOK: H(50-60%)=42/75(56.0%); F=33/75(44%) N=20/75(26.7%), B(56 max)=49/75=65.3%, M=6/75(8%): U=3/75(4%); E=28/75(37.3%); LOD=2.72 76 H 3 X X N S 77 H 3 X X N S 78 F 3 X X N S 79 H 3 X X B S 80 F 3 X X N S 81 H 3 X X N S 82 H 3 X X N S 1 of 2 twice 83 F 3 X X M S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 84 H 3 X X B S 85 H 3 X X B S 1 of 2 twice 86 F 3 X X N S 1 of 2 twice 87# H 3 X X B S 88 H 3 X X N S Handout: EALs, EP-AA-1002, Add. 3 89 H 3 X X B S 90# F 3 X X N S 91 H 3 X X B S 92 H 3 2 X X B E Handout: TS 3.4.16 NRC: Is handout of TS provided? If so, why not a direct lookup? LOD=2, read TS, little discrimination. LIC: : Changed to LOD 2. Yes, TS is provided, but the candidate does not just read the TS. They must compare the different actions for the different activity levels, and must get the limit for I-133 from the Surveillance requirement and decide what constitutes the most limiting action. It is actions "below the line" removing it from RO knowledge, but has actions > 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, so it's not required from memory. 93 H 3 X X B S 94 H 3 X X B S 95 F 3 X X N S 96 F 3 2 X X B E NRC: LOD=2, simple memory. LIC: Changed to LOD 2. 97 H F 3 2 X X M U NRC: 1) LOK=F, memory of cooldown limit; 2) LOD=D, simple math and memory of cooldown limit; 3) Not SRO Only, numbers above the TS line, RO required knowledge. LIC: Replaced question with a "basis" TS question. Rejected K/A and replaced. LOK=F, LOD=2 98# H 3 X X B S 99 F 3 X X B S 100 F 3 - X X N S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only For SRO portion of Exam: LOK: H(50%min)=15/25(60%); F=10/25(40%); N=11/25(44%), B(19 max)=12/25(48%), M=2/25(8%): U=1/25(4%); LOD=72/25=2.8 Were any Bank questions from previous NRC exams? Identified during validation. Remove handout reference from questions. Make sure question numbers aren't included on the Handout list to applicants during exam. Submit original questions for Modified questions to verify they were modified IAW ES 1021. Questions submitted.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 3 X B S 1 of 2 twice 2 H 3 X M S 3 H 3 X B S 4+ H 3 X N S 5 H 3 X B E NRC: Clarification of last bullet in stem, ie, "-makeup paths are OPERATING". LIC: Revised last bullet to clarify that "-BOTH SX Tower Makeup Pumps are RUNNING." 6 F 3 X N S Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: "Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). "Cues": The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). "T/F": The answer choices are a collection of unrelated true/false statements. "Cred. Dist": The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. "Partial": One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: "Job Link": The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). "Minutia": The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). "#/Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). "Backward": The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). A "+" in the "Q#" column indicates that question was reviewed as part of the representative sample of 30 questions. # 2 RO and 3 SRO Only questions submitted for NRC pre-review Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 7 F 3 X X B E NRC: Implausible distractor (A or B). B. is procedural (1BOA, RCP-2, Step5) sub-action of A. If A not true, then B would not be true. Rewrite/replace A or B. LIC: Changed Distractor B to "Check RCP thermal barrier return valve 1CC9438 - OPEN." This is a step in 1BOA PRI-6 for a CC malfunction with 1CC685 closed. It is an incomplete step in that the step has closed bullets listing 1CC685 AND 1CC9438 which makes it incorrect, but it is a plausible choice; however, it will not provide restoration of CC to the thermal barriers. 8+ H 3 X X M E 1 of 2 twice NRC: Why is "0" current plausible? Better distractor would be 20% indication for A and C (ie, wrong use of math). LIC: Changed A and C to 20% current. 9 F 3 2 X B E NRC: LOD=2, memorization of power supply. LIC: Changed to LOD 2. 10 H F H 3 2 X B E NRC: LOD=2, LOK=F, memorization of trip logic, fundamental knowledge (memory). LIC: Changed to LOD 2. Also changed stem of question to include reading and interpreting a gauge, given bistable trips and bypasses, and candidate must decide on how many further failures are required for 2 different ESF actuations. This is now a high cognitive level question. 11 H 3 X B S 12+ F 2 X B S 13 H 3 X B S 1 of 2 twice 14 H 3 X B S 15 H 3 X M S 16+ H 3 X M S 17 F 3 2 X B E NRC: LOD=2, knowledge of trip setpoint. LIC: Changed to LOD 2. 18 H 3 X B S 1 of 2 twice 19 H 3 X M B E NRC: Q M, changed condition in stem but did not change any of the distractors, or change the answer. LIC: Changed to Bank, not Modified. 20 H 3 X B S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 21+ F 3 X B S NRC: Answer key indicates both A. and B. are correct, but justification indicates A. is incorrect. LIC: Fixed the answer key, B is correct. 22 H F 3 2 X N E 1 of 2 twice NRC: 1) LOK=F, memory knowledge (like Q23); 2) LOD=2, memory knowledge, either know or don't know answer. LIC: Changed LOD to 2 and LOK to F. 23 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 24+ H 2 X B 25 H 3 X B E NRC: 1) Identify 0PRO9J and 1/2PR09J in stem. LIC: Added names of monitors. 26 F 3 X B S 2014 NRC ILE 27 H F H 3 2 X B E 2014 NRC ILE NRC: 1) LOD=2, reading simple straightforward gauge and recalling IA relief valve set-point is low level of difficulty. 2) LOK=F, straight forward reading only gauge provided, knowing trip set-point of IA relief valve is fundamental. 3) Clarify distractor D that lead SAC is running "-LOADED." LIC: 1) Changed to LOD 2. 2) Discussed LOK with Examiners, agreed it is H because of interpretation required of meter. 3) Clarified that lead SAC is running "-LOADED." 28+ F 3 2 X B E NRC: LOD=2; very straight forward question, picking out valve with specified isolation. LIC: Changed to LOD 2. 29 F 3 2 X X N E NRC: 1) LOD=2, straight forward question of power supply to reactor trip and bypass breakers. 2) Why are DC 113 and 114 plausible? LIC: 1) Changed LOD to 2. 2) Added to explanation for plausibility of DC 113 and 114. In short, because of the identical numbering of DC busses and AC instrument busses, and that AC 114 is ESF AS power supply, this is a common point of confusion for the ILT student.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 30 H F H 3 2 X N E NRC: LOK=F, LOD=2; knowledge of Adverse Containment parameters is fundamental. Providing basically two different gauges to read each of the adverse parameters does not make the question challenging or greater than fundamental. LIC: Changed to LOD 2. Discussed LOK with Examiners, decided it IS H because of interpretation required of meter. 31 F 3 X B S 32+ F 3 X N S 33 F 3 2 X B E NRC: LOD=2. Question is borderline discriminating. LIC: Changed to LOD 2. 34 H 3 X N S 35 F 3 2 X B E NRC: LOD=2, knowledge of interlocks. LIC: Changed to LOD 2. 36+ H 3 X B S 37 H 3 2 X B E NRC: LOD=2, simple deduction that set-point not low enough to trip interlock and needs to be set lower. LIC: Changed to LOD 2. 38 H F H 3 2 X N E NRC: 1) Distractor A. action not plausible, ie, no switch or indication exists on MCR panel so why is starting 2A SAC from MCR panel believable? 2) LOK=F, similar to Q21. Fundamental knowledge (memory) of action to take and reading the provided gauge. 3) LOD=2, know no auto start and must start locally, LIC: (1) Changed Answer and 1 distractor to remove implausible distractor and to make the question more operationally correct. (2) Discussed LOK with Examiners, decided it is H because of interpretation required of meter. (3) Changed to LOD 2. 39 F 3 X B S 40+ H 3 X B S Handout: Steam Table 41 H 3 X N S 1 of 2 twice 42 H 3 X M S 1 of 2 twice Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 43 H 3 X X B U NRC: 1) A(1) and B(1) not plausible, ie, clogged SI line affects both charging and SI flow. 2) A(2) and B(2) could be same answer as written, ie, "throttle open" and "throttle fully open" could be same position. LIC: Changed A(1) and B(1) to a condition that ordinarily would lower Pzr level - Tave failing low. Added that Master level control is in AUTO to make those plausible. Changed B(2) to maintain 1CV182 at 52% demand. 44+ H F H 3 2 X B E Handout: Fig's: 1BOA Pri 10-3; 1BOA Pri 10-4 NRC: LOD=2, key words and phrases "after refueling" and "subcooled" words appear in stem and on the respective figures (curves), applicant only has to plot and locate the appropriate flowrate number. LOK=F, Knowing the associated meter is not necessary and only contributes to selecting the correct answer. LIC: 1) Changed LOD to 2. 2) Changed initial conditions in stem to eliminate statement "-after refueling" to "-RV head has NOT been detensioned." This changed the correct answer. 3) Also changed stem statement asking flowrate to maintain "the RCS in a subcooled condition" to "maintain the RCS temperature below its saturation temperature". This means the same as subcooled but is not a direct pointer to the correct curve. 4) Discussed LOK with Examiners, decided it is H because of interpretation required of the curves with the change in the wording. 5) Removed the flow meters stated in the answer and distractors. 45 H 3 X B S 46# H 3 X N S 47 H 3 X N S 1 of 2 twice 48+ H 3 X N S 49 F 3 2 X B E NRC: LOD=2, memorizing a number from CAUTION. LIC: Changed LOD to 2. 50 H 3 X N E NRC: 1) Wording for C. and D. doesn't make sense, ie, add "-to be fed-"? 2) D. identified as CORRECT in Answer Explanation. LIC: 1) Changed wording to make the meaning clearer. 2) Changed D to incorrect in explanation. 51 F 3 2 X N E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 52+ F 3 2 X B E NRC: LOD=2, simple/straight forward memorization of basis/reason. LIC: Changed LOD to 2. 53 F 3 X M S 54 H 3 X N S 55+ H 3 X B S 56# H 3 X B S Handout: BCB Fig 20a 57 F 3 X N S 58 F 3 X B E NRC: Add "Immediately" to A. to clarify when boration is to be stopped. LIC: Added IMMEDIATELY to choice A 59 H 3 X B S 60+ H F H 3 2 3 X B E NRC: LOK=F, LOD=2, simple knowledge (memory) of interlock.. LIC: Significantly revised question, such that candidate must determine what mode of operation steam dumps are in, whether it will have a temperature offset, whether the reference temperature is coming from no load or Tref. Added a likely distractor setpoint for condenser pressure based on a BOA. Changed to LOD 2 as noted (but it is now probably LOD 3. 61 F 3 X B S 62 H F H 3 X N E 1 of 2 twice NRC: 1) LOK=F, basic (fundamental memory) knowledge of valve control location and boration flowrate. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of meter. 63 F 3 X B S 64+ F 3 X X B S 65 F 3 X B S 66 F 2 X N S 1 of 2 twice 67 H F H 3 X N E NRC: LOK=F, basic (fundamental memory) knowledge of normal 100% RH system valve position. LIC: Discussed LOK with Examiners, decided it is H because of interpretation required of status lights.
Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 68+ H 3 B U 1 of 2 twice NRC: Does not meet KA: How does question demonstrate ability to use procedure? LIC: Changed question into a 1 of 2 twice with the second part of the question procedurally directed actions. 69 F 3 X B S 70 F 3 X B S 71 H 3 B S 72+ F 3 1 2 X N B U NRC: LOD=1, this is required everyday knowledge for workers and operators signing into the plant each day. Not discriminating knowledge for licensed operator. LIC: Replaced with G2.4.1: BCA 0.0 immediate actions; LOD=2, LOK=F, Bank question. Updated ES-401-3, the Rejected K/As list, the Outline Change Form, and the Exam. 73 F 3 X N E 1 of 2 twice NRC: As written, there would be nothing wrong with notifying the RP before monitoring a second time. Modify stem to reflect the "-procedure requirement-". LIC: Added "In accordance with RP-AA-350-" 74 F 3 X X B S 75 F 3 X B S R RO Exam: LOK: H(50-60%)=42/75(56.0%); F=33/75(44%) N=20/75(26.7%), B(56 max)=49/75=65.3%, M=6/75(8%): U=3/75(4%); E=28/75(37.3%); LOD=2.72 76 H 3 X X N S 77 H 3 X X N S 78 F 3 X X N S 79 H 3 X X B S 80 F 3 X X N S 81 H 3 X X N S 82 H 3 X X N S 1 of 2 twice 83 F 3 X X M S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 84 H 3 X X B S 85 H 3 X X B S 1 of 2 twice 86 F 3 X X N S 1 of 2 twice 87# H 3 X X B S 88 H 3 X X N S Handout: EALs, EP-AA-1002, Add. 3 89 H 3 X X B S 90# F 3 X X N S 91 H 3 X X B S 92 H 3 2 X X B E Handout: TS 3.4.16 NRC: Is handout of TS provided? If so, why not a direct lookup? LOD=2, read TS, little discrimination. LIC: : Changed to LOD 2. Yes, TS is provided, but the candidate does not just read the TS. They must compare the different actions for the different activity levels, and must get the limit for I-133 from the Surveillance requirement and decide what constitutes the most limiting action. It is actions "below the line" removing it from RO knowledge, but has actions > 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, so it's not required from memory. 93 H 3 X X B S 94 H 3 X X B S 95 F 3 X X N S 96 F 3 2 X X B E NRC: LOD=2, simple memory. LIC: Changed to LOD 2. 97 H F 3 2 X X M U NRC: 1) LOK=F, memory of cooldown limit; 2) LOD=D, simple math and memory of cooldown limit; 3) Not SRO Only, numbers above the TS line, RO required knowledge. LIC: Replaced question with a "basis" TS question. Rejected K/A and replaced. LOK=F, LOD=2 98# H 3 X X B S 99 F 3 X X B S 100 F 3 - X X N S Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only For SRO portion of Exam: LOK: H(50%min)=15/25(60%); F=10/25(40%); N=11/25(44%), B(19 max)=12/25(48%), M=2/25(8%): U=1/25(4%); LOD=72/25=2.8 Were any Bank questions from previous NRC exams? Identified during validation. Remove handout reference from questions. Make sure question numbers aren't included on the Handout list to applicants during exam. Submit original questions for Modified questions to verify they were modified IAW ES 1021. Questions submitted.