ML16307A352
| ML16307A352 | |
| Person / Time | |
|---|---|
| Site: | Braidwood  |
| Issue date: | 09/21/2016 |
| From: | Walton R NRC/RGN-III/DRS/OLB |
| To: | |
| Shared Package | |
| ML16306A152 | List: |
| References | |
| Download: ML16307A352 (7) | |
Text
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 2 Y N S 2+ H 4 Y N S Answer explanation has D as correct answer. RE: corrected 3 H 3 Y N S Answer explanation has B & D as correct answer. RE: Corrected 4 F 3 Y M S 5+ H 2 Y B S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
- The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
- The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
- The answer choices are a collection of unrelated true/false statements.
- The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
- One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
- The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
- The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
- The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
- The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 6 F 2 Y N S 7 H 3 Y B S 2009 NRC Exam 8+ H 2 Y N S 9 H 3 Y N S 10 F 2 Y B S 2009 NRC Exam 11 F 2 Y M S 12+ F 2 Y N S 13+ F 2 Y N S 14 H 3 Y B E 2014 NRC Exam First bullet references 1P-455. Should it read 1PT-455? RE: No, P designator is for Pressure control loop, not S
just the transmitter. Stay as is.
15 F 3 Y B S 2009 NRC Exam 16 H 3 Y N S 17 H 3 Y B S 18+ F 2 Y B S Answer explanation has B as correct answer. RE: Corrected Does local mean a mechanical indicator on the breaker? RE: Yes.
2011 NRC Exam 19 H 4 Y B S Would operators know function of Test switches during testing? Is this a surveillance done by OPS? RE: Yes, operators have been trained on this ST, done frequently, included in OJT/TPC.
20 F 2 Y B S 21+ H 2 Y N S 22 F 2 Y B S 23 F 2 Y B S 2009 NRC Exam 24+ F 2 Y N S 25 H 3 Y N S
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 26+ H 3 N B U K/A references, major DC Loads whereas question asks about power on AC busses? 2011 NRC Exam. RE: Replace/Revise question.
27 F 3 Y B S 2011 NRC Exam 28 F 2 Y N S 1/2X2 29 H 2 Y B E 2014 NRC Exam Why is this a high cog question? Conditions in stem dont change choice of correct answer. RE: Higher cog S
question since 1A AFW Pump has self-driven shaft cooling where as 1B AFWP does not have this design feature. Stay as is.
30 F 3 N B U K/A requires monitoring automatic operation with air pressure.
Nothing is operated in automatic. RE: Replace/Revise Q.
2011 NRC Exam 31+ H 3 N B U K/A mismatch since not in A2 format. RE: Q reconsidered to be acceptable. A2 format exception allowed by NUREG ES-401, D.2.a S
32 H 3 Y N S 33 H 3 Y N S 34+ H 2 Y N S 35 H 2 Y N S 36 H 2 Y N S 37 H 3 Y N S 38+ H 2 Y N S 39+ H 2 Y N S 40 H 3 Y B S 2011 NRC Exam 41 H 2 Y B S 2011 NRC Exam 42 F 2 Y N S 43+ F 2 Y M S Answer Explanation box describes 1CV8110 as MOV referenced in PRI 13, but PRI 13 references 1CV8100. RE Corrected typo.
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 44 H 3 Y N S 45 F 2 Y B S 46+ F 2 Y B S 2011 NRC Exam 47 F 2 Y N S 48 H 2 Y B S 49 F 2 Y N S 50+ F 3 X ? B U 2011 NRC Exam. Dont need the initial conditions given to answer the question, only need last sentence. Suggest this question be reconstructed for a SBLOCA with an ATWS. RE: Replace/Revise Q.
51 H 2 Y B E 2011 NRC Exam. How is this a High cog question? How is the sequence changed with event type? RE: LOOP/LOCA vs. LOCA alone changes answer. Keep as Higher Cog.
52 F 3 Y N S 53+ F 2 Y N S 54 H 2 Y B S 1/2X2 55 H 3 Y N S 1/2X2: Third bullet references 1P-457. Should it read 1PT-457?
RE: Sat as is, P designator refers to pressure loop.
56 F 2 Y N S 57+ H 2 Y N S 58 F 2 Y B S 59 H 3 Y B S 2011 NRC Exam 60+ F 2 Y B S 2011 NRC Exam 61 H 2 X ? M U I dont understand the explanation for the correct answer provided in the explanation box. MFW flow reg valve is referenced but there S
would be no MFW flow during a LOOP. Explanation is about AFW and SGWL. I believe AFW would fill the SGs to the top w/o operator intervention. RE: Unit does not trip, FW/Cond stays on line via ABT action. Steaming reduces SGWL. Keep Q as is.
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 62 H 2 Y M S 63+ H 2 Y B S 2009 NRC Exam (Modified question is on 2016 Cert exam).
64 F 2 Y M S 65 H 3 Y M S 66 H 3 Y B S 2009 NRC Exam 67+ F 2 Y N S 68 H 3 Y B S 2009 NRC Exam 69 F 2 Y N S 70+ H 2 Y M S 71 H 2 Y N S 72 H 2 Y B S 2011 NRC Exam 73 F 2 Y N S 74+ F 2 Y N S 75 F 2 Y N S 76 F 2 Y Y B S 2009 NRC Exam 77 F 2 Y Y N S 78+ H 3 Y Y N S 1/2X2 79 H 3 Y Y B S 2009 NRC Exam 80 H 4 Y Y N S 81 H 2 Y Y N S 1/2X2 82+ H 3 Y Y B S 83 F 2 Y Y B S 2011 NRC Exam.
Wrong supporting reference provided (AFW vs EDG).
84 F 2 Y Y N S
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 85+ F 2 Y Y N S 1/2X2 86 H 3 Y Y N S 87 H 2 Y Y N S 88 H 3 X Y Y N U Stem states that alarm came in and cleared in 5 seconds cueing that an auto closure occurred. First part of question is RO knowledge.
Consider changing distractors to TS 3.7.7 is/is not applicable and how RCP CC Barrier is tracked for repair or enter into BOA for RCP malfunction. RE: Q corrected by removing 2nd bullet about HIGH alarm clearing in 5 seconds. Deletes cue.
89+ F 3 Y Y N S 90 F 2 Y Y N S 91 H 2 Y Y N S 1/2X2 Does loss of DC bus 113 result in loss of any panel indicators ON Table M1? RE: No, annunciators lost to all M1 components, but not indications.
92 H 2 Y Y B S 2014 NRC Exam Why is this question Hi Cog? RE: Considered High cog since applicant must have knowledge of limits for different parameters provided in distractors. Stay as is.
93+ F 3 Y Y B S 2009 NRC Exam H Why is this question Low Cog? RE: Changed to High cog since containment pressure introduces adverse containment conditions.
Operators must apply this to SGWL to get correct answer.
94 H 2 Y Y B S 2009 NRC Exam Why is this question Hi Cog? RE: Type and rate of runback determines whether or not chem sample is required. Stay as is.
95 F 2 Y Y M S 96+ H 4 Y Y N S 97 F 3 Y Y M S 98 H 2 Y Y B S 2009 NRC Exam
ES-401 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016)
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 99 H 3 Y Y B S 2009 NRC Exam 100+ F 2 Y Y N S Questions from Previous NRC Exam: 28% New Qs: 36 / 14 RO LOK: F - 33; H - 42 1/2x2 questions: 7 Modified Qs: 8 / 2 SRO LOK: F - 11; H - 14
- of Unsat questions: 4 or 4% Bank Qs: 31 / 9