ML16307A352
| ML16307A352 | |
| Person / Time | |
|---|---|
| Site: | Braidwood  |
| Issue date: | 09/21/2016 |
| From: | Walton R NRC/RGN-III/DRS/OLB |
| To: | |
| Shared Package | |
| ML16306A152 | List: |
| References | |
| Download: ML16307A352 (7) | |
Text
ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 2 Y N S 2+ H 4 Y N S Answer explanation has "D" as correct answer.
RE: corrected 3 H 3 Y N S Answer explanation has "B" & "D" as correct answer.
RE: Corrected 4 F 3 Y M S 5+ H 2 Y B S Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
The answer choices are a collection of unrelated true/false statements.
The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). The question requires reverse logic o r application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO
-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES
-401 Section D.2.f.
- 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory
? 8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).
A "+" in the "Q#" column indicates that question was reviewed as part of the representative sample of 30 questions.
ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 6 F 2 Y N S 7 H 3 Y B S 2009 NRC Exam 8+ H 2 Y N S 9 H 3 Y N S 10 F 2 Y B S 2009 NRC Exam 11 F 2 Y M S 12+ F 2 Y N S 13+ F 2 Y N S 14 H 3 Y B E S 2014 NRC Exam First bullet references "1P
-455". Should it read "1PT-455"? RE: No, "P" designator is for Pressure control loop, not just the transmitter. Stay as is.
15 F 3 Y B S 2009 NRC Exam 16 H 3 Y N S 17 H 3 Y B S 18+ F 2 Y B S Answer explanation has "B" as correct answer.
RE: Corrected Does 'local' mean a mechanical indicator on the breaker?
RE: Yes. 2011 NRC Exam 19 H 4 Y B S Would operators know function of Test switches during testing?
Is this a surveillance done by OPS?
RE: Yes, operators have been trained on this ST, done frequently, included in OJT/TPC.
20 F 2 Y B S 21+ H 2 Y N S 22 F 2 Y B S 23 F 2 Y B S 2009 NRC Exam 24+ F 2 Y N S 25 H 3 Y N S ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 26+ H 3 N B U K/A references, "major DC Loads" whereas question asks about power on AC busses?
2011 NRC Exam. RE: Replace/Revise question. 27 F 3 Y B S 2011 NRC Exam 28 F 2 Y N S 1/2X2 29 H 2 Y B E S 2014 NRC Exam Why is this a high cog question? Conditions in stem don't change choice of correct answer.
RE: Higher cog question since 1A AFW Pump has self
-driven shaft cooling where as 1B AFWP does not have this design feature.
Stay as is.
30 F 3 N B U K/A requires monitoring automatic operation with air pressure.
Nothing is operated in automatic.
RE: Replace/Revise Q.
2011 NRC Exam 31+ H 3 N B U S K/A mismatch since not in 'A2" format.
RE: Q reconsidered to be acceptable. A2 format exception allowed by NUREG ES-401, D.2.a 32 H 3 Y N S 33 H 3 Y N S 34+ H 2 Y N S 35 H 2 Y N S 36 H 2 Y N S 37 H 3 Y N S 38+ H 2 Y N S 39+ H 2 Y N S 40 H 3 Y B S 2011 NRC Exam 41 H 2 Y B S 2011 NRC Exam 42 F 2 Y N S 43+ F 2 Y M S Answer Explanation box describes 1CV8110 as MOV referenced in PRI 13, but PRI 13 references 1CV8100.
RE Corrected typo.
ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 44 H 3 Y N S 45 F 2 Y B S 46+ F 2 Y B S 2011 NRC Exam 47 F 2 Y N S 48 H 2 Y B S 49 F 2 Y N S 50+ F 3 X ? B U 2011 NRC Exam. Don't need the initial conditions given to answer the question, only need last sentence. Suggest this question be reconstructed for a SBLOCA with an ATWS. RE: Replace/Revise Q.
51 H 2 Y B E 2011 NRC Exam. How is this a High cog question? How is the sequence changed with event type?
RE: LOOP/LOCA vs. LOCA alone changes answer. Keep as Higher Cog.
52 F 3 Y N S 53+ F 2 Y N S 54 H 2 Y B S 1/2X2 55 H 3 Y N S 1/2X2: Third bullet references "1P
-45 7". Should it read "1PT
-45 7"? RE: Sat as is, "P" designator refers to pressure loop.
56 F 2 Y N S 57+ H 2 Y N S 58 F 2 Y B S 59 H 3 Y B S 2011 NRC Exam 60+ F 2 Y B S 2011 NRC Exam 61 H 2 X ? M U S I don't understand the explanation for the correct answer provided in the explanation box. MFW flow reg valve is referenced but there would be no MFW flow during a LOOP. Explanation is about AFW and SGWL. I believe AFW would fill the SGs to the top w/o operator intervention.
RE: Unit does not trip, FW/Cond stays on line via ABT action. Steaming reduces SGWL. Keep Q as is.
ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 62 H 2 Y M S 63+ H 2 Y B S 2009 NRC Exam (Modified question is on 2016 Cert exam).
64 F 2 Y M S 65 H 3 Y M S 66 H 3 Y B S 2009 NRC Exam 67+ F 2 Y N S 68 H 3 Y B S 2009 NRC Exam 69 F 2 Y N S 70+ H 2 Y M S 71 H 2 Y N S 72 H 2 Y B S 2011 NRC Exam 73 F 2 Y N S 74+ F 2 Y N S 75 F 2 Y N S 76 F 2 Y Y B S 20 09 NRC Exam 77 F 2 Y Y N S 78+ H 3 Y Y N S 1/2X2 79 H 3 Y Y B S 2009 NRC Exam 80 H 4 Y Y N S 81 H 2 Y Y N S 1/2X2 82+ H 3 Y Y B S 83 F 2 Y Y B S 2011 NRC Exam.
Wrong supporting reference provided (AFW vs EDG).
84 F 2 Y Y N S ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 85+ F 2 Y Y N S 1/2X2 86 H 3 Y Y N S 87 H 2 Y Y N S 88 H 3 X Y Y N U Stem states that alarm came in and cleared in 5 seconds cueing that an auto closure occurred. First part of question is RO knowledge. Consider changing distractors to TS 3.7.7 is/is not applicable and how RCP CC Barrier is tracked for repair or enter into BOA for RCP malfunction.
RE: Q corrected by removing 2 nd bullet about HIGH alarm clearing in 5 seconds. Deletes cue.
89+ F 3 Y Y N S 90 F 2 Y Y N S 91 H 2 Y Y N S 1/2X2 Does loss of DC bus 113 result in loss of any panel indicators ON Table M1? RE: No, annunciators lost to all M1 components, but not indications.
92 H 2 Y Y B S 2014 NRC Exam Why is this question Hi Cog?
RE: Considered High cog since applicant must have knowledge of limits for different parameters provided in distractors. Stay as is.
93+ F H 3 Y Y B S 2009 NRC Exam Why is this question Low Cog? RE: Changed to High cog since containment pressure introduces adverse containment conditions. Operators must apply this to SGWL to get correct answer.
94 H 2 Y Y B S 2009 NRC Exam Why is this question Hi Cog?
RE: Type and rate of runback determines whether or not chem sample is required. Stay as is.
95 F 2 Y Y M S 96+ H 4 Y Y N S 97 F 3 Y Y M S 98 H 2 Y Y B S 2009 NRC Exam
ES-4 01 Written Examination Review Worksheet Form ES-401-9 Braidwood Initial License Exam (2016) Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
- 4. Job Content Flaws
- 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 99 H 3 Y Y B S 2009 NRC Exam 100+ F 2 Y Y N S Questions from Previous NRC Exam: 28%
New Q's: 36 / 14 RO LOK: F
- 3 3; H - 4 2 1/2x2 questions: 7 Modified Q's: 8 / 2 SRO LOK: F
- 1 1; H - 1 4 # of Unsat questions: 4 or 4% Bank Q's: 31 / 9