ML15128A292: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
 
(5 intermediate revisions by the same user not shown)
Line 3: Line 3:
| issue date = 08/22/2014
| issue date = 08/22/2014
| title = 2014 Prairie Island Nuclear Generating Plant Initial License Examination Form ES-401-9
| title = 2014 Prairie Island Nuclear Generating Plant Initial License Examination Form ES-401-9
| author name = Bielby M E
| author name = Bielby M
| author affiliation = NRC/RGN-III/DRS/OLB
| author affiliation = NRC/RGN-III/DRS/OLB
| addressee name =  
| addressee name =  
Line 14: Line 14:
| page count = 15
| page count = 15
}}
}}
=Text=
{{#Wiki_filter:ES-401                                          Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 1+  F    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
2  H    2                                                          Y        N  S 3  F    2                                                          Y        B  S  NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
4+  H    2                                                          Y        B  S  NRC: Why is the CS system drawing provided? Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
NRC: Why is this bank question considered modified? (ES-401D.2.f)
M      Was this bank question ever used on an NRC exam?
5  F    2                                                          Y            E B      Lic: Not used on the last 2 NRC exams. Not Significantly modified-counted as bank question.
NRC: 1 out of 2 taken twice. Why are Distractors D & C plausible 6  H    3                                                          Y        N  S  for the same reasons?
Lic: Changed the justification for D distractor.
NRC: Is F3-17.2 Table 2 provided as a handout?
7  H    3                                                          Y        N  S Lic: No.
NRC: Are the ROs required to know what defines a CC Train as OPERABLE? NOTE: TSs do allow continued operation with only one pump per unit.
Lic: Changed the Stem question to request the number of trains 8+  F    2                                X                        Y        N  U required to be OPERABLE vs number of pumps . . . to reflect RO knowledge requirements. Included adding the adjective MINIMUM to the Stem question.
Question was U and is now considered S.
9  H    2                                                          Y        N  S NRC: By providing the EOP Table in the stem of the question, this becomes a very simple lookup. Was this bank question ever used 1                                                                      U  on an NRC exam?
10  H                                                                Y        B 2                                                                      S  Lic: Must apply the limits for adverse containment to correctly enter the required Table. Question as written is S. Not used on the last 2 NRC exams.
1 of 15
ES-401                                                          Written Examination Review Worksheet                                                                    Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
: 1.      Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 2.      Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
: 3.      Check the appropriate box if a psychometric flaw is identified:
* The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
* The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
* The answer choices are a collection of unrelated true/false statements.
* The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
* One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 4.      Check the appropriate box if a job content error is identified:
* The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
* The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
* The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
* The question requires reverse logic or application compared to the job requirements.
: 5.      Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
: 6.      Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
: 7.      Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8.      At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).
A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.
2 of 15
ES-401                                            Written Examination Review Worksheet                                                        Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 11  H    2                                                          N        M  U  NRC: How does this apply to the Loss of Main Feedwater K/A?
(Related reason for manual control of AFW flow control valves?)
Y        B  E  Why is this bank question considered modified? (ES-401D.2.f) Was this bank question ever used on an NRC exam? Was the bank question also LOK-H?
Lic: Question does match the K/A since the AFW valve must be manually throttled. Not Significantly modified-counted as bank question. Bank question was also H.
12+  H    2                                                          Y        N  E  NRC: Why is this not simply memory of which INVs power which Channel? Was this bank question ever used on an NRC exam?
S Lic: Must apply knowledge of which inverter power supply powers which channel and use the photo to determine which channels have been de-energized. Question is considered S. Not used on the last 2 NRC exams.
13  F    2                                                          Y        N  S  NRC: Distractor D is not plausible for the rational given-doesnt match indications given in stem.
Lic: Changed the justification for D distractor.
14  H    2                                                          N        N  U  NRC: 1 out of 2 taken twice. Is this simply recalling from memory where the instrument panel is located and then reading the figure in Y            S  the stem? The K/A requires locate and operate components.
Lic: Per NUREG-1021, APP B C.1.b, question focuses on the dominant portion of the K/A, and requires applying the knowledge of the power supply vs simply operating a pushbutton. Question matches the K/A and requires analysis to answer correctly; question is considered S.
15  H    2                                                          Y        N  E  NRC: 1 out of 2 taken twice. Is this simply recalling from memory that loss of the Unit 1 train A DC will result in an automatic Reactor S  Trip, after noting the loss from the figure in the stem?
Lic: Applicant must analyze the diagram in the stem to determine that the DC train is de-energized. Question is considered S.
16+  F    2                                                          Y        B  S  NRC: Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
17  F    2                                                          Y        N  S 18  H    3                                                          Y        N  S  NRC: 1 out of 2 taken twice.
3 of 15
ES-401                                            Written Examination Review Worksheet                                                        Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 19  H    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
20+  H    2                                                          Y        B  S  NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
21  F    2                      X                                  Y        N  E  NRC: Distractor D is not plausible for the rational given.
Lic: Changed distractor D to a more plausible value and then re-ordered the distractors logically.
22  F    2                                                          N        N  U  NRC: 1 out of 2 taken twice. The K/A requires knowledge of the concepts of fire-fighting as they apply to a plant fire onsite, not fire-Y            S  fighting equipment operation once the fire is out.
Lic: Question matches the K/A since it addresses the recovery of plant fire-fighting equipment to support operational requirements.
Question is considered S.
23  H    2                      X                                  Y        N  U  NRC: 1 out of 2 taken twice. Distractors A & D are not plausible stand-alone combinations of the two parts of the correct answer.
S Lic: Distractors A & D are plausible since they require analysis of specific PWR plant operational knowledge to prove them incorrect.
Question is considered S.
24+  F    2                                                          N        N  U  NRC: The K/A requires knowledge of the reasons for manipulation of controls to obtain results during abnormal and emergency situations Y            S  affecting containment flooding. The question doesnt address why fire protection is not a source of flooding.
Lic: Knowledge of the K/A is demonstrated by understanding why fire protection cannot be a source of flooding. Question is considered S.
25  F    2                                                          Y        N  S  NRC: 1 out of 2 taken twice. Need an explanation for the difference between Actuation & Isolation when ventilation line-ups and fans are shifting in response to Rad Alarms. This will address the potential of two correct answers.
Lic: Actuation implies Start and Isolation implies Stop, which verifies there is only one correct answer.
26+  F    2                                                          Y        B  S  NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
4 of 15
ES-401                                            Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 27  H    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
28+  F    2                                                          Y        M  E  NRC: Why is this bank question considered modified? (ES-401D.2.f)
Was this bank question ever used on an NRC exam?
B Lic: Not Significantly modified-counted as bank question. Not used on the last 2 NRC exams.
29  H    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
30  F    2                                                          Y        N  S 31  H    2                                                          Y        N  S  NRC: 1 out of 2 taken twice. Is CCW the same as SWS for K/A?
Lic: No, but CCW is the cooling system that serves the same function for RHRS as SWS.
32+  H    2                                                          Y        M  S 33  F    2                                                          Y        M  S 34  H    2                                                          Y        B  S  NRC: Was this bank question ever used on an NRC exam?
Lic: Yes. PI 2010 NRC Exam.
35  H    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
36+  F    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
37  H    3                                                          Y        N  S  NRC: 1 out of 2 taken twice.
38  H    3                                                          Y        B  S  NRC: Is Distractor D plausible for the rational given of no entropy gain? Was this bank question ever used on an NRC exam?
Lic: Plausible based upon misuse of the Mollier Diagram. Not used on the last 2 NRC exams.
39  H    2                      X                                          B  U  NRC: Distractors A & D are not plausible based upon the stem conditions and no statement given to assume the B logic relay does S  not actuate as expected. Was this bank question ever used on an NRC exam?
Lic: Based upon the specific logic diagrams for these relay paths, distractors A & D are plausible. Not used on the last 2 NRC exams. Question is considered S.
5 of 15
ES-401                                            Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 40+  H    2                                                          Y        N  S  NRC: 1 out of 2 taken twice.
41  H    2                      X                                  Y        M  E  NRC: Question is from the PI 2012 NRC Exam. How has this question been modified? Distractor B doesnt appear to be plausible for the rational given.
Lic: Distractor B changed to a plausible distractor. The question changed initial conditions in the stem such that a different answer is correct. Question is considered M.
42  F    2                                                          Y        B  S  NRC: Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
43  H    2    X                                                    N        B  U  NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam? How does this question address the K/A to monitor Y            S  automatic operation of pump starts and MOV positioning? What causes the discharge valves to go back closed and then re-open? Is the correct answer really C?
Lic: Knowledge of the K/A is demonstrated by understanding why the containment spray pump and outlet MOV respond as they do. There is only one correct answer, as stated in the key. Question is considered S, and was not used on the last 2 NRC exams..
44+  H    2                                                          Y        B  S  NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
45  H    3                                                          Y        N  S  NRC: 1 out of 2 taken twice.
46  F    2                                                          Y        B  S  NRC: Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
47  H    3                                                          Y        B  S  NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?
Lic: Not used on the last 2 NRC exams.
48+  F    2                                                          Y        N  S 6 of 15
ES-401                                          Written Examination Review Worksheet                                                        Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 49  H    2                                                          Y        N  E  NRC: F not H. 1 out of 2 taken twice. This is memorization of actions initiated by an S signal, which is a given stem condition; F
higher level cognitive skills are not used to answer the question.
Lic: Agreed that this is a memory/fundamental question.
50  H    2                                                          Y        N  E  NRC: F not H. 1 out of 2 taken twice. This is memorization; higher level cognitive skills are not used to answer the question. Applicant F
either knows, or doesnt know, answer from memory.
Lic: Agreed that this is a memory/fundamental question.
51  H    2                      X                                  Y        N  U  NRC: 1) Non-plausible distractors. Explain how the distractors A., B.
and D. are plausible. Difficult to read meter, may have to expand the S
picture to clearly show the scale and needle deflection position. 2)
Very minor cognitive thought process for how to read scale, getting correct scale, and doing simple math.
Lic: Distractors B & D are plausible if applicant uses incorrect scale direct readings. Photo is adequate to identify values required.
Although the LOD is not high, the applicant is required to perform several functions correctly to arrive at the correct answer, therefore this question is considered an H and is considered S.
52  H    2                                                          Y        N  S 53  F    2                                                          Y        M  E  NRC: Not a modified question. Same question and same answer as previous question. Its just that the new question has part of answer B
in stem where old one had complete correct answer in the answer choice section.
Lic: Not Significantly modified-counted as bank question.
54  H    2                                                          Y        B  S  NRC: This is a GFE type question.
Lic: Applicant requires system knowledge to discern the correct answer.
7 of 15
ES-401                                            Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 55  H    2                            X                            Y        B  U  NRC: 1 out of 2 taken twice. 1) Is this memorization of the Containment Vacuum Breaker setpoint and simple recognition that S
vacuum breakers have exceeded the setpoint and opened? 2) Two Correct Answers (A. and C.). Failed to Close and Open are same concept. Also, Failed to Open and Closed could be considered same concept. 3) What does indication look like if only one vac break opens and other remains closed?
Lic: Added an initial condition to the stem to clarify that both vacuum breakers initially operated as expected. Analysis of the stem conditions is required to discern the correct answer. Question is considered S.
56  H    3                                                          Y        N  S 57  H    2                                                          Y        N  S 58  H    2                      X                                  Y        N  U  NRC: Two Implausible Distractors. 1) A. and D. not plausible numbers. Dont understand justification for A. and D. 2) Just to S
clarify, there is no reference handout to applicant for this question?
Lic: No reference handout is provided to the applicants. Distractor A & D are plausible due to weaknesses in knowledge of the graph bases for axis intercepts and line slope rationale. Question is considered S.
59  H    3                      X                                  Y        N  E  1 out of 2 taken twice.
F                                                                                NRC: F not H. This is recall of Containment Pressure setpoint/TS which is a well-known number. Also recall of the Post LOCA vent system to Shield Bldg. 2) Distractor D. not plausible. Why is 1.3 psig significant?
Lic: Distractor D is plausible because control stations for venting the Units are on the opposite side of the Aux Bldg. Agreed that this is a fundamental/memory question.
60+  F    2                                                          Y        N  S 61  H    2                      X                                  Y        N  E  NRC: F not H. 1) Fundamental memory of what actuates SI. 2) D.
not plausible. Psg greater than PRCS?
F Lic: Changed distractor D the MSL high rad to be plausible. Agreed that this is a fundamental/memory question.
62  F    2                                                          Y        N  S  1 out of 2 taken twice.
8 of 15
ES-401                                            Written Examination Review Worksheet                                                        Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 63  H    3                                                          Y        B  U  NRC: 1) A. and D. not plausible. A., no reading given for or asked for at L/2. D., not believable to use a cube function. 2) Better question S
to require it a point source calculation (L/2 distance), ie, determine line source DR at L/2, then perform point source calculation to determine DR at greater than L/2.
Lic: Distractor A plausible if applicant misapplies line source rule, and D is plausible if plane source rule is misapplied. Question is considered S.
64+  F    2                                                          Y        B  E  NRC: 1) Is this using a higher cognitive skill, or is this memory of how to align cooling water pump as safeguards? 2) Show electrical H
diagram to verify alignment and believability of Bus 15 and 16.
Lic: Analysis of multiple possible power supply lineups. Question is considered an H.
65  H    2                                                          Y        B  E S  NRC: 1 out of 2 taken twice. 1) What higher cognitive skill is used?
Is this memory of minimum fire pressure and action to take? 2) Is the screen wash pump normally running with the screen wash to fire header valve closed as indicated? Would stopping the screen wash pump be an incorrect action? May need more explanation in the question for the screen wash situation. What is the relationship between the fire pressure and screen wash, ie, washing the fire water screen?
Lic: Analysis of the conditions given in the stem (fire is occurring) to correctly discern expected system response. Question is considered S.
66  F    2                      X                                  Y        N  E  NRC: 1) One implausible distractor. C. and F. are basically the same area, so one of the choices is implausible. 2) Include Figure 2 in answer justification (6.1.4). If no Lead ROs, where does the procedure allow both remaining ROs to leave their panels at the same time, ie, out of sight of their indications?
Lic: Changed distractor A to make it plausible.
9 of 15
ES-401                                            Written Examination Review Worksheet                                                        Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                        8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                  Explanation Focus            Dist. Link          units ward K/A Only 67  H    2          X                                              Y      B  U  REF provided.
S  NRC: 1 out of 2 taken twice. 1) First part of question requirement identified in Table 1, i.e., verifies Bank position greater than minimum rod height per Figure C1-8. 2) Second part of question is TS that identifies the requirement and narrows question correct answer to a 50-50 chance of getting it correct.
Lic: The applicant is required to know how to verify Bank position and, using data provided, analyze what conditions are met. Question is considered S.
68+  F    2                      X                                  Y      M  E  NRC: 1 out of 2 taken twice. 1) B. not plausible. Two audible SRIs and one for indication. 2) Not Modified. Basically, same question B
with same answer. Different descriptive words used, but still asking the TS requirement.
Lic: Distractor B is plausible since there are 2 audible SRI that are operable, but only 1 is selected at a time. Not Significantly modified-counted as bank question.
69  F    2                      X                                  Y      N  E  NRC: 1) B. not plausible. When is Self-Check Only a requirement?
: 2) H not F. There would be other times when IV or PC would be H
correct, therefore, have to analyze the given situation.
Lic: Per operations procedure, sometimes are specified when Self-Check ONLY is required, therefore B is plausible. Agree question is considered an H.
70  H    2                                                          Y      B  E  NRC: 1) F not H. As written, it appears the applicant has to know the limits from memory to determine if they are exceeded.
F
: 2) How does applicant determine if the given parameters exceed the PTLR or COLR limit for B., C. and D.? Where is the SL specified in the reference?
Lic: Agreed that this is a fundamental/memory question.
71  F    2                                                          Y      B  S 72+  F    2                                                          Y      B  S 73  F    2                                                          Y      N  S 74  F    3                                                          Y      N  S 10 of 15
ES-401                                                  Written Examination Review Worksheet                                                              Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.        3. Psychometric Flaws        4. Job Content Flaws    5. Other    6. 7.                                      8.
Q#  LOK    LOD (F/H)  (1-5)  Stem Cues T/F Cred. Partial Job-      Minutia  #/ Back- Q= SRO B/M/N U/E/S                                      Explanation Focus              Dist.        Link          units ward K/A Only 75+  F      3                                                              N            N      U  NRC: Does not address KA. KA is knowledge of the bases . . .
question not asking bases, i.e., not asking why?
Y                  S Lic: Knowledge of the K/A is demonstrated by understanding which procedure takes priority based upon conditions in the stem. Question is considered S.
For RO portion of Exam: New = 44Q, Modified = 3Q, Bank = 28Q (37.3%): LOK: 34/75 = 45.3%, H= 54.6%. Initial UNSAT: 1/75 = 1.3%, Enhanced: 15/75 = 20%.
76+  H      3                                                                Y    S    N      S 77#  H      2                                    X                          Y    X    N      U  NRC: 1 out of 2 taken twice. NOT SRO only. RO should be able to answer this based upon integrated plant knowledge and knowledge of the procedures. ROs expected to know how AOPs used in conjunction with EOPs, and to know big picture actions like tripping RCPs and reactor.
Lic: Wrote new question. Re-formatted each choice to say enter procedure and perform action. Question is considered S.
78#  H      3                        X                                    Y      X    N      U  NRC: Implausible distractor and Not SRO Only. 1) Distractor D not plausible. Throttling open a failed open flow control valve. If the flow control valve failed closed, no-flow indications (high discharge pressure with zero gpm flow) would result. Additionally, the stem conditions do not indicate a loss pump or valve malfunction to suggest using AOP 1. 2) Not SRO Only if ROs are trained to trip a pump that has indications of cavitation, then the SRO is not actually directing use of a specific procedure.
Lic: All choices were re-formated for SRO level. Distractor D was changed to manually throttle close the 11/12 RHR HX Bypass Flow Valve instead of throttle open the 12 RHR HX Outlet Flow Control Valve. Question is considered S.
79#  H      3                        X                                    Y      S    M      U  NRC: Implausible distractors. For A distractor what makes a value of 63% (given in stem) plausible vs 33% for switchover criteria?
Also, are the ROs required to recognize the 33% level switchover criteria? Same rational is applicable to D distractor with the 105,000 gal level.
Lic: 1) RWST level too high in stem, 43% would be ok (other criteria is 8%). Adjusted RWST level from 63% to 43% to address distractor A.. 2) Deleted CST level, replace with E.1 procedure. 3) Changed Distractor D. to 1E-1 instead of 1C28.1 AOP2. Question is considered S.
11 of 15
ES-401                                            Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 80#  H    3                                                          Y    S  N  E  REF provided.
NRC: Reword stem to make D. the correct answer.
Lic: Stem revised to have Bus 16 green load rejection lights NOT lit.
This makes choice D the correct answer. Added bullet to stem to state offsite power is NOT available.
81+  H    3                                                          Y    S  B  S 82  H    3                                                          Y    S  B  S  2012 NRC Exam 83+  H    3                                                          Y    S  N  S  REF provided 84+  H    3                                                          Y    S  B  S  REF provided.
NRC: Modify question Explanation to identify ODCM is part of Administrative TS Section 5.0. There is no above line, below line or Safety Limits. All of the Administrative TS items are SRO Only.
Lic: Modified justification for SRO only.
85#  H    1                      X                                  Y    S  B  U  NRC: Low LOD. Two implausible distractors. Based upon the information provided in the stem, why would it be plausible for the applicant to believe that SG pressures are lowering uncontrollably?
Same for B and C.
Lic: Fix the explanations. Changed two distractors: 1E-2 to 1ES-0.1.
and 1E-3 to 1ECA-1.2. Re-ordered distractors using waterfall method. Question is considered S.
86+  H    1                                                          Y    S  N  U  REF provided.
2                                                                      S  NRC: Low LOD. Question is a reading exercise for SRO that requires abiding by a procedure Caution, and comparing given data with that in the procedure. Does the use of a reference makes this question a direct lookup?
Lic: Several decisions/analysis involving recalled knowledge is required to discern the correct answer. Question is considered S.
12 of 15
ES-401                                            Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 87#  H    3                            X                            Y    S  M  E  NRC: 1) Distractor A is currently a subset of distractor D. 2) How is this a modified question? Only changed one distractor and deleted B
stem data that said D1 and D2 EDGs running unloaded with does not change the correct answer.
Lic: 1) added ONLY to distractor A, and reworded distractors to state Enter vice per for answers; put procedures first then actions;.
All choices re-formated. 2) Question is considered B.
88+  F    2                                                          Y    S  N  S  NRC: 1 out of 2 taken twice.
89+  H    3                                                          Y    S  N  S  REF provided.
90#  H    3                      X                                  Y    S  N  E  NRC: Distractor D. not plausible. Also, distractor C is potentially cueing the applicant to the correct answer since it is the only Non-Refueling option to choose.
Lic: Added enter vice per; R-31 piece of distractor D was removed; two distractors were modified; all choices were re-formatted and re-ordered.
91+  H    2                                                          N    S  N  U  REF provided.
Y          S  NRC: Does the use of a reference make this question a direct lookup? Does not meet KA for ability to predict impacts.
Lic: The proper use of TSs does address the impact on the system.
Several iterations of the TS condition 3.3.1 Table is required to discern the correct answer. Question is considered S.
92  H    3                                                          Y    S  M  E  REF provided.
B      NRC: 1 out of 2 taken twice. The applicant handout should not include the TS bases. This question is not modified.
Lic: The reference provided does not include the bases. Not Significantly modified-counted as bank question.
93+  H    3                                                          N    S  N  U  REF provided.
NRC: Does the use of a reference make this question a direct lookup? Does not meet KA for ability to predict impacts.
Lic: A comment was added to the stem to address performance of required actions in order to meet the K/A. Question was U and is now considered S.
13 of 15
ES-401                                            Written Examination Review Worksheet                                                      Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.      3. Psychometric Flaws      4. Job Content Flaws    5. Other 6. 7.                                      8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job-  Minutia  #/ Back- Q= SRO B/M/N U/E/S                                Explanation Focus            Dist. Link          units ward K/A Only 94  H    2                                                          Y    S  M  U  REF provided.
S  NRC: Does the use of a reference and simple math make this question a direct lookup?
Lic: Knowledge of the bases is required to correctly apply the corrections and discern the final answer. Question is considered S.
95+  F    2                                                          Y    S  N  S  1 out of 2 taken twice.
96#  H    2    X                                                    Y    S  N  E  REF provided.
NRC: The question should address mitigation of the Orange path to meet the intent of the K/A; how is this accomplished? Are the ROs required to be knowledgeable of potential impacts of orange paths?
(Potentially not SRO Only as written). Reword the stem question to address the EAL status of the fission product barriers. Also, suggest deleting the RCS status column as this appears to decrease LOD, and reword distractor B. or C. Containment section to say POTENTIAL LOSS.
Lic: The stem was modified to include a statement directing analysis using the SRO level reference.
97#  H    2                      X                                  Y    S  B  E  REF provided.
NRC: Distractor A not plausible. Change to NOT approve.
Lic: Revised distractor A to be plausible, and reordered choices.
98+  F    2                                                          Y    S  B  S  2010 NRC Exam 99#  H    3    X                                                    Y        B  U  REF provided.
NRC: Step 17b RNO is not performed until Narrow Range is greater than 50%, and the stem states level is 12%. Why would the applicant try to perform the RNO action early? Seems like the correct answer is A. Then when level is >50% narrow range and uncontrollable he will perform the 17b RNO. What are we missing?
(UNKNOWN)
Lic: Modified 12 SG indications to ensure that the intended correct answer is the only correct answer. Question is considered S.
100+  F    2                                                          Y        M  E  NRC: This question is not modified.
B      Lic: Not Significantly modified-counted as bank question.
14 of 15
ES-401                                                  Written Examination Review Worksheet                                                    Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam
: 1. 2.        3. Psychometric Flaws          4. Job Content Flaws    5. Other  6. 7.                                8.
Q#  LOK  LOD (F/H) (1-5)  Stem Cues T/F Cred. Partial Job-        Minutia  #/ Back- Q= SRO B/M/N U/E/S                              Explanation Focus              Dist.        Link            units ward K/A Only For SRO portion of Exam: New = 13Q, Modified = 2Q, Bank = 10Q (40%): LOK: 4/25 = 16%, H= 84%. Initial UNSAT: 1/25 = 4%, Enhanced: 2/25 = 8%.
    # SRO Only question submitted for NRC pre-review. Total of 10.                                              6/25 = 24%,          7/25 = 28%
15 of 15}}

Latest revision as of 12:40, 31 October 2019

2014 Prairie Island Nuclear Generating Plant Initial License Examination Form ES-401-9
ML15128A292
Person / Time
Site: Prairie Island  Xcel Energy icon.png
Issue date: 08/22/2014
From: Bielby M
NRC/RGN-III/DRS/OLB
To:
Northern States Power Co
Shared Package
ML13093A444 List:
References
Download: ML15128A292 (15)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1+ F 2 Y N S NRC: 1 out of 2 taken twice.

2 H 2 Y N S 3 F 2 Y B S NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

4+ H 2 Y B S NRC: Why is the CS system drawing provided? Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

NRC: Why is this bank question considered modified? (ES-401D.2.f)

M Was this bank question ever used on an NRC exam?

5 F 2 Y E B Lic: Not used on the last 2 NRC exams. Not Significantly modified-counted as bank question.

NRC: 1 out of 2 taken twice. Why are Distractors D & C plausible 6 H 3 Y N S for the same reasons?

Lic: Changed the justification for D distractor.

NRC: Is F3-17.2 Table 2 provided as a handout?

7 H 3 Y N S Lic: No.

NRC: Are the ROs required to know what defines a CC Train as OPERABLE? NOTE: TSs do allow continued operation with only one pump per unit.

Lic: Changed the Stem question to request the number of trains 8+ F 2 X Y N U required to be OPERABLE vs number of pumps . . . to reflect RO knowledge requirements. Included adding the adjective MINIMUM to the Stem question.

Question was U and is now considered S.

9 H 2 Y N S NRC: By providing the EOP Table in the stem of the question, this becomes a very simple lookup. Was this bank question ever used 1 U on an NRC exam?

10 H Y B 2 S Lic: Must apply the limits for adverse containment to correctly enter the required Table. Question as written is S. Not used on the last 2 NRC exams.

1 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:
  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
  • The answer choices are a collection of unrelated true/false statements.
  • The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:
  • The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  • The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
  • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewers judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

A + in the Q# column indicates that question was reviewed as part of the representative sample of 30 questions.

2 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 11 H 2 N M U NRC: How does this apply to the Loss of Main Feedwater K/A?

(Related reason for manual control of AFW flow control valves?)

Y B E Why is this bank question considered modified? (ES-401D.2.f) Was this bank question ever used on an NRC exam? Was the bank question also LOK-H?

Lic: Question does match the K/A since the AFW valve must be manually throttled. Not Significantly modified-counted as bank question. Bank question was also H.

12+ H 2 Y N E NRC: Why is this not simply memory of which INVs power which Channel? Was this bank question ever used on an NRC exam?

S Lic: Must apply knowledge of which inverter power supply powers which channel and use the photo to determine which channels have been de-energized. Question is considered S. Not used on the last 2 NRC exams.

13 F 2 Y N S NRC: Distractor D is not plausible for the rational given-doesnt match indications given in stem.

Lic: Changed the justification for D distractor.

14 H 2 N N U NRC: 1 out of 2 taken twice. Is this simply recalling from memory where the instrument panel is located and then reading the figure in Y S the stem? The K/A requires locate and operate components.

Lic: Per NUREG-1021, APP B C.1.b, question focuses on the dominant portion of the K/A, and requires applying the knowledge of the power supply vs simply operating a pushbutton. Question matches the K/A and requires analysis to answer correctly; question is considered S.

15 H 2 Y N E NRC: 1 out of 2 taken twice. Is this simply recalling from memory that loss of the Unit 1 train A DC will result in an automatic Reactor S Trip, after noting the loss from the figure in the stem?

Lic: Applicant must analyze the diagram in the stem to determine that the DC train is de-energized. Question is considered S.

16+ F 2 Y B S NRC: Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

17 F 2 Y N S 18 H 3 Y N S NRC: 1 out of 2 taken twice.

3 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 19 H 2 Y N S NRC: 1 out of 2 taken twice.

20+ H 2 Y B S NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

21 F 2 X Y N E NRC: Distractor D is not plausible for the rational given.

Lic: Changed distractor D to a more plausible value and then re-ordered the distractors logically.

22 F 2 N N U NRC: 1 out of 2 taken twice. The K/A requires knowledge of the concepts of fire-fighting as they apply to a plant fire onsite, not fire-Y S fighting equipment operation once the fire is out.

Lic: Question matches the K/A since it addresses the recovery of plant fire-fighting equipment to support operational requirements.

Question is considered S.

23 H 2 X Y N U NRC: 1 out of 2 taken twice. Distractors A & D are not plausible stand-alone combinations of the two parts of the correct answer.

S Lic: Distractors A & D are plausible since they require analysis of specific PWR plant operational knowledge to prove them incorrect.

Question is considered S.

24+ F 2 N N U NRC: The K/A requires knowledge of the reasons for manipulation of controls to obtain results during abnormal and emergency situations Y S affecting containment flooding. The question doesnt address why fire protection is not a source of flooding.

Lic: Knowledge of the K/A is demonstrated by understanding why fire protection cannot be a source of flooding. Question is considered S.

25 F 2 Y N S NRC: 1 out of 2 taken twice. Need an explanation for the difference between Actuation & Isolation when ventilation line-ups and fans are shifting in response to Rad Alarms. This will address the potential of two correct answers.

Lic: Actuation implies Start and Isolation implies Stop, which verifies there is only one correct answer.

26+ F 2 Y B S NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

4 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 27 H 2 Y N S NRC: 1 out of 2 taken twice.

28+ F 2 Y M E NRC: Why is this bank question considered modified? (ES-401D.2.f)

Was this bank question ever used on an NRC exam?

B Lic: Not Significantly modified-counted as bank question. Not used on the last 2 NRC exams.

29 H 2 Y N S NRC: 1 out of 2 taken twice.

30 F 2 Y N S 31 H 2 Y N S NRC: 1 out of 2 taken twice. Is CCW the same as SWS for K/A?

Lic: No, but CCW is the cooling system that serves the same function for RHRS as SWS.

32+ H 2 Y M S 33 F 2 Y M S 34 H 2 Y B S NRC: Was this bank question ever used on an NRC exam?

Lic: Yes. PI 2010 NRC Exam.

35 H 2 Y N S NRC: 1 out of 2 taken twice.

36+ F 2 Y N S NRC: 1 out of 2 taken twice.

37 H 3 Y N S NRC: 1 out of 2 taken twice.

38 H 3 Y B S NRC: Is Distractor D plausible for the rational given of no entropy gain? Was this bank question ever used on an NRC exam?

Lic: Plausible based upon misuse of the Mollier Diagram. Not used on the last 2 NRC exams.

39 H 2 X B U NRC: Distractors A & D are not plausible based upon the stem conditions and no statement given to assume the B logic relay does S not actuate as expected. Was this bank question ever used on an NRC exam?

Lic: Based upon the specific logic diagrams for these relay paths, distractors A & D are plausible. Not used on the last 2 NRC exams. Question is considered S.

5 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 40+ H 2 Y N S NRC: 1 out of 2 taken twice.

41 H 2 X Y M E NRC: Question is from the PI 2012 NRC Exam. How has this question been modified? Distractor B doesnt appear to be plausible for the rational given.

Lic: Distractor B changed to a plausible distractor. The question changed initial conditions in the stem such that a different answer is correct. Question is considered M.

42 F 2 Y B S NRC: Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

43 H 2 X N B U NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam? How does this question address the K/A to monitor Y S automatic operation of pump starts and MOV positioning? What causes the discharge valves to go back closed and then re-open? Is the correct answer really C?

Lic: Knowledge of the K/A is demonstrated by understanding why the containment spray pump and outlet MOV respond as they do. There is only one correct answer, as stated in the key. Question is considered S, and was not used on the last 2 NRC exams..

44+ H 2 Y B S NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

45 H 3 Y N S NRC: 1 out of 2 taken twice.

46 F 2 Y B S NRC: Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

47 H 3 Y B S NRC: 1 out of 2 taken twice. Was this bank question ever used on an NRC exam?

Lic: Not used on the last 2 NRC exams.

48+ F 2 Y N S 6 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 49 H 2 Y N E NRC: F not H. 1 out of 2 taken twice. This is memorization of actions initiated by an S signal, which is a given stem condition; F

higher level cognitive skills are not used to answer the question.

Lic: Agreed that this is a memory/fundamental question.

50 H 2 Y N E NRC: F not H. 1 out of 2 taken twice. This is memorization; higher level cognitive skills are not used to answer the question. Applicant F

either knows, or doesnt know, answer from memory.

Lic: Agreed that this is a memory/fundamental question.

51 H 2 X Y N U NRC: 1) Non-plausible distractors. Explain how the distractors A., B.

and D. are plausible. Difficult to read meter, may have to expand the S

picture to clearly show the scale and needle deflection position. 2)

Very minor cognitive thought process for how to read scale, getting correct scale, and doing simple math.

Lic: Distractors B & D are plausible if applicant uses incorrect scale direct readings. Photo is adequate to identify values required.

Although the LOD is not high, the applicant is required to perform several functions correctly to arrive at the correct answer, therefore this question is considered an H and is considered S.

52 H 2 Y N S 53 F 2 Y M E NRC: Not a modified question. Same question and same answer as previous question. Its just that the new question has part of answer B

in stem where old one had complete correct answer in the answer choice section.

Lic: Not Significantly modified-counted as bank question.

54 H 2 Y B S NRC: This is a GFE type question.

Lic: Applicant requires system knowledge to discern the correct answer.

7 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 55 H 2 X Y B U NRC: 1 out of 2 taken twice. 1) Is this memorization of the Containment Vacuum Breaker setpoint and simple recognition that S

vacuum breakers have exceeded the setpoint and opened? 2) Two Correct Answers (A. and C.). Failed to Close and Open are same concept. Also, Failed to Open and Closed could be considered same concept. 3) What does indication look like if only one vac break opens and other remains closed?

Lic: Added an initial condition to the stem to clarify that both vacuum breakers initially operated as expected. Analysis of the stem conditions is required to discern the correct answer. Question is considered S.

56 H 3 Y N S 57 H 2 Y N S 58 H 2 X Y N U NRC: Two Implausible Distractors. 1) A. and D. not plausible numbers. Dont understand justification for A. and D. 2) Just to S

clarify, there is no reference handout to applicant for this question?

Lic: No reference handout is provided to the applicants. Distractor A & D are plausible due to weaknesses in knowledge of the graph bases for axis intercepts and line slope rationale. Question is considered S.

59 H 3 X Y N E 1 out of 2 taken twice.

F NRC: F not H. This is recall of Containment Pressure setpoint/TS which is a well-known number. Also recall of the Post LOCA vent system to Shield Bldg. 2) Distractor D. not plausible. Why is 1.3 psig significant?

Lic: Distractor D is plausible because control stations for venting the Units are on the opposite side of the Aux Bldg. Agreed that this is a fundamental/memory question.

60+ F 2 Y N S 61 H 2 X Y N E NRC: F not H. 1) Fundamental memory of what actuates SI. 2) D.

not plausible. Psg greater than PRCS?

F Lic: Changed distractor D the MSL high rad to be plausible. Agreed that this is a fundamental/memory question.

62 F 2 Y N S 1 out of 2 taken twice.

8 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 63 H 3 Y B U NRC: 1) A. and D. not plausible. A., no reading given for or asked for at L/2. D., not believable to use a cube function. 2) Better question S

to require it a point source calculation (L/2 distance), ie, determine line source DR at L/2, then perform point source calculation to determine DR at greater than L/2.

Lic: Distractor A plausible if applicant misapplies line source rule, and D is plausible if plane source rule is misapplied. Question is considered S.

64+ F 2 Y B E NRC: 1) Is this using a higher cognitive skill, or is this memory of how to align cooling water pump as safeguards? 2) Show electrical H

diagram to verify alignment and believability of Bus 15 and 16.

Lic: Analysis of multiple possible power supply lineups. Question is considered an H.

65 H 2 Y B E S NRC: 1 out of 2 taken twice. 1) What higher cognitive skill is used?

Is this memory of minimum fire pressure and action to take? 2) Is the screen wash pump normally running with the screen wash to fire header valve closed as indicated? Would stopping the screen wash pump be an incorrect action? May need more explanation in the question for the screen wash situation. What is the relationship between the fire pressure and screen wash, ie, washing the fire water screen?

Lic: Analysis of the conditions given in the stem (fire is occurring) to correctly discern expected system response. Question is considered S.

66 F 2 X Y N E NRC: 1) One implausible distractor. C. and F. are basically the same area, so one of the choices is implausible. 2) Include Figure 2 in answer justification (6.1.4). If no Lead ROs, where does the procedure allow both remaining ROs to leave their panels at the same time, ie, out of sight of their indications?

Lic: Changed distractor A to make it plausible.

9 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 67 H 2 X Y B U REF provided.

S NRC: 1 out of 2 taken twice. 1) First part of question requirement identified in Table 1, i.e., verifies Bank position greater than minimum rod height per Figure C1-8. 2) Second part of question is TS that identifies the requirement and narrows question correct answer to a 50-50 chance of getting it correct.

Lic: The applicant is required to know how to verify Bank position and, using data provided, analyze what conditions are met. Question is considered S.

68+ F 2 X Y M E NRC: 1 out of 2 taken twice. 1) B. not plausible. Two audible SRIs and one for indication. 2) Not Modified. Basically, same question B

with same answer. Different descriptive words used, but still asking the TS requirement.

Lic: Distractor B is plausible since there are 2 audible SRI that are operable, but only 1 is selected at a time. Not Significantly modified-counted as bank question.

69 F 2 X Y N E NRC: 1) B. not plausible. When is Self-Check Only a requirement?

2) H not F. There would be other times when IV or PC would be H

correct, therefore, have to analyze the given situation.

Lic: Per operations procedure, sometimes are specified when Self-Check ONLY is required, therefore B is plausible. Agree question is considered an H.

70 H 2 Y B E NRC: 1) F not H. As written, it appears the applicant has to know the limits from memory to determine if they are exceeded.

F

2) How does applicant determine if the given parameters exceed the PTLR or COLR limit for B., C. and D.? Where is the SL specified in the reference?

Lic: Agreed that this is a fundamental/memory question.

71 F 2 Y B S 72+ F 2 Y B S 73 F 2 Y N S 74 F 3 Y N S 10 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 75+ F 3 N N U NRC: Does not address KA. KA is knowledge of the bases . . .

question not asking bases, i.e., not asking why?

Y S Lic: Knowledge of the K/A is demonstrated by understanding which procedure takes priority based upon conditions in the stem. Question is considered S.

For RO portion of Exam: New = 44Q, Modified = 3Q, Bank = 28Q (37.3%): LOK: 34/75 = 45.3%, H= 54.6%. Initial UNSAT: 1/75 = 1.3%, Enhanced: 15/75 = 20%.

76+ H 3 Y S N S 77# H 2 X Y X N U NRC: 1 out of 2 taken twice. NOT SRO only. RO should be able to answer this based upon integrated plant knowledge and knowledge of the procedures. ROs expected to know how AOPs used in conjunction with EOPs, and to know big picture actions like tripping RCPs and reactor.

Lic: Wrote new question. Re-formatted each choice to say enter procedure and perform action. Question is considered S.

78# H 3 X Y X N U NRC: Implausible distractor and Not SRO Only. 1) Distractor D not plausible. Throttling open a failed open flow control valve. If the flow control valve failed closed, no-flow indications (high discharge pressure with zero gpm flow) would result. Additionally, the stem conditions do not indicate a loss pump or valve malfunction to suggest using AOP 1. 2) Not SRO Only if ROs are trained to trip a pump that has indications of cavitation, then the SRO is not actually directing use of a specific procedure.

Lic: All choices were re-formated for SRO level. Distractor D was changed to manually throttle close the 11/12 RHR HX Bypass Flow Valve instead of throttle open the 12 RHR HX Outlet Flow Control Valve. Question is considered S.

79# H 3 X Y S M U NRC: Implausible distractors. For A distractor what makes a value of 63% (given in stem) plausible vs 33% for switchover criteria?

Also, are the ROs required to recognize the 33% level switchover criteria? Same rational is applicable to D distractor with the 105,000 gal level.

Lic: 1) RWST level too high in stem, 43% would be ok (other criteria is 8%). Adjusted RWST level from 63% to 43% to address distractor A.. 2) Deleted CST level, replace with E.1 procedure. 3) Changed Distractor D. to 1E-1 instead of 1C28.1 AOP2. Question is considered S.

11 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 80# H 3 Y S N E REF provided.

NRC: Reword stem to make D. the correct answer.

Lic: Stem revised to have Bus 16 green load rejection lights NOT lit.

This makes choice D the correct answer. Added bullet to stem to state offsite power is NOT available.

81+ H 3 Y S B S 82 H 3 Y S B S 2012 NRC Exam 83+ H 3 Y S N S REF provided 84+ H 3 Y S B S REF provided.

NRC: Modify question Explanation to identify ODCM is part of Administrative TS Section 5.0. There is no above line, below line or Safety Limits. All of the Administrative TS items are SRO Only.

Lic: Modified justification for SRO only.

85# H 1 X Y S B U NRC: Low LOD. Two implausible distractors. Based upon the information provided in the stem, why would it be plausible for the applicant to believe that SG pressures are lowering uncontrollably?

Same for B and C.

Lic: Fix the explanations. Changed two distractors: 1E-2 to 1ES-0.1.

and 1E-3 to 1ECA-1.2. Re-ordered distractors using waterfall method. Question is considered S.

86+ H 1 Y S N U REF provided.

2 S NRC: Low LOD. Question is a reading exercise for SRO that requires abiding by a procedure Caution, and comparing given data with that in the procedure. Does the use of a reference makes this question a direct lookup?

Lic: Several decisions/analysis involving recalled knowledge is required to discern the correct answer. Question is considered S.

12 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 87# H 3 X Y S M E NRC: 1) Distractor A is currently a subset of distractor D. 2) How is this a modified question? Only changed one distractor and deleted B

stem data that said D1 and D2 EDGs running unloaded with does not change the correct answer.

Lic: 1) added ONLY to distractor A, and reworded distractors to state Enter vice per for answers; put procedures first then actions;.

All choices re-formated. 2) Question is considered B.

88+ F 2 Y S N S NRC: 1 out of 2 taken twice.

89+ H 3 Y S N S REF provided.

90# H 3 X Y S N E NRC: Distractor D. not plausible. Also, distractor C is potentially cueing the applicant to the correct answer since it is the only Non-Refueling option to choose.

Lic: Added enter vice per; R-31 piece of distractor D was removed; two distractors were modified; all choices were re-formatted and re-ordered.

91+ H 2 N S N U REF provided.

Y S NRC: Does the use of a reference make this question a direct lookup? Does not meet KA for ability to predict impacts.

Lic: The proper use of TSs does address the impact on the system.

Several iterations of the TS condition 3.3.1 Table is required to discern the correct answer. Question is considered S.

92 H 3 Y S M E REF provided.

B NRC: 1 out of 2 taken twice. The applicant handout should not include the TS bases. This question is not modified.

Lic: The reference provided does not include the bases. Not Significantly modified-counted as bank question.

93+ H 3 N S N U REF provided.

NRC: Does the use of a reference make this question a direct lookup? Does not meet KA for ability to predict impacts.

Lic: A comment was added to the stem to address performance of required actions in order to meet the K/A. Question was U and is now considered S.

13 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 94 H 2 Y S M U REF provided.

S NRC: Does the use of a reference and simple math make this question a direct lookup?

Lic: Knowledge of the bases is required to correctly apply the corrections and discern the final answer. Question is considered S.

95+ F 2 Y S N S 1 out of 2 taken twice.

96# H 2 X Y S N E REF provided.

NRC: The question should address mitigation of the Orange path to meet the intent of the K/A; how is this accomplished? Are the ROs required to be knowledgeable of potential impacts of orange paths?

(Potentially not SRO Only as written). Reword the stem question to address the EAL status of the fission product barriers. Also, suggest deleting the RCS status column as this appears to decrease LOD, and reword distractor B. or C. Containment section to say POTENTIAL LOSS.

Lic: The stem was modified to include a statement directing analysis using the SRO level reference.

97# H 2 X Y S B E REF provided.

NRC: Distractor A not plausible. Change to NOT approve.

Lic: Revised distractor A to be plausible, and reordered choices.

98+ F 2 Y S B S 2010 NRC Exam 99# H 3 X Y B U REF provided.

NRC: Step 17b RNO is not performed until Narrow Range is greater than 50%, and the stem states level is 12%. Why would the applicant try to perform the RNO action early? Seems like the correct answer is A. Then when level is >50% narrow range and uncontrollable he will perform the 17b RNO. What are we missing?

(UNKNOWN)

Lic: Modified 12 SG indications to ensure that the intended correct answer is the only correct answer. Question is considered S.

100+ F 2 Y M E NRC: This question is not modified.

B Lic: Not Significantly modified-counted as bank question.

14 of 15

ES-401 Written Examination Review Worksheet Form ES-401-9 PRAIRIE ISLAND [August 2014] Exam

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only For SRO portion of Exam: New = 13Q, Modified = 2Q, Bank = 10Q (40%): LOK: 4/25 = 16%, H= 84%. Initial UNSAT: 1/25 = 4%, Enhanced: 2/25 = 8%.

  1. SRO Only question submitted for NRC pre-review. Total of 10. 6/25 = 24%, 7/25 = 28%

15 of 15

Retrieved from "https://kanterella.com/w/index.php?title=ML15128A292&oldid=2033730"