ML15167A147: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
 
(Created page by program invented by StriderTol)
Line 13: Line 13:
| page count = 22
| page count = 22
}}
}}
=Text=
{{#Wiki_filter:ES-401 CGS-2015-04 Written Examination Review Worksheet Form ES-401-9  Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: $ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). $ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). $ The answer choices are a collection of unrelated true/false statements. $ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. $ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: $ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). $ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). $ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). $ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Attachment 8_Rev 1 Page 1 of 22 OBDI 202 - IOLE Process    Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only                You did not follow several of the general comments that were made during the free review, which included General comments for submittal 1. Justification for why one answer is correct does not automatically make other answers incorrect. 2. Use the NUREG standard worksheet ES-401-9 for exam questions and complete all fields, incl LOD from now on. 3. Always put in field for bank question that it has not been used on an NRC exam if that is the case. 4. Don't use slightly modified as a term - use definitions in NUREG. 5. Use noun names for designators in addition to numerical names per NUREG-1021 6. There are too many handouts for this question and too many questions (12) on the SRO exam that have handouts provided. 7. Due to number of changes, for revision control the final submittal has been ES-401 CGS-2015-04 2 Form ES-401-9  Attachment 8_Rev 1 Page 2 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only designated as Rev 2. All questions have been changed to reflect Rev 2. 1 H 3    X    C 41.5  N U  S Free review Did NOT use free review comments to fix question. Once you know first part of answer C then you have the answer since it is unique (ie not a balanced 2 X 2) from the others. C is also a subset of D without "only" in the first part of D. Multiple bad distractors make this question unsat. Licensee wrote new question and as rev1needs to beef up discussion on why pulling control power fuses is now sat. 2 H 3    X    B 41.5  N U  S Not plausible to think reducing flow through heat exchangers causes temperatures to decrease making C and D not credible. Also, this question is GFE knowledge since the question is asking basic heat exchanger knowledge and multiple poor distractors makes the question Unsat. Licensee modified existing question and as rev 1 but it was not in proper format so with merge to rev 2 is now sat. 3 F 2    X    C 41.7  N E  S Knowing generic purpose of a keep fill pump is GFES and allows one to throw out the distractors "A" and 'B." This makes these two distractors partially not credible. After validation some of the LO's picked this distractor as correct, therefore licensee added this to justification and we accepted as sat. 4 H 3 X      C 41.7  N E S Remove "Initially" in the stem since HPCS is normally aligned to the CST. Made edit as recommended and question is now Sat as rev1. 5 H 3    X    A 41.7  B E 
S Free review Did use free comments on this question. The water leg pump P-3 information at the end of each choice is extra information and is not needed to answer the question and therefore should be removed. It only clouds the choices. Also need to balance the second part of distractor D regarding V-4 and V-12 with a different aspect than distractor C, such as V-4 and V-12 remain closed due to loss of initiation logic power. Made edit as recommended and question is now Sat as rev1. 6 F 2    X    C 41.7  N E  S Free review Please provide more information in the justification on section of the worksheet for the credibility of distractors "A" and "B" for MC-7A and 8A. Made edit as recommended and question is now Sat as rev1. 7 F 2        D 41.6  N S Free review-made corrections as requested although sent it as Q4 on free review. 8 H 4  X X    A 41.7 X  M U Free review ES-401 CGS-2015-04 3 Form ES-401-9  Attachment 8_Rev 1 Page 3 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S DID NOT use the free review comments. This is a series of T/F statements, also B and C not credible, and does not meet KA for function and purpose as originally provided in the free review comments, which were "True false question. Loose K/A connection. Purpose and function needs the why controls are the way they are and question asks what (which is function but not purpose)." Multiple bad distractors, T/F statements, and KA mismatch make this question unsat. Licensee wrote new question and as rev1needs to beef up discussion on function and purpose to the two components and is now sat. 9 F 3    X    D 41.7  N U    S If you know that period must be stable for criticality then you can eliminate the other three choices without knowing any other information. Need to make this a balanced 2 X 2 to fix this condition. You could move one piece of information to the stem to complete the three requirements for this question. This is also not H3 question as written. It is really an F3 question as written since it is one item that is memorized. Multiple bad distractors make this question unsat. Licensee modified question as rev1and needs modification to be site specific so still working on solution. Did not address both pieces of KA (both count rate and period) with rev1 of the question. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. 10 F 3        B 41.7 X  N E      S Free review Did use the free review comments, however the change you made gives the appearance now of KA mismatch. The two negatives issue was corrected but the KA is on max disagreement not min disagreement. Also, you are tying this to rod block situations. Are there any other issues that occur when there is disagreement between the flow channels? If so, is there is a value which, upon exceeding, the circuit automatically does something, such as the ind. Channel is auto removed from the ckt, or other things)? If so, then this is what the KA is driving you to ask about. Work on justification of numbers. Updated question via e-mail on 4/6/2015 to incorporate these comments and question is now Sat. 11 H 3        D 41.7  B E S Free review Add RCIC before "full" in distractor "B" to match answer "D" Made edit as recommended and question is now Sat as rev1. 12 F 2        B 41.7  N S Free review 13 H 3        C 41.7  N S Free review 14 F 2        D 41.7  N E Free review No discussion on why distractors A and B are credible but not correct. They seem to ES-401 CGS-2015-04 4 Form ES-401-9  Attachment 8_Rev 1 Page 4 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only    S be correct to me. Is C a made up distractor (ie are there vac bkrs installed below the floor)? If so then it is not allowed. Is this even credible? Do the SRVs even go below the floor anyway? Licensee explained this question to us during validation week, and then added better justifications for distractors and question is now sat. 15 F 4        A 41.7  N S Free review 16 H 4    X    A 41.7  N E  S Free review Distractor D needs to be edited. A "do nothing" distractor is not allowed on an initial exam. The distractor could be the lag fan starts and is unable to maintain secondary containment. This would test the applicant's knowledge of system design that the SGT is 100% redundant between lead and lag fans. Make corrections that also changed the answer and justification as well as repairing the one poor distractor and is sat as rev1. 17 F 3 X      D 41.7  N E S Free review Need to place comma in stem after fire and before direction. 18 H 2    X    D 41.7  N E  S Free review A is not credible as provided in the free review and was ignored. The question could be repaired with suggestions already given in free review. There are also other ways to address this KA. Stem could also be shortened considerably and still have same meaning. Made suggested changes to A and is now Sat as rev1. 19 F 3 X      D 41.5  B E S Used on last NRC Exam Remove "What will be the impact of starting CW-P-1C in the current plant configuration"  from the stem. It is not necessary for the question. Made suggested changes to A and is now Sat as rev1. 20 F 3  X  X    D 41.5  M E   
S Partial cueing with inverter in stem and inverter procedure in answer. Better question would have the percent of voltage (such as 95% and MC-7A or 7F and 90% with MC-7A or MC-7F) with a stem like "As voltage is dropping, what is the first value of voltage that a transfer would occur and to what source would it transfer to." Knowing that the alternate is the first choice is GFES for inverters, but knowing that the alternate is MC-7F for this inverter is plant specific. A. 95% and MC-7A B. 95% and MC-7F C. 90% and MC-7A ES-401 CGS-2015-04 5 Form ES-401-9  Attachment 8_Rev 1 Page 5 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only D. 90% and MC-7F Made suggested changes to A and is now Sat as rev1. 21 H 2 X  X    B 41.7  M E  S The spirit of this KA is for meters, dials, alarms in the control room. Are there any meters or dials in the control room for DC electrical parameters? If so then these should be used to match this KA. Also, if the applicant asks if the CRO2 operator is currently holding the switch how do you answer that during the exam admin and not give away the question? This makes C and D not credible if you give away the information during administration of the exam. Fix distractor D to be annunciator is-valid to match other choice. Licensee made necessary changes to question - Rev 2 of question is now SAT. 22 H 3        B 41.7 X  M E    S Free review Did NOT take the free review comments that the HPCS diesel is not really meant for this system and KA. See system 209002 for HPCS. In that system topic, the HPCS diesel has many KAs for use on the written exam. The Div 3 EDG is not normally credited for SBO either, hence the reason for the comments here. Need to write a question on the Div 1 or 2 EDGs to meet this KA. Wrote question on div 1 EDG (had to resample KA because Div1 /2 EDGs at CGS do not have high crankcase pressure trips) which we reviewed with them via telecom on 4/7/2015 from region and we agreed that the question is now Sat. 23 H 3        C 41.7 X  M U    S Free review Did NOT take the free review comments which were that this is a "K/A mismatch because there was no transfer between the two modes of manual and auto. Fix verb tense in distracter B. You might ask something like when the EDG is in manual mode, what causes it to shift to auto mode and close the O/P breaker. In this question the EDG is essentially in manual and stays that way." Licensee didn't change anything on this question. KA mismatch makes this question unsat. Licensee wrote new question and is now sat. 24 F 2        A 41.2  N S Spell out abbreviations for final submittal on 4/10/2015. 25 F 2        C 41.7  N S Free review 26 F 2    X    C 41.7  N U Free review Made a minor change in an attempt to correct the subset issue but 2 is still a subset of 1. Subsets make the question  Unsat, which was how it was marked in the free review as well. Consider this as an alternative: The RPV water level set-point for the trip of an RCC pump at full power is: A. -50 inches NR ES-401 CGS-2015-04 6 Form ES-401-9  Attachment 8_Rev 1 Page 6 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  S B. -50 inches WR C. -135 inches NR D. -135 inches WR Or something like that-.use different instruments (FZ, WR, NRC, etc) and the actual set-point so you avoid subsets. Multiple bad distractors make this question unsat. Licensee wrote new question as rev1 and is now sat. 27 H 3        B 41.7 X  N U    S KA mismatch-the KA is asking for the effect on the rod block monitor for a malfunction of RMCS, in this case RDCS loss of power. This does not cause any effect on the rod block monitor as you indicate in the answer and worksheet discussion. This is not the spirit of the KA-you should try to cause a loss or malfunction that prevents rod blocks from working-.ie no effect is not what is intended for the KA. You also have the wording wrong in distractor D, "enforced" should come after "be." KA mismatch makes this question unsat. New KA selected K3.01 wrote new question as rev 1 and is now sat. 28 H 4    X X  A 41.7  N U  S B is not credible as written. There is no connection  to the blade with any information system so this is not credible in distractor B. For distractor D, it appears to be correct because as long as you don't move it, you don't get a rod block. This makes the question unsat for two correct answers. Licensee modified question as rev 1 and is now sat. 29 H 3    X    B 41.7  N U S Second part of answer for reactor power is not part of KA and is also not needed to answer the question as written. It actually helps to eliminate knowledge about water chemistry by eliminating  A and C on power knowledge alone, and this is not part of the KA. Just remove the reactor power part of the question and you have a good 2 X 2 on just water chemistry issues. Also, conductivity and pH can be moved into the stem.  "Which of the following are indications of this event?" can be removed from the question. Why are the other answers plausible? Licensee is not providing any basis on any of these questions on why distractors are credible but not correct. Multiple bad distractors make this question unsat. Cueing in rev1 of this question. Needs work. Fixed cueing but this is a major revision to the question so should be rev2 now but distractor C is not credible (during telecom on 4/7/2015). Licensee asked to resubmit  for review. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. 30 H 3 X      A 41.7  N E  Free review Licensee corrected items from free review. However, now you have the TIP detector ES-401 CGS-2015-04 7 Form ES-401-9  Attachment 8_Rev 1 Page 7 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  S auto stops inserting in all four choices, and per NUREG should be moved to stem. Again, you don't have any explanation on the credibility of the distractors. Why would someone pick the shear valve, for example? Licensee made necessary changes to question - Rev 2 of question is now SAT. 31 H 2    X    D 41.7  X N U 
S Free review Did NOT follow feedback on free review comments, which was  "ED setpoints are well known, allowing A and B responses to be thrown out. A is also a true/false statement. Not reasonable to cool SP and stop RPV injection. For generic questions. This is usually associated with SRO type questions and this question could be viewed as SRO-only, depending on how its read." Answers A and C don't seem credible. A is effectively a subset of B. An operator wouldn't prioritize cooling the wetwell over keeping the core covered making C not plausible. Choosing RPV level and ignoring SP temp is SRO level (deciding on which equipment you can lose and which you have to try to save while deep in EOPs). Multiple bad distractors make this question unsat. Licensee modified question and distractor D needs work to ensure it does not read as a no action required and fix tabs to bullets. 32 H 3        B 41.7  N S  33 F 2        B 41.7  N S Free review 34 H 3        C 41.7 X  N U   
S Free review Didn't answer the free review comments to the question of do the valves have auto open and close features for turbine load changes? Licensee rewrote question to address the close only portion of the KA, which doesn't meet the KA now that this change was made. Based on the noun name and the high power level answers B and D don't seem credible. It's intuitive that those valves are closed at full power. Also, there's no justification for using 30% power as a discriminatory value. KA mismatch makes this question unsat. With the rev2 version it isnow sat. 35 H 3 X      B 41.7 X  B E Free review Since the free review did not catch the negatively focused stem it is marked as E instead of U even though it has no correct answer to the stem as written. In the stem eliminate "Which of the following is correct?" and "Based on the above,".
Also, Answer B does not answer the stem of how to reset the RFW pump turbine so ES-401 CGS-2015-04 8 Form ES-401-9  Attachment 8_Rev 1 Page 8 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only   
S no correct answer to the question. All it states is a partial T/F statement that you can't reset the RFW turbine with reset on the switch in the stem. Negatively focused stem and answer combos are discouraged in the NUREG. It would be acceptable if you asked what items needed to be done to actually reset the RFW pump turbine on RPV level issues to meet this KA. Another approach would be to ask in the stem what has to be done to open the HP and LP stop valves for this pump. Another even more appropriate question is the actual high level value that causes the scram and rfp trip. What caused the trip and what is the effect (ie what does it take to restore fw to the RPV)? Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. Ensure "minimum" is capitalized in stem of question for final submittal on 4/10/2015.. Licensee made necessary changes to question - Rev 2 of question is now SAT. 36 F 3    X    A 41.7  N U    S Free review Did not take the free review comments and make recommended changes. Answers C and D seem to be true/false statements that aren't credible since water can't be pumped directly to the destinations. Also, if there are no rad monitors in these lines how it is credible to have them as choices? This makes B not credible as well. Multiple bad distractors  make this question unsat. Licensee wrote new question as rev1 and needs work on stem focus to ensure legally defendable.Fixed concerns and is now sat. 37 F 2    X    A 41.7  N E   
S Free review Changed question based on feedback from free review. Now you have subsets. If you create the following selections it will work: A. Only FP-P-2A B. Only FP-P-2A and FP-P-2B C. Only FP-P-2B and FP-P-1 D. Only FP-P-1 On telecom on 4/7 still had errors not corrected from comments. Remove comma from C and put "Only" in front of choice D for "FP-P-1." Will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 38 H 3 X      X A 41.5 X  N E    Free review The free review did not identify that some choices can be eliminated without knowing information desired because it is not a balanced 2 X 2. Although knowing AOP entry conditions is RO knowledge with the new guidance, it would be better to not have them select the procedure, but use steps from the applicable procedure to combat ES-401 CGS-2015-04 9 Form ES-401-9  Attachment 8_Rev 1 Page 9 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S the condition (Hi Rad). It seems to be minutia that the toilet fan is all that stops for this condition. To meet the safety aspect of hi radiation, at what point does the outside fans stop and recirc fans to the CR start? This is the knowledge that the KA is trying to measure. This one still needs work. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question has proposed answer as B but correct answer is A. we will verify it is corrected on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 39 H 3  X      C 41.10  N E  S Reference needed on draft submittal question Ensure the printout is in color for the exam and that the % power is not highlighted as shown in this attached picture because this cues the applicant to the correct answer. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. References required in Rev 2 of question. 40 H 3        A 41.10  N S Free review 41 H 3    X    D 41.7 X  N E      S Free review The answer to the question is essentially none of the above. These types of questions are not allowed on an initial exam. Each answer could be paired down to look more consistent with each other. Also, RCIC is not credited as a system necessary to assure plant shutdown. K/A mismatch. It is marked as an E because these problems were missed on the free review. Updated question via e-mail late on 4/7/2015 to incorporate these comments and still needs to remove scram air header from all four choices since it clouds the choices. we will verify it is corrected on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 42 F 2        D 41.5  N S Free review 43 H 3    X    D 41.7  N U      S Free review Answers A and B don't appear to be credible since an operator would know that only 2 buttons are required. Having answers combinations of A1, B2 or A2, B1 or A1, A2 should be sufficient to give enough credible distractors. Another consideration is what is the minimum number that the procedure requires and if it states that you must push all four then it could be argued that A or B are also correct from a procedure perspective. From an electrical perspective you would be correct that D is the only correct answer. The stem also states that you press four buttons and yet the answer is you press two buttons, this implies there is no correct answer. Multiple bad distractors and no correct answer make this question unsat.
ES-401 CGS-2015-04 10 Form ES-401-9  Attachment 8_Rev 1 Page 10 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Licensee wrote new question and as rev1 and needs work on distractors C and D. Licensee wrote new question and as rev1 is now Sat. 44 H 2        B 41.10  N S Free review 45 F 2        C 41.8  B E S Remove "Which of the following describes the reason for the required reactor scram?" not necessary for the question. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question still non-credible distractor B. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 46 H 2    X X  A 41.7  N U      S Free review Did NOT fix comments provided during the free review. Even though the justification states otherwise an applicant could argue that HPCS is designed to maintain RPV level because it is a backup system to RCIC. This question may have more than one right answer. Also, why is CRD a credible choice-it is such low flow capacity it could not possibly keep up? This affects the distractor C. More than one correct answer makes this question unsat. Licensee wrote rev1 to question and is now Sat. 47 F 3    X    D 41.5  N E S Why is A credible under these circumstances? This is an F2 question not F3 as you have classified it. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question appears to have no correct answer. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 48 H 3 X  X    C 41.10  N  E   
Add high in front of drywell in the stem. Distractor D does not answer the stem at all-it has nothing to say about wetwell pressure and is therefore not an acceptable distractor. The perspective is also incorrect on distractor B and answer C. When you ask what will happen to wetwell pressure, you have to direct the answers towards wetwell pressure, not drywell pressure. To fix B and answer C, this is what you need to do: B. When the drywell pressure increases to 0.2psid higher than wetwell pressure, the wetwell to drywell vacuum breakers will open, maintaining wetwell pressure 0.2 to 0.5psig lower than drywell pressure. C. The steam leak will pressurize-.. venting drywell to the wetwell, maintaining wetwell pressure approximately 5 psig lower than drywell pressure. Licensee did not use our comments for a second time-reworking this question and ES-401 CGS-2015-04 11 Form ES-401-9  Attachment 8_Rev 1 Page 11 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  S will resubmit prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 49 F 2  X  X    D 41.7  N U    S Free review The stem cues the applicant into looking at a high pressure causing A to be eliminated and B is in the normal range of pressures after a scram. Change the second question in the stem to read "While verifying RPV pressure CRO1 reports that both RRC-P-1A and RRC-P-1B have tripped."  This eliminates the cue and makes all of the distractors credible. Change A to a number greater than 1000 psig for consistency and better credibility. Cueing makes this question unsat. Licensee made recommended changes and as rev1 is now Sat. 50 F 3        A 41.7 X  N U  S KA mismatch-there is no Suppression pool high temperature condition in the stem or answer therefore KA mismatch and question is therefore unsat.if the KA were for temp monitoring conditions for low  SP level then you would have a good question. Question is Unsat due to K/A mismatch. Licensee wrote new question as rev1 and is now Sat. 51 F 3        D 41.10  N S  52 F 3    X    C 41.8  N E  S B is not credible. Use something safety related more along the lines of the other choices in the EOPs. Worksheet missing lots of information. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 53 F 3    X    D 41.7  N E S Free review Answer A is not credible. Basic GFE knowledge would allow an operator to reason that if you allowed level to lower that pump damage would eventually occur with the stem condition that the SP suction valve is open. A better way to ask this question is What is the EOP XX.XX vortexing limit for HPCS when taking a suction from the SP? Then use different values from the table but ensure that the EOP specific info is included in the stem to prevent challenges from other procedures, such as SOPs, FSAR, etc. Formatting issues now - Will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 54 F 2        A 41.5  N S  55 H 2        D 41.7  B S Free Review ES-401 CGS-2015-04 12 Form ES-401-9  Attachment 8_Rev 1 Page 12 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 56 H 3        A 43.4  X N U  S Reference needed on draft submittal question Free review This was given as feedback as SRO-only during free review. Licensee provided no rebuttal on this comment and it still appears to be SRO only question since nothing was changed on the question and is therefore unsat. Licensee wrote new question as rev1 and is now Sat. 57 F 3    X    B 41.8  M U  S Free review Did not use the free review comments, did not change the question or add any justification to why distractors C and D are credible. Two non-credible distractors make the question unsat. Licensee repaired the two bad distractors and as rev1 is now Sat. 58 H 4    X    A 41.4  N U  S Free review Answers B and C are not credible distractors. B doesn't appear to have any safety impact. Just by reading power system stabilizer would cue an applicant that C is incorrect. Two non-credible distractors make this question unsat. Licensee repaired two bad distractors and as rev1 is now Sat. 59 H 4        C 41.10  N E S Free review Need to add without any further operator action to the stem of the question after power increase is stopped to prevent other answers from being argued as correct. Question: Does 24% reactor power ensure that MT is LT 560MWe? Unbalanced distractors as provide in the free review comments. Take MSIVs out of stem and Make the selections like this: A. Reactor SCRAM occurs due to MT trip B. Reactor SCRAM occurs due to MSIV closure C. Reactor remains at power with a slight increase in power D. Reactor remains at power with a slight decrease in power Remove lowering condenser vacuum from entire question since CGS uses backpressure instead of vacuum to remove unnecessary information from stem and change power to 20% in stem and justifications to ensure not challengeable with other loads and demands on the plant. We will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 60 H 3    X    D 41.5  B E  Free review Answer C is not credible. When is RCIC ever used to control RPV water level during a normal plant startup and it is GFES to know that a turbine pump needs higher pressure steam to run than 75psig?  This is also a Fukushima lessons learned event ES-401 CGS-2015-04 13 Form ES-401-9  Attachment 8_Rev 1 Page 13 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  S  with ED. Need to find a plausible injection source that could be used during startup. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 61 F 2    X    B 41.10  B E  S No justification why the distractors are credible. Feedwater is already injecting. A doesn't seem credible. How could the cooldown be violated?  Why is D credible? Multiple non-credible distractors makes this question unsat. Licensee made justification section for each choice and as rev1 the question has minor edits needed - we will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 62 H 2    X X  C 41.7  N U E  S The RPS actuation from the mode switch is an automatic action. Operators have guidance that when an automatic action fails then manual action is required. A competent operator would see that ATWS conditions are indicated and take the other RPS actuation to scram. This would mean pushing the manual scram buttons. Since 1 and 2 are ATWS conditions this would force the operator to push the scram buttons. All answers are true which makes this question unsat. Still needs editing but changes agreed to that we suggested. On telecom review on 4/7 we agreed with licensee that this question is good but needs minor edits and we will review on final submittal on 4/10/2015. Licensee submitted rev2 of question and is now Sat. 63 H 3    X    B 41.4  N E     
S Answers C and D don't seem credible. Are these components located inside the drywell?  Need to find other drywell components that could be impacted to meet intent of NUREG credible distractors. A distractor has poor grammar and therefore would not be selected anyway. A seal failure would not occur for a while anyway. This is an H2 not H3 question. This question is Unsat due to two non-credible distractors. Reselect another KA and write different question. Licensee wrote a new question based on a new KA but the stem doesn't state how the fw controller fails so how can you know what the resulting plant response will be? This affects the answer. Also, this question does not address the reason for the response as required by KA therefore KA mismatch now. Went over with licensee via telecom Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Sent new question and it is now Sat. 64 H 3    X    C 41.8  M U    Free review Why would distractors A and B be credible for the given conditions in the stem? There's no indication that a fire could exist making A and B not credible. Could use other water sources in room to make credible distractors. This would be a modified bank question with changes already made to the original bank question (ie the ES-401 CGS-2015-04 14 Form ES-401-9  Attachment 8_Rev 1 Page 14 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  E  S answer and distractors have been changed). This is an H2 not H3 question. This question is Unsat due to two non-credible distractors. Licensee wrote different distractors and now needs to edit for singularity of instruments in all four choices and it will be Sat as rev1 of the question. This was completed during telecom on 4/7 but water level instruments needs editing for alignment in all choices. We will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 65 H 4    X    D 41.8  N E S Answer A should be changed to some water discharge from LPCS, HPCS, RHR A, or RHR C since all these components are on the same level in the reactor building. Also, A is the only answer that drains to the EDR system making it not as plausible as the others. It appears that RHR A drains to R-1 sump. FYI-This KA is really asking high sump levels cause a ground or ckt malfunction due to high water, not vice-versa. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 66 F 2    X    A 41.10  N U 
S Free review During free review comments were made that this was too easy (LOD = 1) and distractors needed work and nothing was done on this question. On appeal an applicant would be able to argue that any of these conditions could be announced. The question is not discriminating enough. Should be announced is too subjective. Distractor C has a typo and would not be picked anyway. This question is Unsat due to LOD-1. Pulled bank question and is now sat. 67 F 2        C 41.10  B S Free review 68 H 2    X    B 41.10  N E S Free review For D, change distractor to Insertion of the bundle may continue at a slower rate to prevent exceeding 300 cps. For C, is it plausible to leave a bundle suspended on the hook?  If it is then more justification is needed in the justification section of the worksheet. This is an H2 not H3 question. During telecom review on 4/7 noticed that justification does not match the choices above -Licensee to resubmit prior to final submittal on 4/10/2015.Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 69 F 2    X    B 41.5  N E  S Subset issue with minutes. Need to add minimum minutes to the stem or state in the stem as required in the procedure step for this condition. Also, change the SOP to ABN-CRD to make the distractors more plausible.
ES-401 CGS-2015-04 15 Form ES-401-9  Attachment 8_Rev 1 Page 15 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Licensee made changes for telecom on 4/7 and question is now sat. 70 F 2        A 41.10  B E S Not an F3 but F2 question. Licensee made changes and question is now Sat. 71 F 3    X    D 41.7  N E  S Remove "Operation in this range-" from all four choices. During telecom on 4/7 we commented on the need to put "range" at end of answer D to keep all choices balanced and to have no cueing. We will check on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 72 H 3        C 41.12  N S Free review 73 F 3        D 41.12  N S Free review 74 F 3 X      C 41.10  X B S Free review Did not use the free review comments which were the following: Borderline SRO question. This is a tier 3 Generic K/A and the question is too specific for this Tier. You should reject this KA and reselect." We should have given better KA for this question so calling the new question sat and original that was unsat was reclassified as sat due to our providing a poor KA to the licensee. Question is unsat for incorrect license level. Question is Sat with new KA, 2.4.12. 75 H 3        A 41.10  B S Free review 76 H 3    X    D 43.5  N U        S Reference needed on draft submittal question This was a free review question. The stem was changed after the free review which now gives away the Mode 3 piece and therefore makes distracter A not credible. Also, you changed the correct answer without changing the stem meaning so what happened there? Also, verbatim copying of the TS statements is not recommended per the NUREG, especially if you give them all or most of the references needed to answer the question. Requiring the CRS to know a 2 hr TS from memory or details in the bases document on the DC system would be better than handing out 4 different TS statements. Also, without the B2 TS statement from 3.8.4 in the Proposed answer "D" it makes no answer correct anyway and the question is therefore unsat. There are too many handouts for this question and too many questions (12) on the SRO exam that have handouts provided. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat.
ES-401 CGS-2015-04 16 Form ES-401-9  Attachment 8_Rev 1 Page 16 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 77 H 2        B 43.5  X N U S Free review - question replaced. New question is unsatisfactory for the following reasons: Not an SRO only question. Question requires major EOP mitigating steps and systems knowledge at the RO level. It does not become an SRO question by adding "what direction should the CRS give?"  There needs to be a procedural transition requiring the SRO to analyze the situation and decide on the correct sub-procedure or appendix that must be used. This question is unsat due to incorrect license level. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat. 78 H 3  X  X    C 43.5  N E S Reference needed on draft submittal question Free review - question replaced. Comments on new questions are as follows: There is no need for the first of the two questions. The same information is required (# of available bottles for ADS to be considered operable) to answer both questions. Absolute terms such as "ALL" act as cues that the answer choice is incorrect.
Additionally, due to the change in verb tense from the other answer choices, Distractor D could be read in a way where it is considered a subset of the correct answer - replace four bottles, and then exit TS 3.5.1. If four bottles are replaced, you can certainly exit the TS. Can the question be asked, "per the TS basis, what is the minimum number of bottles that must be replaced to return ADS to operable status?" Licensee made changes and question as rev1 is now sat. 79 F 4    X    D 43.5 X  N U   
S Reference needed on draft submittal question Free review - question rewritten. New question is unsatisfactory for the following reasons: K/A mismatch - the question does not constitute a "refueling accident."  Distractors A & B not credible - it is not plausible to believe that an applicant will believe that refueling is being conducted with the unit in Mode 2 or 4. Also, Distractor B is a 1-hr action statement, so SRO applicants will know that this is not the answer as it is RO knowledge. Distractor C - use of "no action required" type statements are not effective as correct answers or distractors as it gives the appearance of a trick question. This question is unsat due to K/A mismatch. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat. 80 H 3  X  X X  A 43.1  B E  Reference needed on draft submittal question Distractor C - Choices that contain qualifying statements like "as long as" are poor ES-401 CGS-2015-04 17 Form ES-401-9  Attachment 8_Rev 1 Page 17 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  S distractors. They are very rarely the correct answer and therefore cues the applicant. Distractor D - could be considered a correct answer. Need to add "without a risk assessment being performed" or something to this effect. Also, need to say 24 hours "from now" or somehow clarify what starting point you are talking about. Licensee made changes and question as rev2 is now sat. References required in Rev 2 of question. 81 H 4        A 43.5  N U S  Reference needed on draft submittal question Free review - question rewritten. New question is unsatisfactory for the following reasons: Direct look-up with the references provided. Handouts should be removed or the question should be replaced. It is reasonable to ask SROs about standard PARs without giving them the flowchart. This question is unsat due to LOD-1. NRC submitted possible new question  and licensee didn't like it so they are writing to GE rad levels and call it rev2 and it is now sat. 82 H 4    X X  D 43.5  N U 
S Reference needed on draft submittal question Distractor C is another correct answer as the questions asks "what actions are required?"  It is required that 3.8.1 A&B are entered and that a source be restored within 72 hrs, even though there is another requirement to restore DG-3 within 12 hrs. Distractor B is not credible. It is not plausible to think that an applicant would not enter Condition A - they are told in the stem specifically that offsite power cannot be restored within TS limits.      This question is unsat due to two non-credible distractors. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat.rev2 with 3800 and minor edits resubmit as sat. References required in Rev 2 of question. 83 H 3        B 43.5  N S Free review - question replaced and is now Sat. 84 H 4        D 43.5  X M U    Reference needed on draft submittal question Not SRO only - 1 hour or less TS required actions are RO knowledge. Distractors A & B are not credible. There is no plausible reason why an applicant would choose these distractors. The question is essentially a direct look-up. Need parent question to determine if question has been adequately modified.
ES-401 CGS-2015-04 18 Form ES-401-9  Attachment 8_Rev 1 Page 18 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S This question is unsat due to incorrect license level. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev2 with adjusted words for D distractor and handout one page of LCS 1.7.1. and is  now sat. References required in Rev 2 of question. 85 H 3        C 43.5  N E 
S Free review - question replaced. Comments on new question are as follows: Is the TS needed as a handout to answer this question?  There are multiple answer choices that differentiate between TS Condition C and Condition A. It is best to have the answer choices reflect the actions that are actually to be taken, not just "enter TS 3.5.1 condition C."  What actions are required per the TS? Licensee improved question, removed the handout, and called it rev1 and is now sat. 86 H 3    X    A 43.2  X N U S Reference needed on draft submittal question Distractor C is not credible. It is not plausible to believe that for a TS basis question the answer will state that everything is fine and no action is required. The other three options have "exceeded EQ" at the start, so it makes distractor C even easier to eliminate.  "No action required" type statements are not good answer choices. Answer choice A and Distractor B - based on "above the line" TS entry requirements which are considered RO level knowledge. Additionally, the SRO-only aspect to the question is a direct lookup with the references provided. The specific page of OI-41 is provided on its own which lists the applicable TS. This question is unsat due to two non-credible distractors. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev2 is now sat. References required in Rev 2 of question. 87 F 2    X X  C 43.2  X N U Distractor A is too close to being considered a potential correct answer. If, as the justification states, the procedure mentions the potential for this alarm to actuate, it can be argued that it is expected. Additionally, it exhibits the "no action required" issue which I have discussed on previous questions. Distractor B is not credible. The TS is not provided as a handout and the LCO time for HPCS is likely common knowledge. It is highly unlikely that an applicant would consider choosing Distractor B. Not SRO only - All you have to know to answer this question is that HPCS is declared inoperable during this surveillance. This is RO level knowledge. Answer C - the answer is written in a way that asks applicants to determine that ES-401 CGS-2015-04 19 Form ES-401-9  Attachment 8_Rev 1 Page 19 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only    S HPCS was declared inoperable. We do not ask applicants to determine what was been done in the past. They should have to make no assumptions to answer the question. Should be rephrased. Again, the correct answer also uses the "no action required" approach. This question is unsat due to incorrect license level. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat. 88 H 3        B 43.5  X N U    S Free review - question replaced. New question is unsatisfactory due to the following: SRO-only:  Question only asks RO related knowledge of RCIC-V-1 design and basic procedure requirements for scramming or shutting down the reactor. This question is unsat due to incorrect license level. NRC wrote new question, references sent were wrong, licensee wrote new question and if justification is improved will be sat as rev1. 89 H 2  X  X    D 43.5  N U  S Cue:  The only choice that discusses performing a risk assessment for maintenance is D. It is too easy to guess the correct procedure based solely on the procedure titles. Distractor A might work as a plausible distractor, but B & C do not. This question is unsat due to cueing. NRC wrote new question, and is sat as rev1. 90 H 3    X    A 43.5  N E    S Distractor C is not credible. It is not plausible for an applicant to decide that the inverter should not be powered up even when the maintenance bypass source is available. Licensee working on distractor C with ABN-FWH as procedure choice as rev 1-will let us know how it validates. Remove transfer to bypass -from answer A and distractor C. Licensee made final changes after telecom on 4/7/2015 and question as rev1 is now sat with these changes. 91 H 4        B 43.5  N E S ADD PER ABN-POWER to distractor B and D. Licensee made changes and question as rev1 is now sat. 92 H 3        B 43.2  N S Reference needed on draft submittal question Free review - question rewritten and is now Sat. References required in Rev 2 of question. 93 F 3    X    C 43.2  N U    Reference needed on draft submittal question First half of question is a direct lookup. It asks if the interlock is required by TS and then provides a handout of the TS which has a list of interlocks that are covered by the spec. Distractors B & D are not credible because of the direct lookup.
ES-401 CGS-2015-04 20 Form ES-401-9  Attachment 8_Rev 1 Page 20 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only  S This question is unsat due to two non-credible distractors. NRC wrote new question  as rev1 and is now sat. 94 F 3        A 43.5  N E  S Validation comments required changes to this for A and C, will resubmit and we will review. Licensee made final changes after telecom on 4/7/2015 and question as rev1 is now sat with these changes. 95 F 4        B 43.7  N S  96 H 2        C 43.3  N E S Reference needed on draft submittal question Need to remove "per section 5.X" from the answer choices and remove the handout. Wrote new question with handout-still not good enough-two distractors were easy to throw out from the reference so licensee going back to look for 50.59 type question. Licensee wrote new question and the question is good but the marked correct answer is incorrect. The correct answer is D not B as indicated. Licensee made corrected changes and is now Sat as rev1. 97 F 3    X    B 43.2  B U 
S Reference needed on draft submittal question Distractor A - using the "no action required" statement is not a plausible distractor. Distractor C is not credible. There is no plausible reason why an applicant would determine the diesel is inoperable. This question is unsat due to two non-credible distractors. Wrote new question and as rev1 is now sat. 98 F 2        C 43.4  X N U    S Free review - feedback was provided that identified the original question as LOD-1. No significant changes were made to the question to increase the level of difficulty. In actuality, the question may have become less challenging. There are more challenging ways to ask an SRO question on liquid releases. An RO is responsible to know what systems are radioactive and who must approves releases. This question is unsat due to incorrect license level. Toss KA since CGS is zero discharge plant and writing new question from new KA. Bank question from Oct 2009 exam with new KA and as rev2 it is now Sat. 99 H 4        A 43.6  N E To "ensure that the OSP is completed" can be interpreted as saying that the functional test is actually performed, which, as stated in the justification, includes a step to check the mode switch is in Refuel. These two distractors need to be clarified in some way to make it clear that the distractor is stating that a previously performed OSP is reviewed, and not necessarily performed again. There is no reason why you can't perform the OSP every 12 hours - while it would not be a good use of resources, it would not be considered incorrect and would meet the requirements to ES-401 CGS-2015-04 21 Form ES-401-9  Attachment 8_Rev 1 Page 21 of 22 OBDI 202 - IOLE Process  Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N  7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S check that the mode switch is in Refuel. Licensee modified question and as rev1 it is now sat. 100 H 3        D 43.5  B S Free review. RO TOTALS: B = 13 F = 35 E = 34 Additional Notes:
M = 4 H = 40 U = 22 N = 58  S = 19 SRO TOTALS: B = 3 F = 7 E = 7 Additional Notes:    M = 0 H = 18 U = 13 N = 22  S = 5 GENERAL COMMENTS 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N 2. Chief Examiner comments are indicated in black. 3. Average difficulty is __2.7    ___  on the RO exam and    3.0 ___      on the SRO exam. 4. The 10CFR55.41/43 distribution is:  RO  /  SRO  41.1 =  13 43.1 = 1  41.2 =  1 43.2 = 5  41.3 =  0 43.3 = 1  41.4 =  2 43.4 = 2  41.5 =  10 43.5 = 15  41.6 =  1 43.6 = 1  41.7 =  40 43.7 = 1  41.8 =  5 ES-401 CGS-2015-04 22 Form ES-401-9  Attachment 8_Rev 1 Page 22 of 22 OBDI 202 - IOLE Process  41.9 =            0  41.10 =          13  41.11 =          0  41.12 =          2  41.13 =          0  41.14 =          0  5. The answer distribution is:  RO  /  SRO A = 17 (23%) /  7 (28%) B = 18 (24%) /  6 (24%) C = 20 (27%) /  6 (24%) D = 20 (27%) /  6 (24%) 6. There are __1__ RO questions with handouts provided and __5___ SRO questions with handouts provided.}}

Revision as of 03:24, 11 June 2018

Columbia Generating Station-2015-04 Draft Written Exam Comments
ML15167A147
Person / Time
Site: Columbia Energy Northwest icon.png
Issue date: 05/05/2015
From: Vincent Gaddy
Operations Branch IV
To:
References
Download: ML15167A147 (22)


Text

ES-401 CGS-2015-04 Written Examination Review Worksheet Form ES-401-9 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified: $ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). $ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc). $ The answer choices are a collection of unrelated true/false statements. $ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. $ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: $ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). $ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). $ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). $ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Attachment 8_Rev 1 Page 1 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only You did not follow several of the general comments that were made during the free review, which included General comments for submittal 1. Justification for why one answer is correct does not automatically make other answers incorrect. 2. Use the NUREG standard worksheet ES-401-9 for exam questions and complete all fields, incl LOD from now on. 3. Always put in field for bank question that it has not been used on an NRC exam if that is the case. 4. Don't use slightly modified as a term - use definitions in NUREG. 5. Use noun names for designators in addition to numerical names per NUREG-1021 6. There are too many handouts for this question and too many questions (12) on the SRO exam that have handouts provided. 7. Due to number of changes, for revision control the final submittal has been ES-401 CGS-2015-04 2 Form ES-401-9 Attachment 8_Rev 1 Page 2 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only designated as Rev 2. All questions have been changed to reflect Rev 2. 1 H 3 X C 41.5 N U S Free review Did NOT use free review comments to fix question. Once you know first part of answer C then you have the answer since it is unique (ie not a balanced 2 X 2) from the others. C is also a subset of D without "only" in the first part of D. Multiple bad distractors make this question unsat. Licensee wrote new question and as rev1needs to beef up discussion on why pulling control power fuses is now sat. 2 H 3 X B 41.5 N U S Not plausible to think reducing flow through heat exchangers causes temperatures to decrease making C and D not credible. Also, this question is GFE knowledge since the question is asking basic heat exchanger knowledge and multiple poor distractors makes the question Unsat. Licensee modified existing question and as rev 1 but it was not in proper format so with merge to rev 2 is now sat. 3 F 2 X C 41.7 N E S Knowing generic purpose of a keep fill pump is GFES and allows one to throw out the distractors "A" and 'B." This makes these two distractors partially not credible. After validation some of the LO's picked this distractor as correct, therefore licensee added this to justification and we accepted as sat. 4 H 3 X C 41.7 N E S Remove "Initially" in the stem since HPCS is normally aligned to the CST. Made edit as recommended and question is now Sat as rev1. 5 H 3 X A 41.7 B E

S Free review Did use free comments on this question. The water leg pump P-3 information at the end of each choice is extra information and is not needed to answer the question and therefore should be removed. It only clouds the choices. Also need to balance the second part of distractor D regarding V-4 and V-12 with a different aspect than distractor C, such as V-4 and V-12 remain closed due to loss of initiation logic power. Made edit as recommended and question is now Sat as rev1. 6 F 2 X C 41.7 N E S Free review Please provide more information in the justification on section of the worksheet for the credibility of distractors "A" and "B" for MC-7A and 8A. Made edit as recommended and question is now Sat as rev1. 7 F 2 D 41.6 N S Free review-made corrections as requested although sent it as Q4 on free review. 8 H 4 X X A 41.7 X M U Free review ES-401 CGS-2015-04 3 Form ES-401-9 Attachment 8_Rev 1 Page 3 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S DID NOT use the free review comments. This is a series of T/F statements, also B and C not credible, and does not meet KA for function and purpose as originally provided in the free review comments, which were "True false question. Loose K/A connection. Purpose and function needs the why controls are the way they are and question asks what (which is function but not purpose)." Multiple bad distractors, T/F statements, and KA mismatch make this question unsat. Licensee wrote new question and as rev1needs to beef up discussion on function and purpose to the two components and is now sat. 9 F 3 X D 41.7 N U S If you know that period must be stable for criticality then you can eliminate the other three choices without knowing any other information. Need to make this a balanced 2 X 2 to fix this condition. You could move one piece of information to the stem to complete the three requirements for this question. This is also not H3 question as written. It is really an F3 question as written since it is one item that is memorized. Multiple bad distractors make this question unsat. Licensee modified question as rev1and needs modification to be site specific so still working on solution. Did not address both pieces of KA (both count rate and period) with rev1 of the question. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. 10 F 3 B 41.7 X N E S Free review Did use the free review comments, however the change you made gives the appearance now of KA mismatch. The two negatives issue was corrected but the KA is on max disagreement not min disagreement. Also, you are tying this to rod block situations. Are there any other issues that occur when there is disagreement between the flow channels? If so, is there is a value which, upon exceeding, the circuit automatically does something, such as the ind. Channel is auto removed from the ckt, or other things)? If so, then this is what the KA is driving you to ask about. Work on justification of numbers. Updated question via e-mail on 4/6/2015 to incorporate these comments and question is now Sat. 11 H 3 D 41.7 B E S Free review Add RCIC before "full" in distractor "B" to match answer "D" Made edit as recommended and question is now Sat as rev1. 12 F 2 B 41.7 N S Free review 13 H 3 C 41.7 N S Free review 14 F 2 D 41.7 N E Free review No discussion on why distractors A and B are credible but not correct. They seem to ES-401 CGS-2015-04 4 Form ES-401-9 Attachment 8_Rev 1 Page 4 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S be correct to me. Is C a made up distractor (ie are there vac bkrs installed below the floor)? If so then it is not allowed. Is this even credible? Do the SRVs even go below the floor anyway? Licensee explained this question to us during validation week, and then added better justifications for distractors and question is now sat. 15 F 4 A 41.7 N S Free review 16 H 4 X A 41.7 N E S Free review Distractor D needs to be edited. A "do nothing" distractor is not allowed on an initial exam. The distractor could be the lag fan starts and is unable to maintain secondary containment. This would test the applicant's knowledge of system design that the SGT is 100% redundant between lead and lag fans. Make corrections that also changed the answer and justification as well as repairing the one poor distractor and is sat as rev1. 17 F 3 X D 41.7 N E S Free review Need to place comma in stem after fire and before direction. 18 H 2 X D 41.7 N E S Free review A is not credible as provided in the free review and was ignored. The question could be repaired with suggestions already given in free review. There are also other ways to address this KA. Stem could also be shortened considerably and still have same meaning. Made suggested changes to A and is now Sat as rev1. 19 F 3 X D 41.5 B E S Used on last NRC Exam Remove "What will be the impact of starting CW-P-1C in the current plant configuration" from the stem. It is not necessary for the question. Made suggested changes to A and is now Sat as rev1. 20 F 3 X X D 41.5 M E

S Partial cueing with inverter in stem and inverter procedure in answer. Better question would have the percent of voltage (such as 95% and MC-7A or 7F and 90% with MC-7A or MC-7F) with a stem like "As voltage is dropping, what is the first value of voltage that a transfer would occur and to what source would it transfer to." Knowing that the alternate is the first choice is GFES for inverters, but knowing that the alternate is MC-7F for this inverter is plant specific. A. 95% and MC-7A B. 95% and MC-7F C. 90% and MC-7A ES-401 CGS-2015-04 5 Form ES-401-9 Attachment 8_Rev 1 Page 5 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only D. 90% and MC-7F Made suggested changes to A and is now Sat as rev1. 21 H 2 X X B 41.7 M E S The spirit of this KA is for meters, dials, alarms in the control room. Are there any meters or dials in the control room for DC electrical parameters? If so then these should be used to match this KA. Also, if the applicant asks if the CRO2 operator is currently holding the switch how do you answer that during the exam admin and not give away the question? This makes C and D not credible if you give away the information during administration of the exam. Fix distractor D to be annunciator is-valid to match other choice. Licensee made necessary changes to question - Rev 2 of question is now SAT. 22 H 3 B 41.7 X M E S Free review Did NOT take the free review comments that the HPCS diesel is not really meant for this system and KA. See system 209002 for HPCS. In that system topic, the HPCS diesel has many KAs for use on the written exam. The Div 3 EDG is not normally credited for SBO either, hence the reason for the comments here. Need to write a question on the Div 1 or 2 EDGs to meet this KA. Wrote question on div 1 EDG (had to resample KA because Div1 /2 EDGs at CGS do not have high crankcase pressure trips) which we reviewed with them via telecom on 4/7/2015 from region and we agreed that the question is now Sat. 23 H 3 C 41.7 X M U S Free review Did NOT take the free review comments which were that this is a "K/A mismatch because there was no transfer between the two modes of manual and auto. Fix verb tense in distracter B. You might ask something like when the EDG is in manual mode, what causes it to shift to auto mode and close the O/P breaker. In this question the EDG is essentially in manual and stays that way." Licensee didn't change anything on this question. KA mismatch makes this question unsat. Licensee wrote new question and is now sat. 24 F 2 A 41.2 N S Spell out abbreviations for final submittal on 4/10/2015. 25 F 2 C 41.7 N S Free review 26 F 2 X C 41.7 N U Free review Made a minor change in an attempt to correct the subset issue but 2 is still a subset of 1. Subsets make the question Unsat, which was how it was marked in the free review as well. Consider this as an alternative: The RPV water level set-point for the trip of an RCC pump at full power is: A. -50 inches NR ES-401 CGS-2015-04 6 Form ES-401-9 Attachment 8_Rev 1 Page 6 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S B. -50 inches WR C. -135 inches NR D. -135 inches WR Or something like that-.use different instruments (FZ, WR, NRC, etc) and the actual set-point so you avoid subsets. Multiple bad distractors make this question unsat. Licensee wrote new question as rev1 and is now sat. 27 H 3 B 41.7 X N U S KA mismatch-the KA is asking for the effect on the rod block monitor for a malfunction of RMCS, in this case RDCS loss of power. This does not cause any effect on the rod block monitor as you indicate in the answer and worksheet discussion. This is not the spirit of the KA-you should try to cause a loss or malfunction that prevents rod blocks from working-.ie no effect is not what is intended for the KA. You also have the wording wrong in distractor D, "enforced" should come after "be." KA mismatch makes this question unsat. New KA selected K3.01 wrote new question as rev 1 and is now sat. 28 H 4 X X A 41.7 N U S B is not credible as written. There is no connection to the blade with any information system so this is not credible in distractor B. For distractor D, it appears to be correct because as long as you don't move it, you don't get a rod block. This makes the question unsat for two correct answers. Licensee modified question as rev 1 and is now sat. 29 H 3 X B 41.7 N U S Second part of answer for reactor power is not part of KA and is also not needed to answer the question as written. It actually helps to eliminate knowledge about water chemistry by eliminating A and C on power knowledge alone, and this is not part of the KA. Just remove the reactor power part of the question and you have a good 2 X 2 on just water chemistry issues. Also, conductivity and pH can be moved into the stem. "Which of the following are indications of this event?" can be removed from the question. Why are the other answers plausible? Licensee is not providing any basis on any of these questions on why distractors are credible but not correct. Multiple bad distractors make this question unsat. Cueing in rev1 of this question. Needs work. Fixed cueing but this is a major revision to the question so should be rev2 now but distractor C is not credible (during telecom on 4/7/2015). Licensee asked to resubmit for review. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. 30 H 3 X A 41.7 N E Free review Licensee corrected items from free review. However, now you have the TIP detector ES-401 CGS-2015-04 7 Form ES-401-9 Attachment 8_Rev 1 Page 7 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S auto stops inserting in all four choices, and per NUREG should be moved to stem. Again, you don't have any explanation on the credibility of the distractors. Why would someone pick the shear valve, for example? Licensee made necessary changes to question - Rev 2 of question is now SAT. 31 H 2 X D 41.7 X N U

S Free review Did NOT follow feedback on free review comments, which was "ED setpoints are well known, allowing A and B responses to be thrown out. A is also a true/false statement. Not reasonable to cool SP and stop RPV injection. For generic questions. This is usually associated with SRO type questions and this question could be viewed as SRO-only, depending on how its read." Answers A and C don't seem credible. A is effectively a subset of B. An operator wouldn't prioritize cooling the wetwell over keeping the core covered making C not plausible. Choosing RPV level and ignoring SP temp is SRO level (deciding on which equipment you can lose and which you have to try to save while deep in EOPs). Multiple bad distractors make this question unsat. Licensee modified question and distractor D needs work to ensure it does not read as a no action required and fix tabs to bullets. 32 H 3 B 41.7 N S 33 F 2 B 41.7 N S Free review 34 H 3 C 41.7 X N U

S Free review Didn't answer the free review comments to the question of do the valves have auto open and close features for turbine load changes? Licensee rewrote question to address the close only portion of the KA, which doesn't meet the KA now that this change was made. Based on the noun name and the high power level answers B and D don't seem credible. It's intuitive that those valves are closed at full power. Also, there's no justification for using 30% power as a discriminatory value. KA mismatch makes this question unsat. With the rev2 version it isnow sat. 35 H 3 X B 41.7 X B E Free review Since the free review did not catch the negatively focused stem it is marked as E instead of U even though it has no correct answer to the stem as written. In the stem eliminate "Which of the following is correct?" and "Based on the above,".

Also, Answer B does not answer the stem of how to reset the RFW pump turbine so ES-401 CGS-2015-04 8 Form ES-401-9 Attachment 8_Rev 1 Page 8 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only

S no correct answer to the question. All it states is a partial T/F statement that you can't reset the RFW turbine with reset on the switch in the stem. Negatively focused stem and answer combos are discouraged in the NUREG. It would be acceptable if you asked what items needed to be done to actually reset the RFW pump turbine on RPV level issues to meet this KA. Another approach would be to ask in the stem what has to be done to open the HP and LP stop valves for this pump. Another even more appropriate question is the actual high level value that causes the scram and rfp trip. What caused the trip and what is the effect (ie what does it take to restore fw to the RPV)? Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. Ensure "minimum" is capitalized in stem of question for final submittal on 4/10/2015.. Licensee made necessary changes to question - Rev 2 of question is now SAT. 36 F 3 X A 41.7 N U S Free review Did not take the free review comments and make recommended changes. Answers C and D seem to be true/false statements that aren't credible since water can't be pumped directly to the destinations. Also, if there are no rad monitors in these lines how it is credible to have them as choices? This makes B not credible as well. Multiple bad distractors make this question unsat. Licensee wrote new question as rev1 and needs work on stem focus to ensure legally defendable.Fixed concerns and is now sat. 37 F 2 X A 41.7 N E

S Free review Changed question based on feedback from free review. Now you have subsets. If you create the following selections it will work: A. Only FP-P-2A B. Only FP-P-2A and FP-P-2B C. Only FP-P-2B and FP-P-1 D. Only FP-P-1 On telecom on 4/7 still had errors not corrected from comments. Remove comma from C and put "Only" in front of choice D for "FP-P-1." Will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 38 H 3 X X A 41.5 X N E Free review The free review did not identify that some choices can be eliminated without knowing information desired because it is not a balanced 2 X 2. Although knowing AOP entry conditions is RO knowledge with the new guidance, it would be better to not have them select the procedure, but use steps from the applicable procedure to combat ES-401 CGS-2015-04 9 Form ES-401-9 Attachment 8_Rev 1 Page 9 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S the condition (Hi Rad). It seems to be minutia that the toilet fan is all that stops for this condition. To meet the safety aspect of hi radiation, at what point does the outside fans stop and recirc fans to the CR start? This is the knowledge that the KA is trying to measure. This one still needs work. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question has proposed answer as B but correct answer is A. we will verify it is corrected on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 39 H 3 X C 41.10 N E S Reference needed on draft submittal question Ensure the printout is in color for the exam and that the % power is not highlighted as shown in this attached picture because this cues the applicant to the correct answer. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question is now Sat. References required in Rev 2 of question. 40 H 3 A 41.10 N S Free review 41 H 3 X D 41.7 X N E S Free review The answer to the question is essentially none of the above. These types of questions are not allowed on an initial exam. Each answer could be paired down to look more consistent with each other. Also, RCIC is not credited as a system necessary to assure plant shutdown. K/A mismatch. It is marked as an E because these problems were missed on the free review. Updated question via e-mail late on 4/7/2015 to incorporate these comments and still needs to remove scram air header from all four choices since it clouds the choices. we will verify it is corrected on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 42 F 2 D 41.5 N S Free review 43 H 3 X D 41.7 N U S Free review Answers A and B don't appear to be credible since an operator would know that only 2 buttons are required. Having answers combinations of A1, B2 or A2, B1 or A1, A2 should be sufficient to give enough credible distractors. Another consideration is what is the minimum number that the procedure requires and if it states that you must push all four then it could be argued that A or B are also correct from a procedure perspective. From an electrical perspective you would be correct that D is the only correct answer. The stem also states that you press four buttons and yet the answer is you press two buttons, this implies there is no correct answer. Multiple bad distractors and no correct answer make this question unsat.

ES-401 CGS-2015-04 10 Form ES-401-9 Attachment 8_Rev 1 Page 10 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Licensee wrote new question and as rev1 and needs work on distractors C and D. Licensee wrote new question and as rev1 is now Sat. 44 H 2 B 41.10 N S Free review 45 F 2 C 41.8 B E S Remove "Which of the following describes the reason for the required reactor scram?" not necessary for the question. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question still non-credible distractor B. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 46 H 2 X X A 41.7 N U S Free review Did NOT fix comments provided during the free review. Even though the justification states otherwise an applicant could argue that HPCS is designed to maintain RPV level because it is a backup system to RCIC. This question may have more than one right answer. Also, why is CRD a credible choice-it is such low flow capacity it could not possibly keep up? This affects the distractor C. More than one correct answer makes this question unsat. Licensee wrote rev1 to question and is now Sat. 47 F 3 X D 41.5 N E S Why is A credible under these circumstances? This is an F2 question not F3 as you have classified it. Updated question via e-mail late on 4/7/2015 to incorporate these comments and question appears to have no correct answer. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 48 H 3 X X C 41.10 N E

Add high in front of drywell in the stem. Distractor D does not answer the stem at all-it has nothing to say about wetwell pressure and is therefore not an acceptable distractor. The perspective is also incorrect on distractor B and answer C. When you ask what will happen to wetwell pressure, you have to direct the answers towards wetwell pressure, not drywell pressure. To fix B and answer C, this is what you need to do: B. When the drywell pressure increases to 0.2psid higher than wetwell pressure, the wetwell to drywell vacuum breakers will open, maintaining wetwell pressure 0.2 to 0.5psig lower than drywell pressure. C. The steam leak will pressurize-.. venting drywell to the wetwell, maintaining wetwell pressure approximately 5 psig lower than drywell pressure. Licensee did not use our comments for a second time-reworking this question and ES-401 CGS-2015-04 11 Form ES-401-9 Attachment 8_Rev 1 Page 11 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S will resubmit prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 49 F 2 X X D 41.7 N U S Free review The stem cues the applicant into looking at a high pressure causing A to be eliminated and B is in the normal range of pressures after a scram. Change the second question in the stem to read "While verifying RPV pressure CRO1 reports that both RRC-P-1A and RRC-P-1B have tripped." This eliminates the cue and makes all of the distractors credible. Change A to a number greater than 1000 psig for consistency and better credibility. Cueing makes this question unsat. Licensee made recommended changes and as rev1 is now Sat. 50 F 3 A 41.7 X N U S KA mismatch-there is no Suppression pool high temperature condition in the stem or answer therefore KA mismatch and question is therefore unsat.if the KA were for temp monitoring conditions for low SP level then you would have a good question. Question is Unsat due to K/A mismatch. Licensee wrote new question as rev1 and is now Sat. 51 F 3 D 41.10 N S 52 F 3 X C 41.8 N E S B is not credible. Use something safety related more along the lines of the other choices in the EOPs. Worksheet missing lots of information. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 53 F 3 X D 41.7 N E S Free review Answer A is not credible. Basic GFE knowledge would allow an operator to reason that if you allowed level to lower that pump damage would eventually occur with the stem condition that the SP suction valve is open. A better way to ask this question is What is the EOP XX.XX vortexing limit for HPCS when taking a suction from the SP? Then use different values from the table but ensure that the EOP specific info is included in the stem to prevent challenges from other procedures, such as SOPs, FSAR, etc. Formatting issues now - Will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 54 F 2 A 41.5 N S 55 H 2 D 41.7 B S Free Review ES-401 CGS-2015-04 12 Form ES-401-9 Attachment 8_Rev 1 Page 12 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 56 H 3 A 43.4 X N U S Reference needed on draft submittal question Free review This was given as feedback as SRO-only during free review. Licensee provided no rebuttal on this comment and it still appears to be SRO only question since nothing was changed on the question and is therefore unsat. Licensee wrote new question as rev1 and is now Sat. 57 F 3 X B 41.8 M U S Free review Did not use the free review comments, did not change the question or add any justification to why distractors C and D are credible. Two non-credible distractors make the question unsat. Licensee repaired the two bad distractors and as rev1 is now Sat. 58 H 4 X A 41.4 N U S Free review Answers B and C are not credible distractors. B doesn't appear to have any safety impact. Just by reading power system stabilizer would cue an applicant that C is incorrect. Two non-credible distractors make this question unsat. Licensee repaired two bad distractors and as rev1 is now Sat. 59 H 4 C 41.10 N E S Free review Need to add without any further operator action to the stem of the question after power increase is stopped to prevent other answers from being argued as correct. Question: Does 24% reactor power ensure that MT is LT 560MWe? Unbalanced distractors as provide in the free review comments. Take MSIVs out of stem and Make the selections like this: A. Reactor SCRAM occurs due to MT trip B. Reactor SCRAM occurs due to MSIV closure C. Reactor remains at power with a slight increase in power D. Reactor remains at power with a slight decrease in power Remove lowering condenser vacuum from entire question since CGS uses backpressure instead of vacuum to remove unnecessary information from stem and change power to 20% in stem and justifications to ensure not challengeable with other loads and demands on the plant. We will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 60 H 3 X D 41.5 B E Free review Answer C is not credible. When is RCIC ever used to control RPV water level during a normal plant startup and it is GFES to know that a turbine pump needs higher pressure steam to run than 75psig? This is also a Fukushima lessons learned event ES-401 CGS-2015-04 13 Form ES-401-9 Attachment 8_Rev 1 Page 13 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S with ED. Need to find a plausible injection source that could be used during startup. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 61 F 2 X B 41.10 B E S No justification why the distractors are credible. Feedwater is already injecting. A doesn't seem credible. How could the cooldown be violated? Why is D credible? Multiple non-credible distractors makes this question unsat. Licensee made justification section for each choice and as rev1 the question has minor edits needed - we will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 62 H 2 X X C 41.7 N U E S The RPS actuation from the mode switch is an automatic action. Operators have guidance that when an automatic action fails then manual action is required. A competent operator would see that ATWS conditions are indicated and take the other RPS actuation to scram. This would mean pushing the manual scram buttons. Since 1 and 2 are ATWS conditions this would force the operator to push the scram buttons. All answers are true which makes this question unsat. Still needs editing but changes agreed to that we suggested. On telecom review on 4/7 we agreed with licensee that this question is good but needs minor edits and we will review on final submittal on 4/10/2015. Licensee submitted rev2 of question and is now Sat. 63 H 3 X B 41.4 N E

S Answers C and D don't seem credible. Are these components located inside the drywell? Need to find other drywell components that could be impacted to meet intent of NUREG credible distractors. A distractor has poor grammar and therefore would not be selected anyway. A seal failure would not occur for a while anyway. This is an H2 not H3 question. This question is Unsat due to two non-credible distractors. Reselect another KA and write different question. Licensee wrote a new question based on a new KA but the stem doesn't state how the fw controller fails so how can you know what the resulting plant response will be? This affects the answer. Also, this question does not address the reason for the response as required by KA therefore KA mismatch now. Went over with licensee via telecom Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Sent new question and it is now Sat. 64 H 3 X C 41.8 M U Free review Why would distractors A and B be credible for the given conditions in the stem? There's no indication that a fire could exist making A and B not credible. Could use other water sources in room to make credible distractors. This would be a modified bank question with changes already made to the original bank question (ie the ES-401 CGS-2015-04 14 Form ES-401-9 Attachment 8_Rev 1 Page 14 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only E S answer and distractors have been changed). This is an H2 not H3 question. This question is Unsat due to two non-credible distractors. Licensee wrote different distractors and now needs to edit for singularity of instruments in all four choices and it will be Sat as rev1 of the question. This was completed during telecom on 4/7 but water level instruments needs editing for alignment in all choices. We will review on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 65 H 4 X D 41.8 N E S Answer A should be changed to some water discharge from LPCS, HPCS, RHR A, or RHR C since all these components are on the same level in the reactor building. Also, A is the only answer that drains to the EDR system making it not as plausible as the others. It appears that RHR A drains to R-1 sump. FYI-This KA is really asking high sump levels cause a ground or ckt malfunction due to high water, not vice-versa. Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 66 F 2 X A 41.10 N U

S Free review During free review comments were made that this was too easy (LOD = 1) and distractors needed work and nothing was done on this question. On appeal an applicant would be able to argue that any of these conditions could be announced. The question is not discriminating enough. Should be announced is too subjective. Distractor C has a typo and would not be picked anyway. This question is Unsat due to LOD-1. Pulled bank question and is now sat. 67 F 2 C 41.10 B S Free review 68 H 2 X B 41.10 N E S Free review For D, change distractor to Insertion of the bundle may continue at a slower rate to prevent exceeding 300 cps. For C, is it plausible to leave a bundle suspended on the hook? If it is then more justification is needed in the justification section of the worksheet. This is an H2 not H3 question. During telecom review on 4/7 noticed that justification does not match the choices above -Licensee to resubmit prior to final submittal on 4/10/2015.Licensee needs to resubmit for our review prior to final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 69 F 2 X B 41.5 N E S Subset issue with minutes. Need to add minimum minutes to the stem or state in the stem as required in the procedure step for this condition. Also, change the SOP to ABN-CRD to make the distractors more plausible.

ES-401 CGS-2015-04 15 Form ES-401-9 Attachment 8_Rev 1 Page 15 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only Licensee made changes for telecom on 4/7 and question is now sat. 70 F 2 A 41.10 B E S Not an F3 but F2 question. Licensee made changes and question is now Sat. 71 F 3 X D 41.7 N E S Remove "Operation in this range-" from all four choices. During telecom on 4/7 we commented on the need to put "range" at end of answer D to keep all choices balanced and to have no cueing. We will check on final submittal on 4/10/2015. Licensee made necessary changes to question - Rev 2 of question is now SAT. 72 H 3 C 41.12 N S Free review 73 F 3 D 41.12 N S Free review 74 F 3 X C 41.10 X B S Free review Did not use the free review comments which were the following: Borderline SRO question. This is a tier 3 Generic K/A and the question is too specific for this Tier. You should reject this KA and reselect." We should have given better KA for this question so calling the new question sat and original that was unsat was reclassified as sat due to our providing a poor KA to the licensee. Question is unsat for incorrect license level. Question is Sat with new KA, 2.4.12. 75 H 3 A 41.10 B S Free review 76 H 3 X D 43.5 N U S Reference needed on draft submittal question This was a free review question. The stem was changed after the free review which now gives away the Mode 3 piece and therefore makes distracter A not credible. Also, you changed the correct answer without changing the stem meaning so what happened there? Also, verbatim copying of the TS statements is not recommended per the NUREG, especially if you give them all or most of the references needed to answer the question. Requiring the CRS to know a 2 hr TS from memory or details in the bases document on the DC system would be better than handing out 4 different TS statements. Also, without the B2 TS statement from 3.8.4 in the Proposed answer "D" it makes no answer correct anyway and the question is therefore unsat. There are too many handouts for this question and too many questions (12) on the SRO exam that have handouts provided. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat.

ES-401 CGS-2015-04 16 Form ES-401-9 Attachment 8_Rev 1 Page 16 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 77 H 2 B 43.5 X N U S Free review - question replaced. New question is unsatisfactory for the following reasons: Not an SRO only question. Question requires major EOP mitigating steps and systems knowledge at the RO level. It does not become an SRO question by adding "what direction should the CRS give?" There needs to be a procedural transition requiring the SRO to analyze the situation and decide on the correct sub-procedure or appendix that must be used. This question is unsat due to incorrect license level. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat. 78 H 3 X X C 43.5 N E S Reference needed on draft submittal question Free review - question replaced. Comments on new questions are as follows: There is no need for the first of the two questions. The same information is required (# of available bottles for ADS to be considered operable) to answer both questions. Absolute terms such as "ALL" act as cues that the answer choice is incorrect.

Additionally, due to the change in verb tense from the other answer choices, Distractor D could be read in a way where it is considered a subset of the correct answer - replace four bottles, and then exit TS 3.5.1. If four bottles are replaced, you can certainly exit the TS. Can the question be asked, "per the TS basis, what is the minimum number of bottles that must be replaced to return ADS to operable status?" Licensee made changes and question as rev1 is now sat. 79 F 4 X D 43.5 X N U

S Reference needed on draft submittal question Free review - question rewritten. New question is unsatisfactory for the following reasons: K/A mismatch - the question does not constitute a "refueling accident." Distractors A & B not credible - it is not plausible to believe that an applicant will believe that refueling is being conducted with the unit in Mode 2 or 4. Also, Distractor B is a 1-hr action statement, so SRO applicants will know that this is not the answer as it is RO knowledge. Distractor C - use of "no action required" type statements are not effective as correct answers or distractors as it gives the appearance of a trick question. This question is unsat due to K/A mismatch. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat. 80 H 3 X X X A 43.1 B E Reference needed on draft submittal question Distractor C - Choices that contain qualifying statements like "as long as" are poor ES-401 CGS-2015-04 17 Form ES-401-9 Attachment 8_Rev 1 Page 17 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S distractors. They are very rarely the correct answer and therefore cues the applicant. Distractor D - could be considered a correct answer. Need to add "without a risk assessment being performed" or something to this effect. Also, need to say 24 hours2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> "from now" or somehow clarify what starting point you are talking about. Licensee made changes and question as rev2 is now sat. References required in Rev 2 of question. 81 H 4 A 43.5 N U S Reference needed on draft submittal question Free review - question rewritten. New question is unsatisfactory for the following reasons: Direct look-up with the references provided. Handouts should be removed or the question should be replaced. It is reasonable to ask SROs about standard PARs without giving them the flowchart. This question is unsat due to LOD-1. NRC submitted possible new question and licensee didn't like it so they are writing to GE rad levels and call it rev2 and it is now sat. 82 H 4 X X D 43.5 N U

S Reference needed on draft submittal question Distractor C is another correct answer as the questions asks "what actions are required?" It is required that 3.8.1 A&B are entered and that a source be restored within 72 hrs, even though there is another requirement to restore DG-3 within 12 hrs. Distractor B is not credible. It is not plausible to think that an applicant would not enter Condition A - they are told in the stem specifically that offsite power cannot be restored within TS limits. This question is unsat due to two non-credible distractors. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat.rev2 with 3800 and minor edits resubmit as sat. References required in Rev 2 of question. 83 H 3 B 43.5 N S Free review - question replaced and is now Sat. 84 H 4 D 43.5 X M U Reference needed on draft submittal question Not SRO only - 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or less TS required actions are RO knowledge. Distractors A & B are not credible. There is no plausible reason why an applicant would choose these distractors. The question is essentially a direct look-up. Need parent question to determine if question has been adequately modified.

ES-401 CGS-2015-04 18 Form ES-401-9 Attachment 8_Rev 1 Page 18 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S This question is unsat due to incorrect license level. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev2 with adjusted words for D distractor and handout one page of LCS 1.7.1. and is now sat. References required in Rev 2 of question. 85 H 3 C 43.5 N E

S Free review - question replaced. Comments on new question are as follows: Is the TS needed as a handout to answer this question? There are multiple answer choices that differentiate between TS Condition C and Condition A. It is best to have the answer choices reflect the actions that are actually to be taken, not just "enter TS 3.5.1 condition C." What actions are required per the TS? Licensee improved question, removed the handout, and called it rev1 and is now sat. 86 H 3 X A 43.2 X N U S Reference needed on draft submittal question Distractor C is not credible. It is not plausible to believe that for a TS basis question the answer will state that everything is fine and no action is required. The other three options have "exceeded EQ" at the start, so it makes distractor C even easier to eliminate. "No action required" type statements are not good answer choices. Answer choice A and Distractor B - based on "above the line" TS entry requirements which are considered RO level knowledge. Additionally, the SRO-only aspect to the question is a direct lookup with the references provided. The specific page of OI-41 is provided on its own which lists the applicable TS. This question is unsat due to two non-credible distractors. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev2 is now sat. References required in Rev 2 of question. 87 F 2 X X C 43.2 X N U Distractor A is too close to being considered a potential correct answer. If, as the justification states, the procedure mentions the potential for this alarm to actuate, it can be argued that it is expected. Additionally, it exhibits the "no action required" issue which I have discussed on previous questions. Distractor B is not credible. The TS is not provided as a handout and the LCO time for HPCS is likely common knowledge. It is highly unlikely that an applicant would consider choosing Distractor B. Not SRO only - All you have to know to answer this question is that HPCS is declared inoperable during this surveillance. This is RO level knowledge. Answer C - the answer is written in a way that asks applicants to determine that ES-401 CGS-2015-04 19 Form ES-401-9 Attachment 8_Rev 1 Page 19 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S HPCS was declared inoperable. We do not ask applicants to determine what was been done in the past. They should have to make no assumptions to answer the question. Should be rephrased. Again, the correct answer also uses the "no action required" approach. This question is unsat due to incorrect license level. NRC submitted possible new question, Licensee modified it for site specific aspects with no reference and as rev1 is now sat. 88 H 3 B 43.5 X N U S Free review - question replaced. New question is unsatisfactory due to the following: SRO-only: Question only asks RO related knowledge of RCIC-V-1 design and basic procedure requirements for scramming or shutting down the reactor. This question is unsat due to incorrect license level. NRC wrote new question, references sent were wrong, licensee wrote new question and if justification is improved will be sat as rev1. 89 H 2 X X D 43.5 N U S Cue: The only choice that discusses performing a risk assessment for maintenance is D. It is too easy to guess the correct procedure based solely on the procedure titles. Distractor A might work as a plausible distractor, but B & C do not. This question is unsat due to cueing. NRC wrote new question, and is sat as rev1. 90 H 3 X A 43.5 N E S Distractor C is not credible. It is not plausible for an applicant to decide that the inverter should not be powered up even when the maintenance bypass source is available. Licensee working on distractor C with ABN-FWH as procedure choice as rev 1-will let us know how it validates. Remove transfer to bypass -from answer A and distractor C. Licensee made final changes after telecom on 4/7/2015 and question as rev1 is now sat with these changes. 91 H 4 B 43.5 N E S ADD PER ABN-POWER to distractor B and D. Licensee made changes and question as rev1 is now sat. 92 H 3 B 43.2 N S Reference needed on draft submittal question Free review - question rewritten and is now Sat. References required in Rev 2 of question. 93 F 3 X C 43.2 N U Reference needed on draft submittal question First half of question is a direct lookup. It asks if the interlock is required by TS and then provides a handout of the TS which has a list of interlocks that are covered by the spec. Distractors B & D are not credible because of the direct lookup.

ES-401 CGS-2015-04 20 Form ES-401-9 Attachment 8_Rev 1 Page 20 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S This question is unsat due to two non-credible distractors. NRC wrote new question as rev1 and is now sat. 94 F 3 A 43.5 N E S Validation comments required changes to this for A and C, will resubmit and we will review. Licensee made final changes after telecom on 4/7/2015 and question as rev1 is now sat with these changes. 95 F 4 B 43.7 N S 96 H 2 C 43.3 N E S Reference needed on draft submittal question Need to remove "per section 5.X" from the answer choices and remove the handout. Wrote new question with handout-still not good enough-two distractors were easy to throw out from the reference so licensee going back to look for 50.59 type question. Licensee wrote new question and the question is good but the marked correct answer is incorrect. The correct answer is D not B as indicated. Licensee made corrected changes and is now Sat as rev1. 97 F 3 X B 43.2 B U

S Reference needed on draft submittal question Distractor A - using the "no action required" statement is not a plausible distractor. Distractor C is not credible. There is no plausible reason why an applicant would determine the diesel is inoperable. This question is unsat due to two non-credible distractors. Wrote new question and as rev1 is now sat. 98 F 2 C 43.4 X N U S Free review - feedback was provided that identified the original question as LOD-1. No significant changes were made to the question to increase the level of difficulty. In actuality, the question may have become less challenging. There are more challenging ways to ask an SRO question on liquid releases. An RO is responsible to know what systems are radioactive and who must approves releases. This question is unsat due to incorrect license level. Toss KA since CGS is zero discharge plant and writing new question from new KA. Bank question from Oct 2009 exam with new KA and as rev2 it is now Sat. 99 H 4 A 43.6 N E To "ensure that the OSP is completed" can be interpreted as saying that the functional test is actually performed, which, as stated in the justification, includes a step to check the mode switch is in Refuel. These two distractors need to be clarified in some way to make it clear that the distractor is stating that a previously performed OSP is reviewed, and not necessarily performed again. There is no reason why you can't perform the OSP every 12 hours1.388889e-4 days <br />0.00333 hours <br />1.984127e-5 weeks <br />4.566e-6 months <br /> - while it would not be a good use of resources, it would not be considered incorrect and would meet the requirements to ES-401 CGS-2015-04 21 Form ES-401-9 Attachment 8_Rev 1 Page 21 of 22 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only S check that the mode switch is in Refuel. Licensee modified question and as rev1 it is now sat. 100 H 3 D 43.5 B S Free review. RO TOTALS: B = 13 F = 35 E = 34 Additional Notes:

M = 4 H = 40 U = 22 N = 58 S = 19 SRO TOTALS: B = 3 F = 7 E = 7 Additional Notes: M = 0 H = 18 U = 13 N = 22 S = 5 GENERAL COMMENTS 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N 2. Chief Examiner comments are indicated in black. 3. Average difficulty is __2.7 ___ on the RO exam and 3.0 ___ on the SRO exam. 4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 13 43.1 = 1 41.2 = 1 43.2 = 5 41.3 = 0 43.3 = 1 41.4 = 2 43.4 = 2 41.5 = 10 43.5 = 15 41.6 = 1 43.6 = 1 41.7 = 40 43.7 = 1 41.8 = 5 ES-401 CGS-2015-04 22 Form ES-401-9 Attachment 8_Rev 1 Page 22 of 22 OBDI 202 - IOLE Process 41.9 = 0 41.10 = 13 41.11 = 0 41.12 = 2 41.13 = 0 41.14 = 0 5. The answer distribution is: RO / SRO A = 17 (23%) / 7 (28%) B = 18 (24%) / 6 (24%) C = 20 (27%) / 6 (24%) D = 20 (27%) / 6 (24%) 6. There are __1__ RO questions with handouts provided and __5___ SRO questions with handouts provided.