ML082830197
| ML082830197 | |
| Person / Time | |
|---|---|
| Site: | San Onofre  |
| Issue date: | 09/26/2008 |
| From: | Garchow S M Operations Branch IV |
| To: | Southern California Edison Co |
| References | |
| Download: ML082830197 (9) | |
Text
ES-401 SO-2008-09 Written Examination Review Worksheet Form ES-401-9 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified:
$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
$ The answer choices are a collection of unrelated true/false statements.
$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:
$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
$ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer
=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any A U@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Attachment 8 Page 1 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO Only1 H 3 B S (41.10) 2 F 2 B S (41.10) 3 H 2 M S (41.5) 4 F 3 B E S (41.10) 'A' needs to be reworded so there is no confusion that the two valve operations are sequential, not simultaneous. 'B' and 'C' are too wordy. Agree - This question was revised to address the comments above. 5 F 3 N S (41.7) 6 F 3 B S (41.7) 7 H 3 N S (41.3) 8 H 2 U B U S (41.7) This question matches K4.01, automatic start of standby pump. This question does not ask the applicant for any knowledge of the power supply to CCW.
ES-401 SO-2008-09 2Form ES-401-9 Attachment 8 Page 2 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO OnlyAfter discussing with the exam writer this question was determined to meet the KA. 9 H 3 N E S (41.5) 'D' needs to be reworded to match the stem sentence flow. D was revised to address the comment above. 10 H 3 B S (41.5) Reference step 3.b RNO, not attached. The reference was provided. 11 H 3 B S (41.7) 12 F 2 B S (C41.7) 13 H 2 B S (41.7) 14 F 2 B S (41.7) 15 H 2 N S (41.7) 16 F 3 B S (B41.7) 17 H 3 B S (C41.4) 18 H 3 B E S (A41.4) Reword the stem. Confusing as written. The stem was rewritten. 19 F 2 N S (A41.4) 20 H 3 B S (C41.10) 21 H 3 B S (D41.7) 22 H 2 N S (B41.7) 23 F 2 U B U S (D41.7) Need to tie in cause/effect to the EDG. How will this failure affect the EDG system? Agree - This question was rewritten. 24 H 3 B S (C41.11) 25 F 2 N S (D41.8) 26 F 2 N S (A41.8) 27 F 3 U N U S (C41.7) Reword / rewrite this question. The answers do not discriminate against what the stem is asking. This question is ok as written.
ES-401 SO-2008-09 3Form ES-401-9 Attachment 8 Page 3 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO Only28 H 3 N E S (B41.7) Reword item 2. so that the stem flows better. This question was rewritten. 29 F 2 B E S (C41.10) Answers contradict the procedure step ordering. The steps in the answer were reordered. 30 H 3 N E S (D41.10) Reword item 2 in stem. Answers seem very similar to Common 2 in required knowledge. Reword answers to match tense in stem. This question is ok as written. 31 F 2 U N U S (A41.6) True/False. Rewrite. Agree - This question was rewritten. 32 F 2 B E S (A41.10) Remove everything above "Which ONE-" Not needed for the answer. This question was revised to address the comment above. 33 H 3 B S (B41.4) 34 H 2 N S (A41.7) 35 H 3 B S (D41.10) 36 F 2 U B U S There is no tie to the CFR for this question. Agree, there is no nuclear safety significance to this question. - This question was rewritten. 37 H 3 B S (A41.4) 38 F 2 U B U (D41.10) Does not ask about operational implications on the Waste Gas Holdup Agree - This question was rewritten. 39 H 2 B S (B41.5) 40 F 2 E B E S (D41.7) Change 'D' so that the sentence is shorter. "Cooling from injection flow-" D was rewritten. 41 F 2 U N U S (D41.7) This does not focus on the reason for the EDG start and no loading. It just asks what the normal preferred source is. Agree - This question was rewritten. 42 F 2 B U S (B41.10) No technical justification for the correct answer. This question is ok as written if procedural guidance is provided supporting the correct answer. - This question was rewritten.
ES-401 SO-2008-09 4Form ES-401-9 Attachment 8 Page 4 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO Only43 H 3 N E S (A41.7) Give a pressure for the Suction Bladder in the stem. The stem was revised to address the comment above. 44 F 2 N S (B41.7) 45 H 3 U B U S (D41.10) Need to elicit a reason from the applicant. Agree - This question was revised. 46 H 2 B S (B41.7) 47 F 2 E B E S (C41.6) 'D' can be eliminated since an assembly would not be opened. If it were changed to de-energized, then it too would be a correct answer unless there is no doubt that the stem is asking for a direct effect. D was revised. 48 F 2 U U N U S (A41.5) The way this question is worded, and depending on what the applicant assumes (large, small, will I look at both steam and feed to determine the mismatch, etc), every answer could be correct. No technical justification is provided. I think this question is ok, but we will talk about this one. - This question was revised. 49 H 2 B S (A41.7) 50 H 3 B S (C41.10) 51 H 3 N S (A41.7) 52 H 3 E B E S (D41.7) Need to ensure that the applicant knows that the two listed alarms are the only ones lit; otherwise, 'A' could be correct. No other annunciators was added to the stem. 53 H 2 B S (D41.7) 54 H 2 B U/E E S (B41.10) Remove the Boost / Buck from the answers; otherwise, this is a direct lookup. Would prefer parameters are provided rather than "above and to the right of the curve" and remove the boost / buck from the distractors. - The question was revised to address these comments. 55 H 2 B U/E E S (B41.10) Change first bullet to "ESDE has been diagnosed-" As written, the applicant could assume no action has been taken. Verify with licensee that value for ESDE subcooling is between 80 and 160, and the applicant should know from conditions given that subcooling is >160. Agree - The comment above was addressed.
ES-401 SO-2008-09 5Form ES-401-9 Attachment 8 Page 5 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO Only56 F 2 N S (B41.10) 57 H 3 N U E S (A41.10) Technical reference does not justify the answer given, or the subject of the question. In order to prevent pellet/clad interaction, rods should be moved at
<3in/min. This does not support the question: "Why are PLCEAs returned to the pre-CRW position?" Provide a technical reference for the correct answer. - This question was revised to address the comments. 58 F 2 N S (D41.6) 59 H 2 N S (A41.5) 60 H 3 N U S (B41.5) Inadequate technical justification provided. This question is ok as written. 61 H 3 U N U S (B41.13) Backward logic as written. Agree, this question is not operationally valid. - This question was revised. 62 F 3 U B U S (C43.2) As written, this is a Tech Spec question. Although the RO may be required to know LCO's, the applicant must know 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> tech spec action statements to answer this question, which is not required per regulations. Revise or replace. Agree - This question was revised. 63 H 3 N U/E S (B41.10) With both QSPDS and CFMS inoperable, REP CET is not available. Where is the technical justification for assuming CET values. Based on the actions required in SO23-12-10, CSF Status Checks, SF-3, RCS Heat Removal, Step 5, the transition to FR-5 on CET >700F could occur while in the LOCA EOI, not the SBO EOI as was provided. Are the applicants to assume a SBO? This question was modified to address the comments above. 64 F 2 U B U S (D41.10) Recalling the source for refilling the purif. filter is not related to corrective actions for high RCS activity. Agree - After discussing with the exam author, this question is ok as written. 65 H 3 M S (B41.10) 66 F 2 N E S (B41.10) Change conditions to condition. Change Starting a to starting. Comments were addressed. 67 F 2 B S (D41.10) 68 F 2 B E (A41.10) Edit 'D' to match form of other distractors.
ES-401 SO-2008-09 6Form ES-401-9 Attachment 8 Page 6 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO OnlyS D was edited. 69 F 2 B E S (D41.10) Edit the stem so that the question makes sense. The question was revised to address the comment above. 70 F 2 B S (B41.12) 71 H 3 B S (B41.11) 72 F 2 N S (B41.11) 73 F 2 N S (B41.10) 74 F 2 N E S (D41.10) Change conditions to condition. This comment was addressed. 75 F 2 B S (B41.10) 76 H 3 N S (A43.5) 77 H 3 N E S (A43.5) Should 'D' be 23 inches or 23 feet? D was corrected. 78 H 2 B S (C43.5) 79 H 3 U N U S (C43.5) Directing activities does not meet the intent of the regulations. Greater than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> TS actions are required w/o references. This is considered unsat because we do not expect candidates to have TS LCO's memorized unless 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or less. This question was revised. 80 H 2 N S (A43.5) 81 H 3 N E S (A43.5) What is the current procedure in effect? Assume SPTA. Are operators expected to activate MSIS ahead of the procedural requirement of 740 psia (step8.c)? After followup discussions, this question was determined to be ok as written. 82 H 2 N E S (A43.5) Change Direct performance of to Enter in 'A' and 'C'. This comment was addressed. 83 H 2 N S (D43.5) 84 H 3 B S (C43.5)
ES-401 SO-2008-09 7Form ES-401-9 Attachment 8 Page 7 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO Only85 H 4 N E S (B43.5) Verify with licensee that this level of detail is required without a reference. This question was revised. 86 H 3 U N U S (A43.5) SRO needs to select procedures. RO knowledge. Agree - This question was revised. 87 H 3 U N U S (A43.5) No reference given to applicant (which would make this partially a direct lookup), and >1 hour TS actions recalled from memory. Agree - same comment as for Q 79 This question was revised. 88 F 3 U N U S (A41.13) RO question. Direct from Limits and Specifications of SO23-8-7, Liquid Rad Waste Disagree this is an RO question but same comment as for Q 79.
After further discussion with the exam author, this question is ok as written. 89 F 2 N E S (C43.1) Reference the procedures in the stem. This comment was addressed. 90 H 3 U N U S (B41.10) As written, this is an RO question. Agree - This question was revised. 91 H 2 N E S (C43.5) Answers 'A' should state: "Enter SO23-3-2.22- to RESET SIAS-" Answer 'C' should state: "Concurrently initiate FS-33-" These comments were addressed. 92 H 3 N E S (D43.7) Answers or item 1.) in stem need to be revised so the answers match the question. This question was revised. 93 H 3 U N U S (A43.2) Tech Spec action statements > 1hour, no reference Agree - This question was rewritten. 94 H 3 U M U S (B43.2) Tech Spec action statements > 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, no reference Agree - This question was revised to provide a reference. 95 F 2 M S (C43.7) 96 F 2 B S (C43.3) 97 F 2 U B U S (B41.10) The term AVAILABLE as supplied for context of this question is defined in the HPSI operating procedure. RO knowledge.
ES-401 SO-2008-09 8Form ES-401-9 Attachment 8 Page 8 of 9 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
B/M/N 7. U/E/S8. Explanation StemFocusCues T/F Cred. Dist. Partial Job-LinkMinutia#/ unitsBack-ward Q= K/ASRO OnlyAgree - This question was revised. 98 F 2 B E S (D43.3) Include the TS number in the Stem. Verify this with licensee. This comment was addressed. 99 F 2 U B U S (C41.10) This is direct from the AOI. RO knowledge. Agree - After discussion with the exam author, this question is ok as written. 100 H 3 U N U S (B41.10) RO knowledge. This question is ok as written. RO TOTALS: B= 44 F=36 E= Additional Notes: M= 2 H=39 U= N= 29 SRO TOTALS: B= 6 F=7 E= Additional Notes: M= 2 H=18 U= N= 17
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.
- 2. Chief Examiner comments are indicated in blue. 3. Average difficulty is __ 2.4___ on the RO exam and _2.6__ on the SRO exam.
- 4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 43.1 = 41.2 = 43.2 = 41.3 = 43.3 = 41.4 = 43.4 =
41.5 = 43.5 = 41.6 = 43.6 = 41.7 = 43.7 =
41.8 = 41.9 = 41.10 =
41.11 = 41.12 =
ES-401 SO-2008-09 9Form ES-401-9 Attachment 8 Page 9 of 9 OBDI 202 - IOLE Process 41.13 = 41.14 =
- 6. There are questions with attachments provided.