ML082830197
| ML082830197 | |
| Person / Time | |
|---|---|
| Site: | San Onofre  |
| Issue date: | 09/26/2008 |
| From: | Garchow S Operations Branch IV |
| To: | Southern California Edison Co |
| References | |
| Download: ML082830197 (9) | |
Text
ES-401 SO-2008-09 Written Examination Review Worksheet Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 1 H 3 B S (41.10) 2 F 2 B S (41.10) 3 H 2 M S (41.5)
(41.10) A needs to be reworded so there is no confusion that the two valve E
4 F 3 B operations are sequential, not simultaneous. B and C are too wordy.
S Agree - This question was revised to address the comments above.
5 F 3 N S (41.7) 6 F 3 B S (41.7) 7 H 3 N S (41.3)
U (41.7) This question matches K4.01, automatic start of standby pump. This question 8 H 2 U B S does not ask the applicant for any knowledge of the power supply to CCW.
Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
- 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
- 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
- 3. Check the appropriate box if a psychometric flaw is identified:
$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
$ The answer choices are a collection of unrelated true/false statements.
$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
- 4. Check the appropriate box if a job content error is identified:
$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
$ The question requires reverse logic or application compared to the job requirements.
- 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
- 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
- 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
- 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 2 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only After discussing with the exam writer this question was determined to meet the KA.
E (41.5) D needs to be reworded to match the stem sentence flow.
9 H 3 N S D was revised to address the comment above.
(41.5) Reference step 3.b RNO, not attached.
10 H 3 B S The reference was provided.
11 H 3 B S (41.7) 12 F 2 B S (C41.7) 13 H 2 B S (41.7) 14 F 2 B S (41.7) 15 H 2 N S (41.7) 16 F 3 B S (B41.7) 17 H 3 B S (C41.4)
E (A41.4) Reword the stem. Confusing as written.
18 H 3 B S The stem was rewritten.
19 F 2 N S (A41.4) 20 H 3 B S (C41.10) 21 H 3 B S (D41.7) 22 H 2 N S (B41.7)
(D41.7) Need to tie in cause/effect to the EDG. How will this failure affect the EDG U
23 F 2 U B system?
S Agree - This question was rewritten.
24 H 3 B S (C41.11) 25 F 2 N S (D41.8) 26 F 2 N S (A41.8)
(C41.7) Reword / rewrite this question. The answers do not discriminate against U
27 F 3 U N what the stem is asking.
S This question is ok as written. Page 2 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 3 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E (B41.7) Reword item 2. so that the stem flows better.
28 H 3 N S This question was rewritten.
E (C41.10) Answers contradict the procedure step ordering.
29 F 2 B S The steps in the answer were reordered.
(D41.10) Reword item 2 in stem. Answers seem very similar to Common 2 in E
30 H 3 N required knowledge. Reword answers to match tense in stem.
S This question is ok as written.
U (A41.6) True/False. Rewrite.
31 F 2 U N S Agree - This question was rewritten.
E (A41.10) Remove everything above Which ONE Not needed for the answer.
32 F 2 B S This question was revised to address the comment above.
33 H 3 B S (B41.4) 34 H 2 N S (A41.7) 35 H 3 B S (D41.10)
There is no tie to the CFR for this question.
U 36 F 2 U B Agree, there is no nuclear safety significance to this question. - This question was S
rewritten.
37 H 3 B S (A41.4)
(D41.10) Does not ask about operational implications on the Waste Gas Holdup 38 F 2 U B U Agree - This question was rewritten.
39 H 2 B S (B41.5)
E (D41.7) Change D so that the sentence is shorter. Cooling from injection flow 40 F 2 E B S D was rewritten.
(D41.7) This does not focus on the reason for the EDG start and no loading. It just U
41 F 2 U N asks what the normal preferred source is.
S Agree - This question was rewritten.
(B41.10) No technical justification for the correct answer.
U 42 F 2 B This question is ok as written if procedural guidance is provided supporting the S
correct answer. - This question was rewritten. Page 3 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 4 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E (A41.7) Give a pressure for the Suction Bladder in the stem.
43 H 3 N S The stem was revised to address the comment above.
44 F 2 N S (B41.7)
U (D41.10) Need to elicit a reason from the applicant.
45 H 3 U B S Agree - This question was revised.
46 H 2 B S (B41.7)
(C41.6) D can be eliminated since an assembly would not be opened. If it were E changed to de-energized, then it too would be a correct answer unless there is no 47 F 2 E B doubt that the stem is asking for a direct effect.
S D was revised.
(A41.5) The way this question is worded, and depending on what the applicant U assumes (large, small, will I look at both steam and feed to determine the mismatch, 48 F 2 U U N etc), every answer could be correct. No technical justification is provided.
S I think this question is ok, but we will talk about this one. - This question was revised.
49 H 2 B S (A41.7) 50 H 3 B S (C41.10) 51 H 3 N S (A41.7)
(D41.7) Need to ensure that the applicant knows that the two listed alarms are the E
52 H 3 E B only ones lit; otherwise, A could be correct.
S No other annunciators was added to the stem.
53 H 2 B S (D41.7)
(B41.10) Remove the Boost / Buck from the answers; otherwise, this is a direct U/E lookup.
54 H 2 B E Would prefer parameters are provided rather than above and to the right of the S curve and remove the boost / buck from the distractors. - The question was revised to address these comments.
(B41.10) Change first bullet to ESDE has been diagnosed As written, the U/E applicant could assume no action has been taken. Verify with licensee that 55 H 2 B E Attachment 5 value for ESDE subcooling is between 80 and 160, and the applicant should know from conditions given that subcooling is >160.
S Agree - The comment above was addressed. Page 4 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 5 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 56 F 2 N S (B41.10)
(A41.10) Technical reference does not justify the answer given, or the subject of the U question. In order to prevent pellet/clad interaction, rods should be moved at
<3in/min. This does not support the question: Why are PLCEAs returned to the pre-57 H 3 N E CRW position?
S Provide a technical reference for the correct answer. - This question was revised to address the comments.
58 F 2 N S (D41.6) 59 H 2 N S (A41.5)
U (B41.5) Inadequate technical justification provided.
60 H 3 N S This question is ok as written.
U (B41.13) Backward logic as written.
61 H 3 U N S Agree, this question is not operationally valid. - This question was revised.
(C43.2) As written, this is a Tech Spec question. Although the RO may be required U to know LCOs, the applicant must know 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> tech spec action statements to 62 F 3 U B answer this question, which is not required per regulations. Revise or replace.
S Agree - This question was revised.
(B41.10) With both QSPDS and CFMS inoperable, REP CET is not available.
Where is the technical justification for assuming CET values. Based on the actions U/E required in SO23-12-10, CSF Status Checks, SF-3, RCS Heat Removal, Step 5, the 63 H 3 N transition to FR-5 on CET >700F could occur while in the LOCA EOI, not the SBO S
EOI as was provided. Are the applicants to assume a SBO?
This question was modified to address the comments above.
(D41.10) Recalling the source for refilling the purif. filter is not related to corrective U
64 F 2 U B actions for high RCS activity.
S Agree - After discussing with the exam author, this question is ok as written.
65 H 3 M S (B41.10)
E (B41.10) Change conditions to condition. Change Starting a to starting.
66 F 2 N S Comments were addressed.
67 F 2 B S (D41.10) 68 F 2 B E (A41.10) Edit D to match form of other distractors. Page 5 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 6 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only S D was edited.
E (D41.10) Edit the stem so that the question makes sense.
69 F 2 B S The question was revised to address the comment above.
70 F 2 B S (B41.12) 71 H 3 B S (B41.11) 72 F 2 N S (B41.11) 73 F 2 N S (B41.10)
E (D41.10) Change conditions to condition.
74 F 2 N S This comment was addressed.
75 F 2 B S (B41.10) 76 H 3 N S (A43.5)
E (A43.5) Should D be 23 inches or 23 feet?
77 H 3 N S D was corrected.
78 H 2 B S (C43.5)
(C43.5) Directing activities does not meet the intent of the regulations. Greater than 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> TS actions are required w/o references.
U 79 H 3 U N This is considered unsat because we do not expect candidates to have TS LCOs S memorized unless 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br /> or less.
This question was revised.
80 H 2 N S (A43.5)
(A43.5) What is the current procedure in effect? Assume SPTA. Are operators E expected to activate MSIS ahead of the procedural requirement of 740 psia 81 H 3 N (step8.c)?
S After followup discussions, this question was determined to be ok as written.
E (A43.5) Change Direct performance of to Enter in A and C.
82 H 2 N S This comment was addressed.
83 H 2 N S (D43.5) 84 H 3 B S (C43.5) Page 6 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 7 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E (B43.5) Verify with licensee that this level of detail is required without a reference.
85 H 4 N S This question was revised.
U (A43.5) SRO needs to select procedures. RO knowledge.
86 H 3 U N S Agree - This question was revised.
(A43.5) No reference given to applicant (which would make this partially a direct U lookup), and >1 hour TS actions recalled from memory.
87 H 3 U N S Agree - same comment as for Q 79 This question was revised.
(A41.13) RO question. Direct from Limits and Specifications of SO23-8-7, Liquid U Rad Waste 88 F 3 U N S Disagree this is an RO question but same comment as for Q 79.
After further discussion with the exam author, this question is ok as written.
E (C43.1) Reference the procedures in the stem.
89 F 2 N S This comment was addressed.
U (B41.10) As written, this is an RO question.
90 H 3 U N S Agree - This question was revised.
(C43.5) Answers A should state: Enter SO23-3-2.22 to RESET SIAS Answer E
91 H 2 N C should state: Concurrently initiate FS-33 S
These comments were addressed.
(D43.7) Answers or item 1.) in stem need to be revised so the answers match the E
92 H 3 N question.
S This question was revised.
U (A43.2) Tech Spec action statements > 1hour, no reference 93 H 3 U N S Agree - This question was rewritten.
U (B43.2) Tech Spec action statements > 1 hour1.157407e-5 days <br />2.777778e-4 hours <br />1.653439e-6 weeks <br />3.805e-7 months <br />, no reference 94 H 3 U M S Agree - This question was revised to provide a reference.
95 F 2 M S (C43.7) 96 F 2 B S (C43.3)
U (B41.10) The term AVAILABLE as supplied for context of this question is defined in 97 F 2 U B S the HPSI operating procedure. RO knowledge. Page 7 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 8 Form ES-401-9
- 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.
Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Agree - This question was revised.
E (D43.3) Include the TS number in the Stem. Verify this with licensee.
98 F 2 B S This comment was addressed.
U (C41.10) This is direct from the AOI. RO knowledge.
99 F 2 U B S Agree - After discussion with the exam author, this question is ok as written.
U (B41.10) RO knowledge.
100 H 3 U N S This question is ok as written.
B= 44 F= 36 E=
RO TOTALS: M= 2 H= 39 U= Additional Notes:
N= 29 B= 6 F= 7 E=
SRO TOTALS: M= 2 H= 18 U= Additional Notes:
N= 17
GENERAL COMMENT
S:
- 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N.
- 2. Chief Examiner comments are indicated in blue.
41.2 = 43.2 =
41.3 = 43.3 =
41.4 = 43.4 =
41.5 = 43.5 =
41.6 = 43.6 =
41.7 = 43.7 =
41.8 =
41.9 =
41.10 =
41.11 =
41.12 = Page 8 of 9 OBDI 202 - IOLE Process
ES-401 SO-2008-09 9 Form ES-401-9 41.13 =
41.14 =
- 6. There are questions with attachments provided. Page 9 of 9 OBDI 202 - IOLE Process