ML101890111: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
 
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:ES-401           Written Examination Review Worksheet Form ES-401-9 GG-2010-06 Instructions [Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.] 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified:
{{#Wiki_filter:ES-401             GG-2010-06                                              Written Examination Review Worksheet                                                                         Form ES-401-9
$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
: 1.     2.           3. Psychometric Flaws              4. Job Content Flaws        5. Other      6.     7.                                               8.
$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
Q#   LOK     LOD (F/H)   (1-5)   Stem Cues T/F        Cred. Partial Job- Minutia #/ Back-            Q= SRO B/M/N U/E/S                                               Explanation Focus                 Dist.           Link             units ward    K/A Only C, CR, 41.7 - Stem needs enhancement to clarify that power is for operation of the valve. As written, there are 2 answers since technically 15P11 does supply power to the MOV motor space heater.
$ The answer choices are a collection of unrelated true/false statements.
E    Editorial - the Explanation contains a reference to LPCI vice LPCS 1      F      2                                                                                          N            Editorial - the Learning Objective related to this in GLP-OPS-E1200 is 7.3.
$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
S Agree with comments.
$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:
$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with ques tion in gallons).
$ The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable). 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer
=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any A U@ ratings (e.g., how the Appendix B psychometric attributes are not being met).
Attachment 8 Page 1 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia#/ units Back-wardQ= K/A SROOnly 1 F 2            N E S C, CR, 41.7 - Stem needs enhancement to clarify that power is for operation of the valve. As written, there are 2 answers since technically 15P11 does supply power to the MOV motor space heater.
Editorial - the Explanation contains a reference to LPCI vice LPCS Editorial - the Learning Objective related to this in GLP-OPS-E1200 is 7.3. Agree with comments.
Licensee will add text to make clearer.
Licensee will add text to make clearer.
Revised, change acceptable. Question is satisfactory.
Revised, change acceptable. Question is satisfactory.
2 F 2            N S E S A, CR, 41.7 The stem needs clarifying so it cannot be misinterpreted to mean instrument air piping from the discharge of the air compressor. If this were the case than distractor B is correct. Suggest adding "System piping-in the steam tunnel-" to the stem. Disregard comment after discussion. Question is satisfactory
A, CR, 41.7 S    The stem needs clarifying so it cannot be misinterpreted to mean instrument air 2      F      2                                                                                          N      E    piping from the discharge of the air compressor. If this were the case than distractor S    B is correct. Suggest adding System pipingin the steam tunnel to the stem.
Disregard comment after discussion. Question is satisfactory Instructions
[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
: 3. Check the appropriate box if a psychometric flaw is identified:
      $    The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
      $    The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
      $    The answer choices are a collection of unrelated true/false statements.
      $    The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
      $    One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 4. Check the appropriate box if a job content error is identified:
      $ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
      $ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
      $ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
      $ The question requires reverse logic or application compared to the job requirements.
: 5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
: 6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
: 7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).                                                                                      Page 1 of 12                                                          OBDI 202 - IOLE Process


ES-401 2 Form ES-401-9 GG-2010-06 Attachment 8 Page 2 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
ES-401       GG-2010-06                                                            2                                                                           Form ES-401-9
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                             8.
Dist. Partial Job-LinkMinutia#/
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                        Explanation Focus              Dist.        Link          units ward K/A Only 3   F     3                                                             X         N   S B, CR, 41.10 4   F     3                       X                                               N   S C, CR, 41.10 5   F     3                                                                         N   S B, CR, 41.7 A, CR, 41.10 - DISCUSS: Wouldnt distracters C and D be plausible if you subtracted Loop A jet pump flows, vice Loop B?
units Back-wardQ= K/A SROOnly 3 F 3          X  N S B, CR, 41.10 4 F 3   X       N S C, CR, 41.10 5 F 3           N S B, CR, 41.7 6 H 3    X        N E S A, CR, 41.10 - DISCUSS: Wouldn't distracters C and D be plausible if you subtracted Loop A jet pump flows, vice Loop B?
E 6    H    3                        X                                                N      Agree with comment - discuss with exam author.
Agree with comment - discuss with exam author.
S Licensee will review question.
Licensee will review question.
Revised, changes acceptable. Question is satisfactory 7   F     4                                                                         N   S B, CR, 41.10 8   H     3                                                                         N   S C, CR, 41.7 9   H     2                                                                         N   S D, CR, 41.7 D, CR, 41.7 10    H    3            X                                                            N    S  A revision was made based on comments provided by internal licensee review.
Revised, changes acceptable. Question is satisfactory 7 F 4           N S B, CR, 41.10 8 H 3           N S C, CR, 41.7 9 H 2           N S D, CR, 41.7 10 H 3  X          N S D, CR, 41.7 A revision was made based on comments provided by internal licensee review. Question and changes reviewed, content is satisfactory.
Question and changes reviewed, content is satisfactory.
11 H 3           N S D, CR, 41.7 12 F 2           N S C, CR, 41.7 13 H 3     X       N S C, CR, 41.10 14 H 3    X        N E S C, CR, 41.10 - Since the question stem tells you the level is in between the given levels on the curve, why would 18.0 feet be selectable (answer B)?
11   H     3                                                                         N   S D, CR, 41.7 12   F     2                                                                         N   S C, CR, 41.7 13   H     3                             X                                         N   S C, CR, 41.10 C, CR, 41.10 - Since the question stem tells you the level is in between the given E  levels on the curve, why would 18.0 feet be selectable (answer B)?
Licensee will reword question to resolve any issues. Discussed basis for distractor B, will leave in the exam. Question is satisfactory.
14    H    3                        X                                                N S  Licensee will reword question to resolve any issues.
15 F 3           N S B, CR, 41.10 16 H 3          X  N S D, CR, 41.14 17 F 3           N S D, CR, 41.10 18 F 2           N S A, CR, 41.10 19 F 3            N S A, CR, 41.11 and .12 Changes were made as a result of an internal exam review. The revisions were reviewed, and the question is satisfactory.
Discussed basis for distractor B, will leave in the exam. Question is satisfactory.
ES-401 3 Form ES-401-9 GG-2010-06 Attachment 8 Page 3 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
15   F     3                                                                         N   S B, CR, 41.10 16   H     3                                                             X         N   S D, CR, 41.14 17   F     3                                                                         N   S D, CR, 41.10 18   F     2                                                                         N   S A, CR, 41.10 A, CR, 41.11 and .12 19    F    3                                                                        N    S  Changes were made as a result of an internal exam review. The revisions were reviewed, and the question is satisfactory.                                                                     Page 2 of 12                                                  OBDI 202 - IOLE Process
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
 
Dist. Partial Job-LinkMinutia#/
ES-401       GG-2010-06                                                            3                                                                             Form ES-401-9
units Back-wardQ= K/A SROOnly 20 H 3    X        B E S D, CR, 41.10 - Should RO know minimum T.S. flowrate requirements for SLC pump? Also, what are bases for distracters A & B (i.e. are they plausible?) Editorial comment on Exam Outline - the outline says that this question is classified as a "F" when it is a "H."
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                             8.
Discuss with exam author - are the expected flow rates the same as the minimum tech spec flow rates? After discussion, questions are resolved. Editorial issue will be resolved. Revision reviewed, question is satisfactory.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                        Explanation Focus              Dist.        Link          units ward K/A Only D, CR, 41.10 - Should RO know minimum T.S. flowrate requirements for SLC pump?
21 F 4           N S D, CR, 41.6 22 F 3           N S D, CR, 41.10 23 F 3        X  N U E S B, CR, 41.11 - This question structure is in conflict with NUREG-1021, Appendix B, Item C.1.h (reverse logic). Also, this is in conflict with NUREG-1021, ES-401, Item D.2.a, first paragraph (use of Tier 3 questions to test systems knowledge). Rewrite question so that it is not negatively stated. Per ES-401-E.2.d, this is an enhancement and therefore is not Unsat but needs revision. Given this is a tier 3 question, recommend selecting a different KA. Licensee will resolve issue. Rejected K/A , selected new Tier 3 K/A for question. New question is satisfactory.
Also, what are bases for distracters A & B (i.e. are they plausible?)
24 H 4           N S A, CR, 41.10 25 H 4    X      N U S C, CR, 41.4 - Procedure 05-1-02-V-1 (Loss of Component Cooling Water), Section 3.1.3, calls for Answer D to be completed as part of its set of subsequent actions. This and RWCU actions happen before adjustments to the Recirc pumps in Section 3.1.4. Further, in taking actions with the Recirc pumps, it says to reduce load on them to 67 lbm/hr, see if things stabilize, and if they don't, switch them to slow speed. I could see an applicant answering C because it is the part of the first action taken here. I could also see the applicant arguing that answer D is incorrect because load on the Recirc pumps should be reduced prior to trying the speed shift. Partially incorrect answer for answer D, partially correct answer on answer C. Discuss with exam author. Licensee will change Distracter D to resolve issue.
Editorial comment on Exam Outline - the outline says that this question is classified E  as a F when it is a H.
20    H    3                        X                                                B S  Discuss with exam author - are the expected flow rates the same as the minimum tech spec flow rates?
After discussion, questions are resolved. Editorial issue will be resolved.
Revision reviewed, question is satisfactory.
21   F     4                                                                         N   S D, CR, 41.6 22   F     3                                                                         N   S D, CR, 41.10 B, CR, 41.11 - This question structure is in conflict with NUREG-1021, Appendix B, Item C.1.h (reverse logic). Also, this is in conflict with NUREG-1021, ES-401, Item D.2.a, first paragraph (use of Tier 3 questions to test systems knowledge).
U Rewrite question so that it is not negatively stated. Per ES-401-E.2.d, this is an 23    F    3                                                          X            N    E  enhancement and therefore is not Unsat but needs revision. Given this is a tier 3 S  question, recommend selecting a different KA.
Licensee will resolve issue.
Rejected K/A , selected new Tier 3 K/A for question. New question is satisfactory.
24   H     4                                                                         N   S A, CR, 41.10 C, CR, 41.4 - Procedure 05-1-02-V-1 (Loss of Component Cooling Water), Section 3.1.3, calls for Answer D to be completed as part of its set of subsequent actions.
This and RWCU actions happen before adjustments to the Recirc pumps in Section 3.1.4. Further, in taking actions with the Recirc pumps, it says to reduce load on them to 67 lbm/hr, see if things stabilize, and if they dont, switch them to slow speed.
U  I could see an applicant answering C because it is the part of the first action taken 25    H    4                              X                                          N      here. I could also see the applicant arguing that answer D is incorrect because load S  on the Recirc pumps should be reduced prior to trying the speed shift. Partially incorrect answer for answer D, partially correct answer on answer C.
Discuss with exam author.
Licensee will change Distracter D to resolve issue.
Distractor D and B have been revised, question is satisfactory.
Distractor D and B have been revised, question is satisfactory.
26 F 3            N S A, CR, 41.4 None of the references for this question are provided. They are needed.
A, CR, 41.4 26    F    3                                                                        N    S  None of the references for this question are provided. They are needed.
Discuss with exam author.
Discuss with exam author.                                                                     Page 3 of 12                                                  OBDI 202 - IOLE Process
ES-401 4 Form ES-401-9 GG-2010-06 Attachment 8 Page 4 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
 
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
ES-401       GG-2010-06                                                            4                                                                           Form ES-401-9
Dist. Partial Job-LinkMinutia#/
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                               8.
units Back-wardQ= K/A SROOnlyLicensee will send needed materials. Materials sent. The question is satisfactory.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                          Explanation Focus              Dist.        Link          units ward K/A Only Licensee will send needed materials.
27 H 3            N E S A, CR, 41.1 and .9 -It says that the correct answer is "C," but the explanation says the correct answer is "A."  Resolve. Discuss with exam author. Licensee will resolve issue.
Materials sent. The question is satisfactory.
A, CR, 41.1 and .9 -It says that the correct answer is C, but the explanation says the correct answer is A. Resolve.
E 27    H    3                                                                        N      Discuss with exam author.
S Licensee will resolve issue.
Change made, question is satisfactory.
Change made, question is satisfactory.
28 H 3            N E S  D, CR, 41.5 - GLP-OPS-C5104 page 11 defines APRM GAF as (actual thermal power/rated thermal power) /(APRM meter reading/100). A loss of RWCU input into the calculated thermal power would cause it to decrease per the explanation. However, based on the above equation, the AGAF would actually be <1 not >1 per explanation. Even though AGAF<1, it would be non-conservative due to the failed computer input. Answer need clarification with lesson material.   
D, CR, 41.5 - GLP-OPS-C5104 page 11 defines APRM GAF as (actual thermal power/rated thermal power) /(APRM meter reading/100). A loss of RWCU input into the calculated thermal power would cause it to decrease per the explanation.
However, based on the above equation, the AGAF would actually be <1 not >1 per explanation. Even though AGAF<1, it would be non-conservative due to the failed computer input.
E  Answer need clarification with lesson material.
28    H    3                                                                        N    S Also, should initial condition state that the current AGAF is 1. If it is initially greater than 1, the decrease in calculated power due to RWCU input will lower AGAF, however it may stay above 1.
Agree with both comments.
Licensee will resolve issues.
Revisions made, question is satisfactory.
29    H    2                                                                        N    S  A, CR, 41.8 30    H    4                                                                        N    S  B, CR, 41.7 31    H    3                                                                        N    S  C, CR, 41.6 32    H    4                                                                        N    S  D, CR, 41.10 33    F    2                                                                        N    S  A, CR, 41.10 34    H    3                                                                        N    S C, CR, 41.7 35    F    2                                                                        N    S  D, CR, 41.6 B, CR, 41.7 - This question is a fundamental knowledge one (F) vice higher E  cognitive testing (H). It is testing knowledge of LPCI/Containment Spray initiation 36    H    2                                                                        N      set points/permissives only.
S This is an editorial comment that has no impact on the actual question.                                                                    Page 4 of 12                                                    OBDI 202 - IOLE Process


Also, should initial condition state that the current AGAF is 1. If it is initially greater than 1, the decrease in calculated power due to RWCU input will lower AGAF, however it may stay above 1. Agree with both comments. Licensee will resolve issues. Revisions made, question is satisfactory.
ES-401       GG-2010-06                                                            5                                                                         Form ES-401-9
29 H 2            N S A, CR, 41.8 30 H 4            N S B, CR, 41.7 31 H 3            N S C, CR, 41.6 32 H 4            N S D, CR, 41.10 33 F 2            N S A, CR, 41.10 34 H 3            N S C, CR, 41.7 35 F 2            N S D, CR, 41.6 36 H 2            N E S B, CR, 41.7 - This question is a fundamental knowledge one ("F") vice higher cognitive testing ("H"). It is testing knowledge of LPCI/Containment Spray initiation set points/permissives only. This is an editorial comment that has no impact on the actual question.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                           8.
ES-401 5 Form ES-401-9 GG-2010-06 Attachment 8 Page 5 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                        Explanation Focus              Dist.        Link          units ward K/A Only Question revised, it is satisfactory.
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
37   F     2                                                                         N   S A, CR, 41.3 E A, CR, 41.7 - Replace distractors C and D as they are not credible.
Dist. Partial Job-LinkMinutia#/
38    H    4                                                                        N S  Changes to distracters C and D acceptable, question is satisfactory.
units Back-wardQ= K/A SROOnlyQuestion revised, it is satisfactory.
39   H     3                                                                         N   S B, CR, 41.7 D, CR, 41.10 Answer C is correct i.a.w 05-1-02-III-1 para 3.4.3. The procedure says line-up S  RWCU alternate SDC. It doesnt ask about existing decay heat. After placing RWCU 40    H    3                                                                        N    E  alternate SDC inservice, the procedure directs performance of answer D.
37 F 2           N S A, CR, 41.3 38 H 4            N E S A, CR, 41.7 - Replace distractors C and D as they are not credible. Changes to distracters C and D acceptable, question is satisfactory. 39 H 3           N S B, CR, 41.7 40 H 3            N S E S D, CR, 41.10 Answer C is correct i.a.w 05-1-02-III-1 para 3.4.3. The procedure says line-up RWCU alternate SDC. It doesn't ask about existing decay heat. After placing RWCU alternate SDC inservice, the procedure directs performance of answer D. Agree with comment. Licensee will review question. Revisions made, question is satisfactory.
S  Agree with comment.
41 H 3           N S A, CR, 41.12 42 F 3    X        N E S B, CR, 41.3 and .5 - Is "post-scram" needed in the stem? It is contradictory with "A" which gives condition of ATWS. "D" is c onfusing and subjective as far as "avoid pressure transients." SRV's are not SSCs that prevent (avoid) a transient, they mitigate it. The distractor could be dismissed by this inaccuracy. Suggest changing "post-scram" to "following a condition requiring a scram". Discuss with exam author. Licensee will resolve text issue.
Licensee will review question.
Revisions made, question is satisfactory.
41   H     3                                                                         N   S A, CR, 41.12 B, CR, 41.3 and .5 - Is post-scram needed in the stem? It is contradictory with A which gives condition of ATWS. D is confusing and subjective as far as avoid pressure transients. SRVs are not SSCs that prevent (avoid) a transient, they E  mitigate it. The distractor could be dismissed by this inaccuracy.
42    F    3                        X                                                N      Suggest changing post-scram to following a condition requiring a scram. Discuss S
with exam author.
Licensee will resolve text issue.
Revisions made, question is satisfactory.
Revisions made, question is satisfactory.
43 H 2           N S D, CR, 41.8 44 F 3           N S B, CR, 41.4 45 H 3           N S C, CR, 41.10 46 H 3           N S D, CR, 41.6 47 H 4           N S B, CR, 41.7 48 H 4           N S A, CR, 41.7 49 H 4           N S B, CR, 41.7 50 H 4           N S C, CR, 41.10 ES-401 6 Form ES-401-9 GG-2010-06 Attachment 8 Page 6 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
43   H     2                                                                         N   S D, CR, 41.8 44   F     3                                                                         N   S B, CR, 41.4 45   H     3                                                                         N   S C, CR, 41.10 46   H     3                                                                         N   S D, CR, 41.6 47   H     4                                                                         N   S B, CR, 41.7 48   H     4                                                                         N   S A, CR, 41.7 49   H     4                                                                         N   S B, CR, 41.7 50   H     4                                                                         N   S C, CR, 41.10                                                                     Page 5 of 12                                                OBDI 202 - IOLE Process
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
 
Dist. Partial Job-LinkMinutia#/
ES-401       GG-2010-06                                                             6                                                                           Form ES-401-9
units Back-wardQ= K/A SROOnly 51 H 3           N S A, CR, 41.7 52 F 3           N S B, CR, 41.7 53 F 3           N S A, CR, 41.7 54 H 3            N E S S B, CR, 41.7 - Move "wait for the --." to the stem since in all answers. Consider selecting a more credible reset action for C and D (i.e. reset the RFPT locally) This question Is S since this is a minor editorial comment - recommend making the suggested change.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                             8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                          Explanation Focus              Dist.        Link          units ward K/A Only 51   H     3                                                                         N   S A, CR, 41.7 52   F     3                                                                         N   S B, CR, 41.7 53   F     3                                                                         N   S A, CR, 41.7 B, CR, 41.7 - Move wait for the . to the stem since in all answers. Consider E  selecting a more credible reset action for C and D (i.e. reset the RFPT locally) 54    H    3                                                                        N    S  This question Is S since this is a minor editorial comment - recommend making the S  suggested change.
Revision made, question is satisfactory.
Revision made, question is satisfactory.
55 F 3           N S C, CR, 41.4 56 H 4            N E S D, CR, 41.7 - 15AA and 16AB have identical BUV schemes, therefore modify C to read "15AA and 16AB are the only buses powered from offsite sources" to improve credibility. Explanation line 3 - add a "not" after did Agree with comments. Licensee will review question.
55   F     3                                                                         N   S C, CR, 41.4 D, CR, 41.7 - 15AA and 16AB have identical BUV schemes, therefore modify C to read 15AA and 16AB are the only buses powered from offsite sources to improve credibility.
E  Explanation line 3 - add a not after did 56    H    4                                                                        N S  Agree with comments.
Licensee will review question.
Changes made, question is satisfactory.
Changes made, question is satisfactory.
57 F 2        X  B U E S C, CR, 41.4 - NUREG-1021, Appendix B, Item C.1.h discourages questions written in this format.
C, CR, 41.4 - NUREG-1021, Appendix B, Item C.1.h discourages questions written in U  this format.
Revise this question so that it is not phrased negatively. Licensee will resolve issue. Question revised, it is satisfactory.
57    F    2                                                          X            B    E  Revise this question so that it is not phrased negatively.
58 H 3    X        N E S C, CR, 41.4 - Remove "Just" from answer A Distractor B and D are similar. Remove "Just" from distractor A and add "only" to the end of the same distractor. Distractors B and D are OK as written. Revision made, question is acceptable. Question is "New" vice "Bank."
S  Licensee will resolve issue.
59 F 3            N E S C, CR, 41.4 - Need to add the justification for why answer D is incorrect.
Question revised, it is satisfactory.
This is a minor editorial comment that has no affect on the question and is not required by NUREG-1021. Agree with comment.
C, CR, 41.4 - Remove Just from answer A Distractor B and D are similar.
E 58    H    3                        X                                                N      Remove Just from distractor A and add only to the end of the same distractor.
S Distractors B and D are OK as written.
Revision made, question is acceptable. Question is New vice Bank.
C, CR, 41.4 - Need to add the justification for why answer D is incorrect.
E  This is a minor editorial comment that has no affect on the question and is not 59    F    3                                                                        N S  required by NUREG-1021. Agree with comment.
Revision made, question is satisfactory.
Revision made, question is satisfactory.
60 H 4            N S A, CR, 41.10 - A revision was made based on the results of an internal licensee review. The revision was reviewed, and the question is satisfactory.
A, CR, 41.10 - A revision was made based on the results of an internal licensee 60    H    4                                                                        N    S review. The revision was reviewed, and the question is satisfactory.                                                                     Page 6 of 12                                                  OBDI 202 - IOLE Process
ES-401 7 Form ES-401-9 GG-2010-06 Attachment 8 Page 7 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
 
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
ES-401       GG-2010-06                                                            7                                                                           Form ES-401-9
Dist. Partial Job-LinkMinutia#/
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                             8.
units Back-wardQ= K/A SROOnly 61 F 2    X        N S B, CR, 41.10 - A revision was made based on the results of an internal licensee review. The revision was reviewed, and the question is satisfactory.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                        Explanation Focus              Dist.        Link          units ward K/A Only B, CR, 41.10 - A revision was made based on the results of an internal licensee 61    F    2                        X                                                N    S review. The revision was reviewed, and the question is satisfactory.
62 H 3            N S E S B, CR, 41.10 Answer A is not plausible since stem says CCW temp = SFP temp, therefore there is no differential to "rapidly turn temperature." Agree - Discuss with exam author.
B, CR, 41.10 S  Answer A is not plausible since stem says CCW temp = SFP temp, therefore there is no differential to rapidly turn temperature.
This issue is based on actual operating experience at plant. Question is satisfactory as written.
62    H    3                                                                        N    E Agree - Discuss with exam author.
63 H 3           N S D, CR, 41.10 64 F 3   X       N S B, CR, 41.10 65 H 3    X  X    N U S B, CR, 41.4 - A & B - Order of sequencing between RHRP B and RHRP C is too specific for closed book test. C & D - Not credible that identical pumps would have differing power requirements without more information in the stem (i.e. differing configurations)   Agree - Suggest making C & D 200 KW and 200 KW instead of 200 KW and 800 KW as currently written to improve credibility. The sequencing of B and C are reasonable memory questions so A and B are OK as written. Licensee will resolve issue. Question resolved. It is now satisfactory.
S This issue is based on actual operating experience at plant. Question is satisfactory as written.
66 H 3    X        B E S A, CR, 41.4 - Answer C is not plausible. Is there a situation where the applicant could plausibly deduct that the bus voltage could be 60 VDC? Agree with comment.
63   H     3                                                                         N   S D, CR, 41.10 64   F     3                       X                                               N   S B, CR, 41.10 B, CR, 41.4 - A & B - Order of sequencing between RHRP B and RHRP C is too specific for closed book test.
C & D - Not credible that identical pumps would have differing power requirements without more information in the stem (i.e. differing configurations)
U 65    H    3                        X                    X                          N      Agree - Suggest making C & D 200 KW and 200 KW instead of 200 KW and 800 KW S  as currently written to improve credibility. The sequencing of B and C are reasonable memory questions so A and B are OK as written.
Licensee will resolve issue.
Question resolved. It is now satisfactory.
A, CR, 41.4 - Answer C is not plausible. Is there a situation where the applicant E  could plausibly deduct that the bus voltage could be 60 VDC?
66    H    3                        X                                                B S  Agree with comment.
Licensee explained answer, it is plausible. No further issues.
Licensee explained answer, it is plausible. No further issues.
67 H 3           B S C, CR, 41.7 68 F 4            N E U S A, CR, 41.4 - This appears to be a SRO question, vice RO. Is it expected that RO's understand these controls? This is an SRO only question since is testing on the refueling bridge interlocks. Recommend rejecting this KA.
67   H     3                                                                         B   S C, CR, 41.7 A, CR, 41.4 - This appears to be a SRO question, vice RO. Is it expected that ROs understand these controls?
Licensee will re-select K/A and address question.
This is an SRO only question since is testing on the refueling bridge interlocks.
During validation, the licensee proposed a new question to address the existing K/A, instead of re-selecting the K/A. They were able to write a question testing RO knowledge of controls/indications in the control room during the new conditions provided. Proposed change is acceptable. Question is satisfactory.
E  Recommend rejecting this KA.
ES-401 8 Form ES-401-9 GG-2010-06 Attachment 8 Page 8 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
68    F    4                                                                        N    U  Licensee will re-select K/A and address question.
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
S  During validation, the licensee proposed a new question to address the existing K/A, instead of re-selecting the K/A. They were able to write a question testing RO knowledge of controls/indications in the control room during the new conditions provided. Proposed change is acceptable. Question is satisfactory.                                                                     Page 7 of 12                                                  OBDI 202 - IOLE Process
Dist. Partial Job-LinkMinutia#/
 
units Back-wardQ= K/A SROOnly 69 H 3   X       N S C, CR, 41.4 70 F 2    X        N U E S D, CR, 41.10 - Since the explanations for answers B and C say that there is no evidence that these effects can happen, and there are no requirements associated with them, how can you say that they are plausible? Distractor C is plausible since some heat exchangers do have heatup rate limits associated with them. Distractor B is not plausible.
ES-401       GG-2010-06                                                            8                                                                           Form ES-401-9
Licensee will review and propose change to distracter B.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                             8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                        Explanation Focus              Dist.        Link          units ward K/A Only 69   H     3                       X                                               N   S C, CR, 41.4 D, CR, 41.10 - Since the explanations for answers B and C say that there is no evidence that these effects can happen, and there are no requirements associated U  with them, how can you say that they are plausible?
70    F    2                        X                                                N    E  Distractor C is plausible since some heat exchangers do have heatup rate limits associated with them. Distractor B is not plausible.
S Licensee will review and propose change to distracter B.
Change made to the distracter in question. Question is satisfactory.
Change made to the distracter in question. Question is satisfactory.
71 F 4            N E S A, CR, 41.10 - Tier 3 questions are expected to be based on plant-wide procedures, and not system specific. This question is system specific. See NUREG-1021, ES-401, Item D.2.a (first paragraph). Agree with comment - this needs to be a generic question.
A, CR, 41.10 - Tier 3 questions are expected to be based on plant-wide procedures, and not system specific. This question is system specific. See NUREG-1021, ES-E  401, Item D.2.a (first paragraph).
Licensee will review and resolve issue. Changes made. Question is satisfactory. Now assigned to 55.41(b)(10).
71    F    4                                                                        N      Agree with comment - this needs to be a generic question.
72 F 3           N S A, CR, 41.10 73 H 3          X  B S A, CR, 41.10 74 F 2    X        N E S A, CR, 41.4 - Answer C suggests a rise in voltage on a bus when grounds are present. When there are additional loads for current draw, the voltage can only go down. This is an implausible distracter. Editorial - In the Explanation, it would appear that the statement saying the correct answer is C should say A. The stem indicated a local ground detector in alarm. It is common operator knowledge that a local alarm will also give an alarm in the control room (for the majority of cases). "A" is the only answer that indicates an alarm in the control room making it the logical answer. Answers do not discriminate adequate knowledge. This question needs credible distractors. Agree with comments.
S Licensee will review and resolve issue.
Changes made. Question is satisfactory. Now assigned to 55.41(b)(10).
72   F     3                                                                         N   S A, CR, 41.10 73   H     3                                                             X         B   S A, CR, 41.10 A, CR, 41.4 - Answer C suggests a rise in voltage on a bus when grounds are present. When there are additional loads for current draw, the voltage can only go down. This is an implausible distracter.
Editorial - In the Explanation, it would appear that the statement saying the correct answer is C should say A.
The stem indicated a local ground detector in alarm. It is common operator knowledge that a local alarm will also give an alarm in the control room (for the E  majority of cases). A is the only answer that indicates an alarm in the control room 74    F    2                        X                                                N      making it the logical answer. Answers do not discriminate adequate knowledge. This S  question needs credible distractors.
Agree with comments.
Answer C is plausible based on observed level of knowledge of applicants based on LOK of ground detection.
Answer C is plausible based on observed level of knowledge of applicants based on LOK of ground detection.
Answer A question resolved by fix of editorial issue. Licensee will review for further changes.
Answer A question resolved by fix of editorial issue.
Licensee will review for further changes.
Question revised. It is satisfactory.
75    F    2                                                                        N    S  D, CR, 41.4                                                                    Page 8 of 12                                                  OBDI 202 - IOLE Process
 
ES-401        GG-2010-06                                                            9                                                                          Form ES-401-9
: 1. 2.      3. Psychometric Flaws          4. Job Content Flaws  5. Other  6. 7.                                              8.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F      Cred. Partial Job- Minutia #/ Back-    Q= SRO B/M/N U/E/S                                          Explanation Focus              Dist.        Link          units ward K/A Only D, CR, 43.2 - NUREG-1021, Appendix B, Item C.1.h discourages questions written in U  this format.
76    F    3                                                          X            N    E  Rewrite question so that it is not written in a negative context.
S  Licensee will resolve issue.
Question revised. It is satisfactory.
Question revised. It is satisfactory.
75 F 2            N S D, CR, 41.4 ES-401 9 Form ES-401-9 GG-2010-06 Attachment 8 Page 9 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
77   H     2                 X                                           X         N   S A, OR, 43.5 78   H     3                                                                         N   S C, CR, 43.2 79   F     3                                                                         B   S D, CR, 43.5 80   H     3                       X                                               N   S B, OR, 43.5 81   H     4                                                                         N   S A, OR 43.2 D, CR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43(b)(2)] doesnt match with this. That is for Tech Spec/TRM application. There should be a better 82    H    3                                                                        N    S  match.
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
Agree with comment.
Dist. Partial Job-LinkMinutia#/
Changed the attribute to 55.41(b)(10).
units Back-wardQ= K/A SROOnly 76 F 3        X  N U E S D, CR, 43.2 - NUREG-1021, Appendix B, Item C.1.h discourages questions written in this format.
B, OR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43.(b)(10)] doesnt 83    H    2                                                                        N    S  exist. Review for better match.
Rewrite question so that it is not written in a negative context.
Agree with comment. Changed to 55.41(b)(10).
Licensee will resolve issue. Question revised. It is satisfactory.
84   H     3                                                             X         B   S A, OR, 43.2 85   H     3                                                                         N   S D, CR, 43.2 D, CR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43(b)(2)] doesnt match with this. That is for Tech Spec/TRM application. There should be a better 86    F    3                                                                        N    S  match.
77 H 2   X       X N S A, OR, 43.5 78 H 3           N S C, CR, 43.2 79 F 3           B S D, CR, 43.5 80 H 3   X       N S B, OR, 43.5 81 H 4           N S A, OR 43.2 82 H 3            N S D, CR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43(b)(2)] doesn't match with this. That is for Tech Spec/TRM application. There should be a better match. Agree with comment. Changed the attribute to 55.41(b)(10).
Agree with comment. Changed to 55.41 (b)(10).
83 H 2            N S B, OR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43.(b)(10)] doesn't exist. Review for better match. Agree with comment. Changed to 55.41(b)(10).
87   H     3                             X                               X         N   S D, OR, 43.2 B, OR, 43.5 - Editorial comment - In the explanation, the last sentence says C is correct when B is the actual answer.
84 H 3          X  B S A, OR, 43.2 85 H 3           N S D, CR, 43.2 86 F 3            N S D, CR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43(b)(2)] doesn't match with this. That is for Tech Spec/TRM application. There should be a better match. Agree with comment. Changed to 55.41 (b)(10).
88    H    3                                                                        N    S Correct the typo.
87 H 3     X     X N S D, OR, 43.2 88 H 3            N S B, OR, 43.5 - Editorial comment - In the explanation, the last sentence says "C" is correct when "B" is the actual answer. Correct the typo. Typographical error resolved.
Typographical error resolved.
89 H 4           X N U C, CR, 43.5 - This question can be answered with systems knowledge of the effects ES-401 10 Form ES-401-9 GG-2010-06 Attachment 8 Page 10 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
89   H     4                                                                   X   N   U C, CR, 43.5 - This question can be answered with systems knowledge of the effects                                                                     Page 9 of 12                                                  OBDI 202 - IOLE Process
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
 
Dist. Partial Job-LinkMinutia#/
ES-401       GG-2010-06                                                             10                                                                         Form ES-401-9
units Back-wardQ= K/A SROOnly E S of instrument air loss. No special knowledge of decision making in EP procedures is needed to come to the right answer. This is an RO question.
: 1. 2.       3. Psychometric Flaws         4. Job Content Flaws   5. Other   6. 7.                                             8.
This is not an RO question as it is te sting on decision making involving the EOPs given a set of conditions. Discuss with the exam author why distractor D is credible. Licensee will review question and propose changes as needed.
Q#  LOK  LOD (F/H) (1-5) Stem Cues T/F     Cred. Partial Job- Minutia #/ Back-   Q= SRO B/M/N U/E/S                                        Explanation Focus              Dist.        Link          units ward K/A Only E of instrument air loss. No special knowledge of decision making in EP procedures is S  needed to come to the right answer. This is an RO question.
This is not an RO question as it is testing on decision making involving the EOPs given a set of conditions. Discuss with the exam author why distractor D is credible.
Licensee will review question and propose changes as needed.
Distracter D justification modified. Question is satisfactory.
Distracter D justification modified. Question is satisfactory.
90 F 2    X        B E S C, CR, 43.2 - Answer A - Remove "just" and clarify to increase plausibility. Is using RWCU in answer D plausible? Is it a means of decay heat removal that is credited? If it isn't, then the remainder of the answer is the same as answer C. Discuss Tech Spec bases. This question is OK as written. The RWCU is plausible since it is one of the alternate decay heat removal systems normally available. Agree with removing "Just" from distracter A.
C, CR, 43.2 - Answer A - Remove just and clarify to increase plausibility.
Is using RWCU in answer D plausible? Is it a means of decay heat removal that is credited? If it isnt, then the remainder of the answer is the same as answer C.
E  Discuss Tech Spec bases.
90    F    2                        X                                                B S  This question is OK as written. The RWCU is plausible since it is one of the alternate decay heat removal systems normally available. Agree with removing Just from distracter A.
Editorial change has been made.
Editorial change has been made.
91 F 4           N S D, CR, 43.7 92 F 3            N E S B, CR, 43.4 - Add 10CFR55.43 reference This question is S since the editorial comment has no effect on the question. Connected to 55.43(b)(4).
91   F     4                                                                         N S D, CR, 43.7 B, CR, 43.4 - Add 10CFR55.43 reference E
93 F 3           N S D, CR, 43.4 94 H 3           N S A, OR, 43.5 95 F 2           N S A, CR, 43.2 96 H 3          X  N U S C, CR, 43.10 - Editorial: The attributed 10 CFR 55 section [55.43.(b)(10)] doesn't exist. Review for better match. Question does not address reactor/turbine pressure regulating system Agree with comment except it also does not address a safety related equipment.
92    F    3                                                                          N    This question is S since the editorial comment has no effect on the question.
S Connected to 55.43(b)(4).
93   F     3                                                                         N S D, CR, 43.4 94   H     3                                                                         N S A, OR, 43.5 95   F     2                                                                         N S A, CR, 43.2 C, CR, 43.10 - Editorial: The attributed 10 CFR 55 section [55.43.(b)(10)] doesnt exist. Review for better match.
U  Question does not address reactor/turbine pressure regulating system 96    H    3                                                              X          N S  Agree with comment except it also does not address a safety related equipment.
Licensee will reject the K/A and reselect for new question.
Licensee will reject the K/A and reselect for new question.
Question has been revised. It is satisfactory.
Question has been revised. It is satisfactory.
97 H 3           N S C, OR, 43.5 98 H 4    X        N E S B, OR, 43.5 - If the applicant establishes that the plant didn't meet the conditions to enter EP-3 (Suppression Pool temperature greater than 95F) for answer C or D, then he/she automatically knows the other is wrong. One of these distracters should be revised.
97   H     3                                                                         N S C, OR, 43.5 B, OR, 43.5 - If the applicant establishes that the plant didnt meet the conditions to E  enter EP-3 (Suppression Pool temperature greater than 95F) for answer C or D, then 98    H    4                        X                                                N S  he/she automatically knows the other is wrong. One of these distracters should be revised.                                                                   Page 10 of 12                                                  OBDI 202 - IOLE Process
ES-401 11 Form ES-401-9 GG-2010-06 Attachment 8 Page 11 of 12 OBDI 202 - IOLE Process Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6.
 
B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred.
ES-401           GG-2010-06                                                                  11                                                                       Form ES-401-9
Dist. Partial Job-LinkMinutia#/
: 1.     2.       3. Psychometric Flaws           4. Job Content Flaws     5. Other   6. 7.                                           8.
units Back-wardQ= K/A SROOnlyAgree with comment. Licensee will resolve issue with question. Distracter D changed. Question is satisfactory.
Q#    LOK    LOD (F/H)  (1-5) Stem Cues T/F       Cred. Partial Job- Minutia #/ Back-       Q= SRO B/M/N U/E/S                                      Explanation Focus                Dist.        Link          units ward    K/A Only Agree with comment.
99 H 3           N S C, CR, 43.5 100 H 3            N E S A, CR, 41.10 - Reference to 55.43(b)(4) is not correct.
Licensee will resolve issue with question.
Question is S - Agree with comment.
Distracter D changed. Question is satisfactory.
Referenced to 55.41(b)(10). RO TOTALS:
99     H       3                                                                               N S C, CR, 43.5 A, CR, 41.10 - Reference to 55.43(b)(4) is not correct.
B= 5 F=32 (43%) E=21* Additional Notes: *After review and comment resolution, all questions initially categorized as needing enhancement (E) or unsatisfactory (U) are satisfactory.
E 100    H      3                                                                              N    Question is S - Agree with comment.
M= 0 H=43 (57%) U=3* N= 70   SRO TOTALS:
S Referenced to 55.41(b)(10).
B= 3 F=8 (32%) E=3* Additional Notes: *After review and comment resolution, all questions initially categorized as needing enhancement (E) or unsatisfactory (U) are satisfactory.
B= 5               F= 32 (43%)           E= 21*
M= 0 H=17 (68%) U=1* N= 22  
Additional Notes: *After review and comment resolution, all questions initially RO TOTALS:                M= 0                H= 43 (57%)          U= 3*
categorized as needing enhancement (E) or unsatisfactory (U) are satisfactory.
N= 70 B= 3               F= 8 (32%)           E= 3*
Additional Notes: *After review and comment resolution, all questions initially SRO TOTALS:                M= 0                H= 17 (68%)          U= 1*
categorized as needing enhancement (E) or unsatisfactory (U) are satisfactory.
N= 22


==GENERAL COMMENT==
==GENERAL COMMENT==
Line 147: Line 230:
: 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
: 1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
: 2. Chief Examiner comments are indicated in blue.
: 2. Chief Examiner comments are indicated in blue.
: 3. Average difficulty is __
: 3. Average difficulty is __   3__ on the RO exam and         3__     on the SRO exam.
3__   on the RO exam and 3__     on the SRO exam.
: 4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 1*               43.1 = 0 41.2 = 0                 43.2 = 10 41.3 = 2*               43.3 = 0 41.4 = 14               43.4 = 2 41.5 = 2*               43.5 = 9 41.6 = 4*               43.6 = 0 41.7 = 22               43.7 = 1 41.8 = 2                 (3 SRO questions 10 CFR 55.43 designation TBD) 41.9 = 1*
: 4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 1* 43.1 = 0 41.2 = 0 43.2 = 10 41.3 = 2* 43.3 = 0 41.4 = 14 43.4 = 2 41.5 = 2* 43.5 = 9 41.6 = 4* 43.6 = 0 41.7 = 22 43.7 = 1 41.8 = 2 (3 SRO questions 10 CFR 55.43 designation TBD) 41.9 = 1* 41.10 = 27* 41.11 = 2*
41.10 = 27*
41.12 = 2*
41.11 = 2*
ES-401 12 Form ES-401-9 GG-2010-06 Attachment 8 Page 12 of 12 OBDI 202 - IOLE Process 41.13 = 0 41.14 = 1       *Questions written credited addressing more than one 10 CFR 41(b) section.
41.12 = 2*                                                                             Page 11 of 12                                                OBDI 202 - IOLE Process
: 5. The answer distribution is: RO / SRO A =     20 / 6   B =     18 / 5   C =     19 / 6 D =     18 /8
 
: 6. There are 10   questions with attachments provided.
ES-401           GG-2010-06                                                        12                       Form ES-401-9 41.13 = 0 41.14 = 1
: 7. This review was done by S. Garchow, S Hedger, and T. Pate the week of 5/17/2010.
    *Questions written credited addressing more than one 10 CFR 41(b) section.
: 8.           The SRO exam contains 68% higher cognitive questions and this is above the 60% limit.}}
: 5.       The answer distribution is: RO / SRO A=     20 / 6 B=     18 / 5 C=     19 / 6 D=     18 /8
: 6.       There are 10     questions with attachments provided.
: 7.       This review was done by S. Garchow, S Hedger, and T. Pate the week of 5/17/2010.
: 8.       The SRO exam contains 68% higher cognitive questions and this is above the 60% limit.                                                                   Page 12 of 12    OBDI 202 - IOLE Process}}

Latest revision as of 17:26, 13 November 2019

2010-06-DRAFT-Written Exam Comments
ML101890111
Person / Time
Site: Grand Gulf Entergy icon.png
Issue date: 06/25/2010
From:
NRC Region 4
To:
Entergy Operations
References
50-416/10-301
Download: ML101890111 (12)
v • d • e


Text

ES-401 GG-2010-06 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only C, CR, 41.7 - Stem needs enhancement to clarify that power is for operation of the valve. As written, there are 2 answers since technically 15P11 does supply power to the MOV motor space heater.

E Editorial - the Explanation contains a reference to LPCI vice LPCS 1 F 2 N Editorial - the Learning Objective related to this in GLP-OPS-E1200 is 7.3.

S Agree with comments.

Licensee will add text to make clearer.

Revised, change acceptable. Question is satisfactory.

A, CR, 41.7 S The stem needs clarifying so it cannot be misinterpreted to mean instrument air 2 F 2 N E piping from the discharge of the air compressor. If this were the case than distractor S B is correct. Suggest adding System pipingin the steam tunnel to the stem.

Disregard comment after discussion. Question is satisfactory Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

$ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

$ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

$ The answer choices are a collection of unrelated true/false statements.

$ The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

$ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

$ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

$ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

$ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

$ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met). Page 1 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 3 F 3 X N S B, CR, 41.10 4 F 3 X N S C, CR, 41.10 5 F 3 N S B, CR, 41.7 A, CR, 41.10 - DISCUSS: Wouldnt distracters C and D be plausible if you subtracted Loop A jet pump flows, vice Loop B?

E 6 H 3 X N Agree with comment - discuss with exam author.

S Licensee will review question.

Revised, changes acceptable. Question is satisfactory 7 F 4 N S B, CR, 41.10 8 H 3 N S C, CR, 41.7 9 H 2 N S D, CR, 41.7 D, CR, 41.7 10 H 3 X N S A revision was made based on comments provided by internal licensee review.

Question and changes reviewed, content is satisfactory.

11 H 3 N S D, CR, 41.7 12 F 2 N S C, CR, 41.7 13 H 3 X N S C, CR, 41.10 C, CR, 41.10 - Since the question stem tells you the level is in between the given E levels on the curve, why would 18.0 feet be selectable (answer B)?

14 H 3 X N S Licensee will reword question to resolve any issues.

Discussed basis for distractor B, will leave in the exam. Question is satisfactory.

15 F 3 N S B, CR, 41.10 16 H 3 X N S D, CR, 41.14 17 F 3 N S D, CR, 41.10 18 F 2 N S A, CR, 41.10 A, CR, 41.11 and .12 19 F 3 N S Changes were made as a result of an internal exam review. The revisions were reviewed, and the question is satisfactory. Page 2 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D, CR, 41.10 - Should RO know minimum T.S. flowrate requirements for SLC pump?

Also, what are bases for distracters A & B (i.e. are they plausible?)

Editorial comment on Exam Outline - the outline says that this question is classified E as a F when it is a H.

20 H 3 X B S Discuss with exam author - are the expected flow rates the same as the minimum tech spec flow rates?

After discussion, questions are resolved. Editorial issue will be resolved.

Revision reviewed, question is satisfactory.

21 F 4 N S D, CR, 41.6 22 F 3 N S D, CR, 41.10 B, CR, 41.11 - This question structure is in conflict with NUREG-1021, Appendix B, Item C.1.h (reverse logic). Also, this is in conflict with NUREG-1021, ES-401, Item D.2.a, first paragraph (use of Tier 3 questions to test systems knowledge).

U Rewrite question so that it is not negatively stated. Per ES-401-E.2.d, this is an 23 F 3 X N E enhancement and therefore is not Unsat but needs revision. Given this is a tier 3 S question, recommend selecting a different KA.

Licensee will resolve issue.

Rejected K/A , selected new Tier 3 K/A for question. New question is satisfactory.

24 H 4 N S A, CR, 41.10 C, CR, 41.4 - Procedure 05-1-02-V-1 (Loss of Component Cooling Water), Section 3.1.3, calls for Answer D to be completed as part of its set of subsequent actions.

This and RWCU actions happen before adjustments to the Recirc pumps in Section 3.1.4. Further, in taking actions with the Recirc pumps, it says to reduce load on them to 67 lbm/hr, see if things stabilize, and if they dont, switch them to slow speed.

U I could see an applicant answering C because it is the part of the first action taken 25 H 4 X N here. I could also see the applicant arguing that answer D is incorrect because load S on the Recirc pumps should be reduced prior to trying the speed shift. Partially incorrect answer for answer D, partially correct answer on answer C.

Discuss with exam author.

Licensee will change Distracter D to resolve issue.

Distractor D and B have been revised, question is satisfactory.

A, CR, 41.4 26 F 3 N S None of the references for this question are provided. They are needed.

Discuss with exam author. Page 3 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Licensee will send needed materials.

Materials sent. The question is satisfactory.

A, CR, 41.1 and .9 -It says that the correct answer is C, but the explanation says the correct answer is A. Resolve.

E 27 H 3 N Discuss with exam author.

S Licensee will resolve issue.

Change made, question is satisfactory.

D, CR, 41.5 - GLP-OPS-C5104 page 11 defines APRM GAF as (actual thermal power/rated thermal power) /(APRM meter reading/100). A loss of RWCU input into the calculated thermal power would cause it to decrease per the explanation.

However, based on the above equation, the AGAF would actually be <1 not >1 per explanation. Even though AGAF<1, it would be non-conservative due to the failed computer input.

E Answer need clarification with lesson material.

28 H 3 N S Also, should initial condition state that the current AGAF is 1. If it is initially greater than 1, the decrease in calculated power due to RWCU input will lower AGAF, however it may stay above 1.

Agree with both comments.

Licensee will resolve issues.

Revisions made, question is satisfactory.

29 H 2 N S A, CR, 41.8 30 H 4 N S B, CR, 41.7 31 H 3 N S C, CR, 41.6 32 H 4 N S D, CR, 41.10 33 F 2 N S A, CR, 41.10 34 H 3 N S C, CR, 41.7 35 F 2 N S D, CR, 41.6 B, CR, 41.7 - This question is a fundamental knowledge one (F) vice higher E cognitive testing (H). It is testing knowledge of LPCI/Containment Spray initiation 36 H 2 N set points/permissives only.

S This is an editorial comment that has no impact on the actual question. Page 4 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Question revised, it is satisfactory.

37 F 2 N S A, CR, 41.3 E A, CR, 41.7 - Replace distractors C and D as they are not credible.

38 H 4 N S Changes to distracters C and D acceptable, question is satisfactory.

39 H 3 N S B, CR, 41.7 D, CR, 41.10 Answer C is correct i.a.w 05-1-02-III-1 para 3.4.3. The procedure says line-up S RWCU alternate SDC. It doesnt ask about existing decay heat. After placing RWCU 40 H 3 N E alternate SDC inservice, the procedure directs performance of answer D.

S Agree with comment.

Licensee will review question.

Revisions made, question is satisfactory.

41 H 3 N S A, CR, 41.12 B, CR, 41.3 and .5 - Is post-scram needed in the stem? It is contradictory with A which gives condition of ATWS. D is confusing and subjective as far as avoid pressure transients. SRVs are not SSCs that prevent (avoid) a transient, they E mitigate it. The distractor could be dismissed by this inaccuracy.

42 F 3 X N Suggest changing post-scram to following a condition requiring a scram. Discuss S

with exam author.

Licensee will resolve text issue.

Revisions made, question is satisfactory.

43 H 2 N S D, CR, 41.8 44 F 3 N S B, CR, 41.4 45 H 3 N S C, CR, 41.10 46 H 3 N S D, CR, 41.6 47 H 4 N S B, CR, 41.7 48 H 4 N S A, CR, 41.7 49 H 4 N S B, CR, 41.7 50 H 4 N S C, CR, 41.10 Page 5 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 51 H 3 N S A, CR, 41.7 52 F 3 N S B, CR, 41.7 53 F 3 N S A, CR, 41.7 B, CR, 41.7 - Move wait for the . to the stem since in all answers. Consider E selecting a more credible reset action for C and D (i.e. reset the RFPT locally) 54 H 3 N S This question Is S since this is a minor editorial comment - recommend making the S suggested change.

Revision made, question is satisfactory.

55 F 3 N S C, CR, 41.4 D, CR, 41.7 - 15AA and 16AB have identical BUV schemes, therefore modify C to read 15AA and 16AB are the only buses powered from offsite sources to improve credibility.

E Explanation line 3 - add a not after did 56 H 4 N S Agree with comments.

Licensee will review question.

Changes made, question is satisfactory.

C, CR, 41.4 - NUREG-1021, Appendix B, Item C.1.h discourages questions written in U this format.

57 F 2 X B E Revise this question so that it is not phrased negatively.

S Licensee will resolve issue.

Question revised, it is satisfactory.

C, CR, 41.4 - Remove Just from answer A Distractor B and D are similar.

E 58 H 3 X N Remove Just from distractor A and add only to the end of the same distractor.

S Distractors B and D are OK as written.

Revision made, question is acceptable. Question is New vice Bank.

C, CR, 41.4 - Need to add the justification for why answer D is incorrect.

E This is a minor editorial comment that has no affect on the question and is not 59 F 3 N S required by NUREG-1021. Agree with comment.

Revision made, question is satisfactory.

A, CR, 41.10 - A revision was made based on the results of an internal licensee 60 H 4 N S review. The revision was reviewed, and the question is satisfactory. Page 6 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 7 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only B, CR, 41.10 - A revision was made based on the results of an internal licensee 61 F 2 X N S review. The revision was reviewed, and the question is satisfactory.

B, CR, 41.10 S Answer A is not plausible since stem says CCW temp = SFP temp, therefore there is no differential to rapidly turn temperature.

62 H 3 N E Agree - Discuss with exam author.

S This issue is based on actual operating experience at plant. Question is satisfactory as written.

63 H 3 N S D, CR, 41.10 64 F 3 X N S B, CR, 41.10 B, CR, 41.4 - A & B - Order of sequencing between RHRP B and RHRP C is too specific for closed book test.

C & D - Not credible that identical pumps would have differing power requirements without more information in the stem (i.e. differing configurations)

U 65 H 3 X X N Agree - Suggest making C & D 200 KW and 200 KW instead of 200 KW and 800 KW S as currently written to improve credibility. The sequencing of B and C are reasonable memory questions so A and B are OK as written.

Licensee will resolve issue.

Question resolved. It is now satisfactory.

A, CR, 41.4 - Answer C is not plausible. Is there a situation where the applicant E could plausibly deduct that the bus voltage could be 60 VDC?

66 H 3 X B S Agree with comment.

Licensee explained answer, it is plausible. No further issues.

67 H 3 B S C, CR, 41.7 A, CR, 41.4 - This appears to be a SRO question, vice RO. Is it expected that ROs understand these controls?

This is an SRO only question since is testing on the refueling bridge interlocks.

E Recommend rejecting this KA.

68 F 4 N U Licensee will re-select K/A and address question.

S During validation, the licensee proposed a new question to address the existing K/A, instead of re-selecting the K/A. They were able to write a question testing RO knowledge of controls/indications in the control room during the new conditions provided. Proposed change is acceptable. Question is satisfactory. Page 7 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 8 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only 69 H 3 X N S C, CR, 41.4 D, CR, 41.10 - Since the explanations for answers B and C say that there is no evidence that these effects can happen, and there are no requirements associated U with them, how can you say that they are plausible?

70 F 2 X N E Distractor C is plausible since some heat exchangers do have heatup rate limits associated with them. Distractor B is not plausible.

S Licensee will review and propose change to distracter B.

Change made to the distracter in question. Question is satisfactory.

A, CR, 41.10 - Tier 3 questions are expected to be based on plant-wide procedures, and not system specific. This question is system specific. See NUREG-1021, ES-E 401, Item D.2.a (first paragraph).

71 F 4 N Agree with comment - this needs to be a generic question.

S Licensee will review and resolve issue.

Changes made. Question is satisfactory. Now assigned to 55.41(b)(10).

72 F 3 N S A, CR, 41.10 73 H 3 X B S A, CR, 41.10 A, CR, 41.4 - Answer C suggests a rise in voltage on a bus when grounds are present. When there are additional loads for current draw, the voltage can only go down. This is an implausible distracter.

Editorial - In the Explanation, it would appear that the statement saying the correct answer is C should say A.

The stem indicated a local ground detector in alarm. It is common operator knowledge that a local alarm will also give an alarm in the control room (for the E majority of cases). A is the only answer that indicates an alarm in the control room 74 F 2 X N making it the logical answer. Answers do not discriminate adequate knowledge. This S question needs credible distractors.

Agree with comments.

Answer C is plausible based on observed level of knowledge of applicants based on LOK of ground detection.

Answer A question resolved by fix of editorial issue.

Licensee will review for further changes.

Question revised. It is satisfactory.

75 F 2 N S D, CR, 41.4 Page 8 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 9 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only D, CR, 43.2 - NUREG-1021, Appendix B, Item C.1.h discourages questions written in U this format.

76 F 3 X N E Rewrite question so that it is not written in a negative context.

S Licensee will resolve issue.

Question revised. It is satisfactory.

77 H 2 X X N S A, OR, 43.5 78 H 3 N S C, CR, 43.2 79 F 3 B S D, CR, 43.5 80 H 3 X N S B, OR, 43.5 81 H 4 N S A, OR 43.2 D, CR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43(b)(2)] doesnt match with this. That is for Tech Spec/TRM application. There should be a better 82 H 3 N S match.

Agree with comment.

Changed the attribute to 55.41(b)(10).

B, OR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43.(b)(10)] doesnt 83 H 2 N S exist. Review for better match.

Agree with comment. Changed to 55.41(b)(10).

84 H 3 X B S A, OR, 43.2 85 H 3 N S D, CR, 43.2 D, CR, 41.10 - Editorial: The attributed 10 CFR 55 section [55.43(b)(2)] doesnt match with this. That is for Tech Spec/TRM application. There should be a better 86 F 3 N S match.

Agree with comment. Changed to 55.41 (b)(10).

87 H 3 X X N S D, OR, 43.2 B, OR, 43.5 - Editorial comment - In the explanation, the last sentence says C is correct when B is the actual answer.

88 H 3 N S Correct the typo.

Typographical error resolved.

89 H 4 X N U C, CR, 43.5 - This question can be answered with systems knowledge of the effects Page 9 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 10 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only E of instrument air loss. No special knowledge of decision making in EP procedures is S needed to come to the right answer. This is an RO question.

This is not an RO question as it is testing on decision making involving the EOPs given a set of conditions. Discuss with the exam author why distractor D is credible.

Licensee will review question and propose changes as needed.

Distracter D justification modified. Question is satisfactory.

C, CR, 43.2 - Answer A - Remove just and clarify to increase plausibility.

Is using RWCU in answer D plausible? Is it a means of decay heat removal that is credited? If it isnt, then the remainder of the answer is the same as answer C.

E Discuss Tech Spec bases.

90 F 2 X B S This question is OK as written. The RWCU is plausible since it is one of the alternate decay heat removal systems normally available. Agree with removing Just from distracter A.

Editorial change has been made.

91 F 4 N S D, CR, 43.7 B, CR, 43.4 - Add 10CFR55.43 reference E

92 F 3 N This question is S since the editorial comment has no effect on the question.

S Connected to 55.43(b)(4).

93 F 3 N S D, CR, 43.4 94 H 3 N S A, OR, 43.5 95 F 2 N S A, CR, 43.2 C, CR, 43.10 - Editorial: The attributed 10 CFR 55 section [55.43.(b)(10)] doesnt exist. Review for better match.

U Question does not address reactor/turbine pressure regulating system 96 H 3 X N S Agree with comment except it also does not address a safety related equipment.

Licensee will reject the K/A and reselect for new question.

Question has been revised. It is satisfactory.

97 H 3 N S C, OR, 43.5 B, OR, 43.5 - If the applicant establishes that the plant didnt meet the conditions to E enter EP-3 (Suppression Pool temperature greater than 95F) for answer C or D, then 98 H 4 X N S he/she automatically knows the other is wrong. One of these distracters should be revised. Page 10 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 11 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward K/A Only Agree with comment.

Licensee will resolve issue with question.

Distracter D changed. Question is satisfactory.

99 H 3 N S C, CR, 43.5 A, CR, 41.10 - Reference to 55.43(b)(4) is not correct.

E 100 H 3 N Question is S - Agree with comment.

S Referenced to 55.41(b)(10).

B= 5 F= 32 (43%) E= 21*

Additional Notes: *After review and comment resolution, all questions initially RO TOTALS: M= 0 H= 43 (57%) U= 3*

categorized as needing enhancement (E) or unsatisfactory (U) are satisfactory.

N= 70 B= 3 F= 8 (32%) E= 3*

Additional Notes: *After review and comment resolution, all questions initially SRO TOTALS: M= 0 H= 17 (68%) U= 1*

categorized as needing enhancement (E) or unsatisfactory (U) are satisfactory.

N= 22

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N
2. Chief Examiner comments are indicated in blue.
3. Average difficulty is __ 3__ on the RO exam and 3__ on the SRO exam.
4. The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 1* 43.1 = 0 41.2 = 0 43.2 = 10 41.3 = 2* 43.3 = 0 41.4 = 14 43.4 = 2 41.5 = 2* 43.5 = 9 41.6 = 4* 43.6 = 0 41.7 = 22 43.7 = 1 41.8 = 2 (3 SRO questions 10 CFR 55.43 designation TBD) 41.9 = 1*

41.10 = 27*

41.11 = 2*

41.12 = 2* Page 11 of 12 OBDI 202 - IOLE Process

ES-401 GG-2010-06 12 Form ES-401-9 41.13 = 0 41.14 = 1

  • Questions written credited addressing more than one 10 CFR 41(b) section.
5. The answer distribution is: RO / SRO A= 20 / 6 B= 18 / 5 C= 19 / 6 D= 18 /8
6. There are 10 questions with attachments provided.
7. This review was done by S. Garchow, S Hedger, and T. Pate the week of 5/17/2010.
8. The SRO exam contains 68% higher cognitive questions and this is above the 60% limit. Page 12 of 12 OBDI 202 - IOLE Process
Retrieved from "https://kanterella.com/w/index.php?title=ML101890111&oldid=2131152"