ML110110502

From kanterella
Jump to navigation Jump to search

NRC Comments on Draft Licensee-Developed Exam (Written & Operating Tests) (Folder 2)
ML110110502
Person / Time
Site: Vermont Yankee Entergy icon.png
Issue date: 10/13/2010
From:
Entergy Nuclear Operations
To: D'Antonio J
Operations Branch I
Hansell S
Shared Package
ML102070016 List:
References
TAC U01769
Download: ML110110502 (17)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. B.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 h 3 e B implausible 2 h 3 s 3 f 2 s 4 h 2 2 5 f 2 x u Two correct answers b,c. Reduce EDG load?

6 f 3 s 7 f 2 e Unclear wording B h 3 s Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.)

7. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

B. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

9. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

  • The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.
  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:
    • The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

    • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer'S judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

B. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 8ack- Q= SRO 8/M/N utE/S Explanation Focus Dist. Link units ward KIA Only 9 f 2 x u Change level 10 f 2 s 11 h 3 e Swap HPCI/RCIC in 8 12 f 3 s 13 h 2 2 14 h 3 s 15 h 2 s 16 f 2 e Verify and explain wrong answers (d)

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

10. Enter the level of knowiedge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
11. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
12. Check the appropriate box if a psychometric flaw is identified:
  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowiedge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

5.

  • The question requires reverse logic or application compared to the job requirements.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6. Enter question source: (8)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix 8 psychometric attributes are not being met).

ES*401 Written Examination Review Worksheet Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/M/N utE/S Explanation Focus Dist. Link units ward KIA Only 17 f 2 s 18 f 2 s 19 h 3 x u A, B not plaus ALL Pps start; Change Q to running and injecting 20 h 2 e 21 h 2 s 22 h 3 s 23 h 3 s 24 h 2 s Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

13. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
14. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
15. Check the appropriate box if a psychometric flaw is identified:
    • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

4.

  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

5.

  • The question requires reverse logic or application compared to the job requirements.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-on/y (KIA and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q= SRO B/M/N utE/S Explanation Focus Oist. Link units ward KIA Only 25 f 2 s 26 h 2 x e D too severe 27 h 2 e EPR OUTPUT STROKE per procedure 28 h 2 s Validate on site 29 f 2 s 30 f 2 x u C,O implausible source 31 f 2 s 32 f 2 x e Have distracters involving ability to reopen valves/c implausible Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

16. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
17. Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
18. Check the appropriate box if a psychometric flaw is identified:
    • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

    • The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

4.

  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

Check the appropriate box if a job content error is identified:

    • The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

  • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

5.

  • The question requires reverse logic or application compared to the job requirements.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES*401 Written Examination Review Worksheet Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN utEtS Explanation Focus Dist. Link units ward KIA Only 33 h 2 x e Predict impact? What high temperature?

34 h 2 s 35 f 2 e Window dressing 36 f 2 s 37 f 2 s Comment, keep 38 f 1 x U 39 h 2 s 40 h 2 s Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

19. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
20. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
21. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated trueifalse statements.

The distractors ,are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:
    • The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).

    • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (Blank, (M)odified, or (N)ew. Check that {M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written {Ulnsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 41 h 2 s 42 h 3 e wording 43 f 2 s f not h 44 h 3 s Verify RO knowledge 45 f 3 s 46 f 2 s 47 h 2 s 48 h 2 x u A, C change to both systems cst suction close when torus starts to open or full open (different forHPCli, RCIC)

Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

22. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
23. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy difficult) rating scale (questions in the 2 - 4 range are acceptable).
24. Check the appropriate box if a psychometric flaw is identified:
    • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (I.e., clues, specific determiners, phrasing, length, etc).

    • The answer choices are a collection of unrelated truelfalse statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

4.

  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

Check the appropriate box if a job content error is identified:

    • The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (I.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES*401 Written Examination Review Worksheet Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO BIMIN utEtS Explanation Focus Oist. Link units ward KIA Only 49 h 2 e 2nd power supply q. Give Pp A ols to upgrade 50 f 2 s 51 h 4 s Verify RO knowledge 52 f 2 s 53 f 2 s 54 h 2 e Valve names in stem; 30&2 vice immediately 55 f 2 e Relation to 4 rod display? Perhaps use bank Q kla.

56 h 3 x u A, bypass valves open vs close Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

25. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive leve/.
26. Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
27. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent. more information is needed, or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length. etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; Single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:
    • The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are deSignated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (Blank, {M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section 0.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of {E)ditorial enhancement, or {S)atisfactory?
8. At a minimum, explain any "U* ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues TIF Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utEIS Explanation Focus Dis!. Link units ward KIA Only 57 h 3 e Remove "only" from b, d 58 f 2 s 59 h 2 s 60 f 2 x x u Stating insert in stem makes b. dimplausible 61 f 3 s 62 h 2 e Plausibility of d?

63 f 2 s 64 f 2 s Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

28. Enter the level of knowiedge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
29. Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy difficult) rating scale (questions in the 2 - 4 range are acceptable).
30. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent. more information is needed. or too much needless information).

The stem or distractors contain cues (Le., clues, specific determiners. phrasing, length. etc).

The answer choices are a collection of unrelated trueffalse statements.

    • The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (Le., the question has a valid KIA but. as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

    • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified. or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section 0.2.1.
7. Based on the reviewer'S judgment. is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet Form ES-401-9

4. Job Content Flaws 5. Other J 6. 8.

I LOK I LOD I Q# I I T/F ICredo Partial Job- Minutia #1 Dist. Link Sack-units ward Q= SROI S/M/N KIA Only Explanation 65 f 2 s 66 f 1 s keep 67 h 2 s 68 f 1 u Provide an RWP ask a question. Perhaps use for 66 also?

69 h 3 70 h 2 2

3 Explanation B refers to A RHR. distractor is B RHR 3

3 Technical problem 2

Instructions

ES*401 Written Examination Review Worksheet Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F CreeL Partial Job- Minutia #1 Back- Q= SRO B/MIN UlEtS Explanation Focus Dist. Link units ward KIA Only 76 h 2 x s Why would level produce actual vs ind power delta?

77 h 2 s 78 h 3 s 79 h 2 s 80 f 2 s 81 h 2 s 82 h 1 u No main condenser eliminates 3 answers 83 h 2 s Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (I.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated truelfalse statements.

    • The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:
    • The question is not linked to the job reqUirements (Le., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (I.e., it is not required to be known from memory).

    • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

S. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U* ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues TtF Credo Partial Job- Minutia #1 Back- Q= SRO B/MIN UtE/S Explanation Focus Dist. Link units ward KIA Only 84 h 3 check 85 h 1 DLO, rework without reference 86 h 2 s 87 h 2 e D implausible 88 h 2 s 89 h 2 x u k/aMM 90 f 2 s 91 f 2 x U k/a MM Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

4. Enter the level of knowiedge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
5. Enter the level of difficulty (LOD) of each question using a 1 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).
6. Check the appropriate box if a psychometric flaw is identified:
    • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (I.e., clues, specific determiners, phrasing, length, etc).

    • The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

4.

  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

Check the appropriate box if a job content error is identified:

  • The question is not linked to the job requirements the question has a valid KIA but, as written. is not operational in content).
    • The question requires the recall of knowledge that too for the closed reference test mode (I.e.* it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g .* panel meter in percent with question in gallons).

5.

  • The question requires reverse logic or application compared to the job requirements.

Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).

6. Enter question source: (8)ank. (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement). in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any *u" ratings (e.g .* how the Appendix B psychometric attributes are not being met).

ES*401 Written Examination Review Worksheet Form ES*401*9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utE/S Explanation Focus Dis!. Link units ward KIA Only 92 f 1 u DLO 93 h 3 e Verify adequate info in stem to get answer 94 h 2 s 95 h 2 x U k/aMM  !

96 h 2 s 97 h 3 s 98 f 2 x U Implausible distractors 99 f 2 s 100 h 3 s Instructions

  • Reactor Operator Administrative JPMs o A1a: changed the initial conditions and initiating cue to support a shorter JPM.

o A1 b: modified step 13 to allow evaluator to get a better idea of whether or not the candidate calculates period correctly; corrected temperature indication in step 12 (145F) o A2: added "pump" to initial conditions o A4: modified initiating cue to state to make announcements lAW Figure 1; revised step 2 to state fire alarms in zones 27, 28, 29, and 30 vice just zones 27 and 29.

  • Senior Reactor Operator Administrative JPMs o A1a: Changed the initial conditions and initiating cue to support the candidate determining which of three operators meets the requirements to resume duties as an on shift CRS; removed the specific portions of AP 0151 in the initiating cue- determined that this was leading the candidate too much.

o A1 b: clarified that SAG-2 has been entered in the initial conditions and NOT just the SAGs.

o A2: modified step 6 standard to make it more specific; corrected 2 problems with paperwork from surveillance (stroke time for CS-12A and initial for step 9.12).

o A3: Revised initial conditions to state that radiation level in the RB do not permit access to the Refuel Floor and SW system is unavailable; added evaluator note to step 1; added total does received to evaluator note in step 5.

o A4: Added PAR to the JPM as requested.

  • Reactor Operator/Senior Reactor Operator In-Plant JPMs o P1: added 500 KVAR (out) to the initial conditions at request to OM; at request of SH and JT, truncated the JPM with step 10 being the last evaluation step.

o P2: No changes o P3: changed the initiating cue at the request of OM- removed the statement to reset PPP trips and replaced it with a statement to perform section "F". This removed the confusion over how much of the section to perform; clarified the evaluator cue for step 4 to describe the MG output breaker position.

  • Reactor Operator/Senior Reactor Operator Simulator JPMs o S1: matched initial conditions & initiating cue for the evaluator with that provided on the student handout; added 2 Simulator setup instructions; o 52: changed pressure in the initial conditions to account for a slight drift in pressure when the MPR malfunction is inserted. This will minimize the chance that the candidate will incorrectly diagnose the malfunction and want to shut MSIVs as well as scram the reactor (changed from 984-7 976 psig).

o 53: added an evaluator note to step 13 identifying how long the UNSAT closure time is based on the malfunction severity.

o S4, 55, and S6 will be run simultaneously.

o S4: broke up step 3 standard into 2; added evaluator note in step 7 to identify what CRO parameters are; clarified the Simulator setup for how to reset between candidates with three JPMs being conducted at once.

o S5: clarified the Simulator setup for how to reset between candidates with three JPMs being conducted at once; added that the test from the front standard is not required in the initiating cue.

o 56: clarified the Simulator setup for how to reset between candidates with three JPMs being conducted at once.

o 57: Removed the cue to secure the "A" RFP in the initiating cue; changed RFPs to condensate pumps in the standard of step 2; o S8: modified initial conditions and initiating cue to state Loss of Bus 3 procedure vice Loss of Bus 8 procedure; added to start drywell RRUs in Simulator setup to ensure drywell pressure doesn't reach 2.5 psig during electric plant shift.

Scenario modifications:

Scenario #1 :

  • Changed BOP name to CRS on page 4
  • Added specific RE instruction for conduct of the power reduction
  • Changed the order of the events to have the Bus logic failure first in the scenario
  • Changed the summary order
  • Changed the order of event and keys
  • Clarified how long after the scram to insert key 5 and key 6
  • Changed the typo from "An EDG to "B" EDG
  • Changed AC-6 nomenclature to SGT-6
  • Added timeline criterion and time slots for critical tasks
  • Placed critical tasks with the applicable evaluation step
  • Added contingency for cross tying buses 6 and 7

Scenario #2:

  • Removed CCT-1: there was no critical aspect for recognizing an LSSS had been exceeded due to an instrument malfunction.
  • Added verification in Simulator setup that the Recirculation flow controller be set at 81.00 and power verified ~100%.
  • Swapped events 4 and 5 in timeline and throughout the scenario
  • Added discussion & instruction for when to allow CRD-22 to be closed to get rods in. This would force the OATC to attempt another Appendix to insert Control Rods.
  • Added timeline criterion and time slots for critical tasks
  • Placed critical tasks with the applicable evaluation step
  • Put in specific Appendix "GG" terminate and prevent details
  • Changed critical task time for Terminate and Prevent from 5 minutes to 7 minutes due to a FRV being in manual from previous event.
  • Changed critical task time for insertion of control rods from 10 minutes to 15 minutes due to a multiple failures with SLC and CRD-22.
  • Added specificTS references for the loss of Bus 8 Scenario #3:
  • Changed when KEY 5 is inserted
  • Corrected typos
  • Added timeline criterion and time slots for critical tasks
  • Placed critical tasks with the applicable evaluation step
  • Added specific TS references for the loss of Bus 8
  • Added details for where and how Appendix "AA" is performed and added it to the scenario
  • Removed component credit for event 5; it's a major evolution for the crew therefore candidates cannot receive credit for the component failure as well Scenario #4 {SPARE}:
  • Added key 6 to delete the SLC tank leak malfunction one the leak is isolated
  • Added step for the BOP to continue monitoring turbine warm-up parameters during rod withdrawal
  • Removed IRM "A" being out of service to start scenario. We wouldn't startup in this condition
  • Added noun names for valves and defined obscure acronyms throughout
  • Added timeline criterion and time slots for critical tasks
  • Placed critical tasks with the applicable evaluation step
  • Put in specific Appendix "GG" terminate and prevent details
  • Deleted evaluation to use the Aux FRV to control level during an LNP
  • Written exam questions:

o NRC requested changes:

  • Question #33: Redraw (Applicable KIA to the facility); redrew A2.15 and wrote new question; different from JPM in that the cause is Refuel Floor high radiation and NOT reactor building high radiation. Additionally the failure mode of the JPM dealt with RB ventilation failing to isolate.
  • Question #60: NEW (Better KIA match); new question from the exam bank

(#3635) to match KIA; Question went from K based to CIA based and from New to Bank.

  • Question #68: NEW (Level of difficulty)
  • Question #83: Modify distractor "A"- removes SAG procedural direction and kept the procedural direction lAW EOP-3. Could not add recommendation to add distractor dealing with OWSIL curve in UNSAFE region due to drywell temperature would be above 280 in the unsafe region therefore an RPV-EO would be required in that case as well. Added Sprays for distractor "A" and modified the answer ("B") to include sprays AND RPV-EO o Station requested changes following BEFORE validations:
  • Question #15: Reworded step and distractors for better readability.

(enhancement)

  • Question #28: Added a note to state that the "c" phase was the ONLY overcurrent condition. Since 51B has a lower setting, this would have resulted in the overload alarm and another correct answer. (fixed potential for 2 correct answers)
  • Question #56: removed the word "constant" in answers "C" and "0" since it maintains pressure in the band not at a specific constant pressure.

(enhancement)

  • Question #67: removed the word "scrammed" and changed it to match the wording in Technical Specifications "shutdown" (enhancement)
  • Question #70: reformatted stem bullet (enter) for readability (enhancement)
  • Question #71: swapped LEL levels in "c" and "0" to make for feasible to an actual condition that might exist in Containment following de-inerting. Answer changed to "B" based on this change. (enhancement)
  • Question #85: removed the capital "B" on bypass in "c" and "0" (enhancement)
  • Question #86: removed the undervoltage trip and replaced with electrical trip; changed 480 to 120V in stem; removed "bus" in "A" and "c" (enhancement)
  • Question #97: put in table format to better align the numbers. (enhancement)
  • Question #100: replaced the word "verified" in "N'tC" with "confirmed/initiated" (enhancement)

o Station requested changes following FOLLOWING validations:

  • Question #5 MODIFIED: changed correct answer to make it match what the operators relate it to most (required flow to EDGs lAW ON 3148 discussion)
  • Question #9 NEW: KIA redraw to AK 2.04 due to not being able to write an operationally valid question from previous KIA; NEW question. ES 401-1 and 401-4 forms updated.
  • Question #12 ENHANCEMENT: capitalized "AIR" to avoid misreading "AC" which also has a system interrelationship with MSIVs.
  • Question #16 ENHANCEMENT: removed "OP 3020" from stem- validators believe that this was superfluous information that made the question confusing.
  • Question #20 MODIFIED: Changed initial power. Validators believed that 87%

was too close to the high power run back setpoint. Changed to 80% and ran on the Simulator to get data to support a jet pump failure. Also ran loss of condensate pump to see what the level/power transient would be.

  • Question #34 ENHANCEMENT: Changed first sentence in the stem to state that house loads are on the Startup Transformers. Combined two sentences in distractor "8" for readability. Added "if any" to the stem for 80P actions to support distractor "8" not having 80P actions.
  • Question #38 NEW: Replaced the question based on validators believing this is a trick question/obscure knowledge of SRM/RPS interrelationship. KIA remains the same.
  • Question #48 MODIFIED: Changed distractor "8" setpoinf' from 300% to 290%

based on 300% being an older setpoint. 290% would be below both the new and old setpoints.

  • Question #74 NEW: !\lew question based on conflicting information in OP 0105 and OP 2115 concerning when the 24 hour2.777778e-4 days <br />0.00667 hours <br />3.968254e-5 weeks <br />9.132e-6 months <br /> clock starts for having Containment inerted. KIA remains the same.
  • Question #80 MODIFIED: Changed distractor "D" based on being too close to expected damage to RTF-5; reworded the stem for readability; added RTF-5 to answer "C".

Post validation summary:

  • 3 new questions
  • 5 modified questions
  • 4 question enhancements
Retrieved from "https://kanterella.com/w/index.php?title=ML110110502&oldid=2552558"