ML093340464

From kanterella
Revision as of 19:31, 16 April 2019 by StriderTol (talk | contribs) (Created page by program invented by StriderTol)
Jump to navigation Jump to search

Comments on Exam - NRC Comments on Draft Licensee-Developed Written Exam
ML093340464
Person / Time
Site: Seabrook NextEra Energy icon.png
Issue date: 08/14/2009
From: Fish T H
Operations Branch I
To:
NextEra Energy Seabrook
Hansell S
Shared Package
ML092330362 List:
References
50-443/09-302, ES-401, ES-401-9, TAC U01798 50-443/09-302
Download: ML093340464 (6)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401*9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. UtE IS 8. Explanation Stem Focus Cues T/F Credo Dist. Partial Job-Link Minutia #1 units ward Q= KIA SRO Only 1 H 3 Y B S 2 H 3 Y M S 3 H 2 N B U KJA MM. Replaced question 4 H 3 Y B S 5 H 4 Y B S 6 H 3 Y B S 7 F 2 Y N S 8 F 2 Y B S 1. 2. 3. 4. 5. 6. 7. 8. Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy -difficult) rating scale (questions in the 2 4 range are acceptable).

Check the appropriate box if a psychometric flaw is identified:

  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).
  • The answer choices are a collection of unrelated truelfalse statements.
  • The distractors are not credible; Single implausible distractors should be repaired, more than one is unacceptable.
  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). Check the appropriate box if a job content error is identified:
  • The question is not linked to the job requirements (I.e., the question has a valid KIA but, as written, is not operational in content).
  • The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
  • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • The question requires reverse logic or application compared to the job requirements.

Check questions that are sampled for conformance with the approved KJA and those that are designated SRO-<Jnly (KIA and license level mismatches are unacceptable).

Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met). ES-401 Written Examination Review Worksheet Form ES-401*9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) SI,m c,"

Job-Minutia #1 Back-Q= SRO B/M/N UtE IS Explanation Focus t. Link units ward KIA Only 1 H 3 Y B S 2 H 3 Y M S 3 H 2 N B U KIA MM. Replaced question 4 H 3 Y B S 5 H 4 Y B S 6 H 3 Y B S 7 F 2 Y EHiB 8 F 2 Y Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question using a 1 -5 (easy -difficult) rating scale (questions in the 2 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:
  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
  • The stem or distractors contain cues (Le., clues, specific determiners, phrasing, length, etc).
  • The answer choices are a collection of unrelated truelfalse statements.
  • The distractors are not credible; Single implausible distractors should be repaired, more than one is unacceptable.
  • One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified:
  • The question is not linked to the job requirements (Le., the question has a valid KIA but, as written, is not operational in content).
  • The question requires the recall of knowledge that is too specific for the closed reference test mode (Le., it is not required to be known from memory).
  • The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  • The question requires reverse logic or application compared to the job requirements.
5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-<Jnly (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. utE IS 8. Explanation Stem Focus Cues T/F Credo Dis!. Partial Job-Link Minutia #1 units ward Q= KIA SRO Only 9 H 3 Y N S 10 H 3 Y M S 11 H 3 Y M S 12 H 4 Y M S 13 H 2 Y M S 14 F 2 Y B S 15 H 3 Y M E Added "First" after the word crew 16 H 3 Y B S 17 F 2 Y B S 18 H 3 Y M S 19 H 4 Y B S 20 H 4 Y M E 21 H 2 Y B S 22 H 3 Y N S 23 2 4 N B U KIA MM. Replaced question and/or KIA 24 F 3 Y B S 25 H 3 Y N S 26 H 3 Y B S 27 F 3 Y B S 28 F 3 Y B S ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) 8tem Cues T/F Credo Partial Job-Minutia #1 Back-Q= 8RO B/M/N utE IS Explanation Focus Dis!. Link units ward KIA Only 9 H 3 Y N 8 10 H 3 Y M S 11 H 3 Y M S 12 H 4 Y M 8 13 H 2 Y M 8 14 F 2 Y B * 'F;",' ,fie, ,,, wo,' "',. 15 H 3 Y M H 3 EE y B F 2 Y B S 18 H 3 Y M S 19 H 4 Y B S 20 H 4 Y M E 21 H 2 Y B S 22 H 3 Y N S 23 2 4 N B U KIA MM. Replaced question and/or KIA 24 F 3 Y B 8 25 H 3 Y N 8 y B 8 Y B 8 Y B 8 ES-401 2 Form ES-401-9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Credo Dis!. Partial Job-Link Minutia #1 units ward Q= KIA SRO Only 29 H 2 Y B S 30 H 2 Y N S 31 F 3 Y B S 32 F 3 Y B S 33 H 2 Y M S 34 H 2 Y B S 35 H 2 Y B S 36 H 3 Y B S 37 H 4 Y B S 38 H 4 Y M S 39 H 3 Y M S Question matches part (a) of KIA 40 H 2 Y B S 41 H 2 Y B S 42 H 2 Y B S 43 F 2 Y N S 44 F 3 Y N E Added power level to stem 45 H 3 Y B S 46 F 3 Y B S 47 H 3 Y N E Reorganized answers and distracters 48 F 2 Y B S ES-401 2 Form ES-401-9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dis!. Link units ward KIA Only 29 H 2 Y I 30 H 2 Y 31 F 3 0 Y 32 F 3 Y 33 H 2 Y 34 H 2 Y B 35 H 2 Y B 36 H 3 Y B S 37 H 4 Y B S 38 H 4 Y M S 39 H 3 Y M S Question matches part (a) of KIA 40 H 2 Y B S 41 H Y B S 42 H 2 Y B S 43 F 2 Y N S 44 F 3 Y N E Added power level to stem 45 H 3 Y B S 46 F 3 Y B S 47 H 3 Y N E Reorganized answers and distracters 48 F 2 Y B S ES*401 2 Form ES*401*9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Credo Dist. Partial Job-Link Minutia #1 units ward Q= KIA SRO Only 49 F 3 Y B S 50 H 2 N B U KIA mm; Replaced question 51 F 3 Y B S 52 H 2 Y B S 53 F 3 Y B E Clarified context of choice 'a' 54 H 3 Y B S 55 F 3 Y B S 56 H 3 Y B S 57 H 3 Y B S 58 H 2 Y N S 59 F 2 X N B U KIA MM. No operational validity.

Replaced KIA & Q 60 H 3 Y B S 61 H 2 N N U KIA MM. Replaced question.

62 F 3 Y B S 63 H 3 Y N S 64 H 2 Y B S 65 F 3 Y B S 66 F 2 Y B E Reworded distracters

'a' and 'b' 67 H 2 Y N S 68 F 2 X X Y M U Replaced question ES*401 2 Form ES*401*9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 F 3 Y B S 50 H 2 N B U KIA mm; Replaced question 51 F 3 Y B S 52 H 2 Y B S 53 F 3 Y B E Clarified context of choice 'a' 54 H 3 Y B S 55 F 3 Y B S 56 H 3 Y B S 57 H 3 Y B S 58 H 2 Y N S 59 F 2 X N B U KIA MM. No operational validity.

Replaced KIA & Q 60 H 3 Y B S 61 H 2 N N U KIA MM. Replaced question.

62 F 3 Y B S 63 H 3 Y N S 64 H 2 Y B S 65 F 3 Y B S 66 F 2 Y B E Reworded distracters

'a' and 'b' 67 H 2 Y N S 68 F 2 X X Y M U Replaced question ES*401 2 Form ES*401*9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. UlE/S 8. Explanation Stem Focus Cues T/F Credo Dis!. Partial Job-Link Minutia #1 units ward Q= KIA SRO Only 69 H 3 Y B S 70 F 3 Y B S 71 F 2 Y B S KIA match based on Q referencing fixed rad monitor 72 F 2 Y B S 73 H 2 Y B S 74 F 3 Y B S 75 F 3 Y B S 76 H 2 Y Y B S 77 H 2 Y Y B S 78 H 2 Y Y N S KIA match because malfunction of valve 182 will affect makeup to RCS (seal flow is affected) 79 H 2 Y Y B S 80 H 3 Y Y B S 81 F 2 Y Y B S 82 H 2 Y Y M S 83 F 3 Y Y B S 84 H 3 Y Y B S 85 F 3 Y Y B S 86 H 2 Y Y B S 87 H 3 Y Y N ES*401 2 Form ES*401*9 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N UlE/S Explanation Focus Dis!. Link units ward KIA Only 69 H 3 Y B S 70 F 3 Y B S 71 F 2 Y B S KIA match based on Q referencing fixed rad monitor 72 F 2 Y B S 73 H 2 Y B S 74 F 3 Y B S 75 F 3 Y B S 76 H 2 Y Y B S 77 H 2 Y Y B S 78 H 2 Y Y N S KIA match because malfunction of valve 182 will affect makeup to RCS (seal flow is affected) 79 H 2 Y Y B S 80 H 3 Y Y B S 81 F 2 Y Y B S 82 H 2 Y Y M S 83 F 3 Y Y B S 84 H 3 Y Y B S 85 F 3 Y Y B S 86 H 2 Y Y B S 87 H 3 Y Y N Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Credo Dis!. Partial Job-Link Minutia #1 units ward Q= KIA SRO Only 88 H 2 Y Y B S Question matches part (b) of KIA 89 H 2 Y Y N S 90 H 3 Y Y M S 91 H 2 Y Y M S Question matches part (b) of KIA 92 H 3 Y Y M S Question matches part (b) of KIA 93 H 3 Y Y B S Question matches part (b) of KIA 94 F 3 Y Y B S Q is valid at Seabrook based on recent OE 95 F 2 X Y Y B U Distracters

'a' and 'c' implausible.

Replaced distracters.

96 F 1 X Y Y B U LOD=1. Distracters

'c' and 'd' implausible, replaced distracters 97 F 2 Y Y B S 98 F 2 Y Y M S 99 H 3 Y Y B S 100 F 2 Y Y B S 1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8. Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job-Minutia #1 Back-Q= SRO B/M/N U/E/S Explanation Focus Dis!. Link units ward KIA Only 88 H 2 Y Y B S Question matches part (b) of KIA 89 H 2 Y Y N S 90 H 3 Y Y M S 91 H 2 Y Y M S Question matches part (b) of KIA 92 H 3 Y Y M S Question matches part (b) of KIA 93 H 3 Y Y B S Question matches part (b) of KIA 94 F 3 Y Y B S Q is valid at Seabrook based on recent OE 95 F 2 X Y Y B U Distracters

'a' and 'c' implausible.

Replaced distracters.

96 F 1 X Y Y B U LOD=1. Distracters

'c' and 'd' implausible, replaced distracters 97 F 2 Y Y B S 98 F 2 Y Y M S 99 H 3 Y Y B S 100 F 2 Y Y B S

Retrieved from "https://kanterella.com/w/index.php?title=ML093340464&oldid=1465635"