ML15293A556: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
Line 15: Line 15:


=Text=
=Text=
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 1 H 3            N S  2 H 3            B S Used on 1999 NRC exam 3 H 2            B S  4 H 3            N S  5 H 3            N S  6 H 2            B S  7 F 2            N S  8 F 2            B S  Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2  and 4 are acceptable) 3. Check the appropriate box if a psychometric flaw is identified: "Stem Focus":  The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).  "Cues":  The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).  "T/F":  The answer choices are a collection of unrelated true/false statements.  "Cred. Dist":  The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.  "Partial":  One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: "Job Link":  The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). "Minutia":  The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). "# / Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). "Backward":  The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable) 6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
{{#Wiki_filter:ES-401 Written Examination Review Worksheet Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
ES-401 2 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 9 H 3            N S  10 H 3            N S  11 F 2            N S  12 H 3            B S Used on 1999 NRC exam 13 H 3            N S  14 H 3            B S  15 H 3            N S  16 H 3            N S  17 H 3            B S  18 F 2            N S  19 H 3            N S  20 H 3            N S  21 H 3            N S  22 H 3            N S  23 F 2            B S  24 H 3            N S  25 F 2            B S ES-401 3 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 26 F 3            N S  27 H 3            B S Used on 2010 NRC exam 28 F 2            N S  29 H 3            N S  30 H 3            M S Used on 2009 NRC exam 31 H 3            B S Used on 2009 NRC exam 32 H 3            B S Used on 2003 NRC exam 33 H 3            N S  34 F 2            N S  35 F 2            N S  36 H 3            B S Used on 2009 NRC exam 37 H 3            N S  38 H 2            B S  39 H 3            B S Used on 2007 NRC exam 40 F 3            B S  41 H 3            N S  42 H 4            N S ES-401 4 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 43 F 3            N S  44 F 3            N S  45 F 2            B S Used on 2005 NRC exam 46 F 3            B S Used on 2010 NRC exam 47 H 3            N S  48 H 3            N S  49 H 3            B S  50 H 3            N S  51 F 3            B S Used on 2007 NRC exam 52 H 3            N S  53 F 2            M S  54 H 4            N S  55 F 3            N S  56 F 2            N S  57 F 2            N S  58 H 3            B S Used on 2005 NRC exam 59 F 2            N S ES-401 5 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 60 H 3            N S  61 H 3            B S Used on 2009 NRC exam 62 F 2            N S  63 F 2            B S  64 H 3            N S  65 H 4            N S Used on 1995 NRC exam 66 F 3            N S  67 F 2            N S  68 F 2            N S  69 F 2            N S  70 F 4            N S  71 F 2            N S  72 H 3            M S  73 F 2            N S  74 H 3            N S  75 F 2            N S  76 H 4            B S Used on 2001 NRC exam ES-401 6 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 77 F 3            N S  78 H 3            B S Used on 2011 NRC exam 79 F 3            B S  80 F 4            N S  81 H 3            N S  82 H 3            N S  83 F 3            N S  84 H 3            B S  85 H 3            N S  86 F 3            N S  87 H 3            B S Used on 2005 NRC exam 88 F 4            N S  89 H 3            B S Used on 2007 NRC exam 90 H 3            B S Used on 2007 NRC exam 91 H 3            N S  92 H 3            N S  93 F 2            N S ES-401 7 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 94 H 3            B S  95 H 3            N S  96 F 2            N S  97 F 2            N S  98 F 3            N S  99 H 3            N S  100 F 3            B S Used on the 1997 NRC exam ES-401 Written Examination Review Worksheet Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 1 H 3            N S  2 H 3            B S Used on 1999 NRC exam 3 H 2            B S  4 H 3            N S  5 H 3            N S  6 H 2            B S  7 F 2            N S  8 F 2            B S  Instructions (Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level. 2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2  and 4 are acceptable) 3. Check the appropriate box if a psychometric flaw is identified: "Stem Focus":  The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).  "Cues":  The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).  "T/F":  The answer choices are a collection of unrelated true/false statements.  "Cred. Dist":  The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.  "Partial":  One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem). 4. Check the appropriate box if a job content error is identified: "Job Link":  The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). "Minutia":  The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). "# / Units": The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). "Backward":  The question requires reverse logic or application compared to the job requirements. 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO-only (K/A and license level mismatches are unacceptable) 6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-401 Section D.2.f. 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory? 8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
: 4. Job Content Flaws
ES-401 2 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 9 H 3            N S  10 H 3            N S  11 F 2            N S  12 H 3            B S Used on 1999 NRC exam 13 H 3            N S  14 H 3            B S  15 H 3            N S  16 H 3            N S  17 H 3            B S  18 F 2            N S  19 H 3            N S  20 H 3            N S  21 H 3            N S  22 H 3            N S  23 F 2            B S  24 H 3            N S  25 F 2            B S ES-401 3 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 26 F 3            N S  27 H 3            B S Used on 2010 NRC exam 28 F 2            N S  29 H 3            N S  30 H 3            M S Used on 2009 NRC exam 31 H 3            B S Used on 2009 NRC exam 32 H 3            B S Used on 2003 NRC exam 33 H 3            N S  34 F 2            N S  35 F 2            N S  36 H 3            B S Used on 2009 NRC exam 37 H 3            N S  38 H 2            B S  39 H 3            B S Used on 2007 NRC exam 40 F 3            B S  41 H 3            N S  42 H 4            N S ES-401 4 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 43 F 3            N S  44 F 3            N S  45 F 2            B S Used on 2005 NRC exam 46 F 3            B S Used on 2010 NRC exam 47 H 3            N S  48 H 3            N S  49 H 3            B S  50 H 3            N S  51 F 3            B S Used on 2007 NRC exam 52 H 3            N S  53 F 2            M S  54 H 4            N S  55 F 3            N S  56 F 2            N S  57 F 2            N S  58 H 3            B S Used on 2005 NRC exam 59 F 2            N S ES-401 5 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 60 H 3            N S  61 H 3            B S Used on 2009 NRC exam 62 F 2            N S  63 F 2            B S  64 H 3            N S  65 H 4            N S Used on 1995 NRC exam 66 F 3            N S  67 F 2            N S  68 F 2            N S  69 F 2            N S  70 F 4            N S  71 F 2            N S  72 H 3            M S  73 F 2            N S  74 H 3            N S  75 F 2            N S  76 H 4            B S Used on 2001 NRC exam ES-401 6 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 77 F 3            N S  78 H 3            B S Used on 2011 NRC exam 79 F 3            B S  80 F 4            N S  81 H 3            N S  82 H 3            N S  83 F 3            N S  84 H 3            B S  85 H 3            N S  86 F 3            N S  87 H 3            B S Used on 2005 NRC exam 88 F 4            N S  89 H 3            B S Used on 2007 NRC exam 90 H 3            B S Used on 2007 NRC exam 91 H 3            N S  92 H 3            N S  93 F 2            N S ES-401 7 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. Source (B/ M / N) 7. Status (U /E /S) 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 94 H 3            B S  95 H 3            N S  96 F 2            N S  97 F 2            N S  98 F 3            N S  99 H 3            N S  100 F 3            B S Used on the 1997 NRC exam}}
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 1 H 3            N S  2 H 3            B S Used on 1999 NRC exam 3 H 2            B S  4 H 3            N S  5 H 3            N S  6 H 2            B S  7 F 2            N S  8 F 2            B S  Instructions (Refer to Section D of ES
-401 and Appendix B for additional information regarding each of the following concepts:
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2  and 4 are acceptable)
: 3. Check the appropriate box if a psychometric flaw is identified:
  "Stem Focus":  The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).  
   "Cues":  The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).  
   "T/F":  The answer choices are a collection of unrelated true/false statements.  
   "Cred. Dist":  The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.  "Partial":  One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 4. Check the appropriate box if a job content error is identified:
  "Job Link":  The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  "Minutia":  The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
  "# / Units":
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  "Backward":  The question requires reverse logic or application compared to the job requirements.
: 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO
-only (K/A and license level mismatches are unacceptable)
: 6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES
-401 Section D.2.f.
: 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
 
ES-401 2 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 9 H 3            N S  10 H 3            N S  11 F 2            N S  12 H 3            B S Used on 1999 NRC exam 13 H 3            N S  14 H 3            B S  15 H 3            N S  16 H 3            N S  17 H 3            B S  18 F 2            N S  19 H 3            N S  20 H 3            N S  21 H 3            N S  22 H 3            N S  23 F 2            B S  24 H 3            N S  25 F 2            B S ES-401 3 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 26 F 3            N S  27 H 3            B S Used on 2010 NRC exam 28 F 2            N S  29 H 3            N S  30 H 3            M S Used on 2009 NRC exam 31 H 3            B S Used on 2009 NRC exam 32 H 3            B S Used on 2003 NRC exam 33 H 3            N S  34 F 2            N S  35 F 2            N S  36 H 3            B S Used on 2009 NRC exam 37 H 3            N S  38 H 2            B S  39 H 3            B S Used on 2007 NRC exam 40 F 3            B S  41 H 3            N S  42 H 4            N S ES-401 4 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 43 F 3            N S  44 F 3            N S  45 F 2            B S Used on 2005 NRC exam 46 F 3            B S Used on 2010 NRC exam 47 H 3            N S  48 H 3            N S  49 H 3            B S  50 H 3            N S  51 F 3            B S Used on 2007 NRC exam 52 H 3            N S  53 F 2            M S  54 H 4            N S  55 F 3            N S  56 F 2            N S  57 F 2            N S  58 H 3            B S Used on 2005 NRC exam 59 F 2            N S ES-401 5 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 60 H 3            N S  61 H 3            B S Used on 2009 NRC exam 62 F 2            N S  63 F 2            B S  64 H 3            N S  65 H 4            N S Used on 1995 NRC exam 66 F 3            N S  67 F 2            N S  68 F 2            N S  69 F 2            N S  70 F 4            N S  71 F 2            N S  72 H 3            M S  73 F 2            N S  74 H 3            N S  75 F 2            N S  76 H 4            B S Used on 2001 NRC exam
 
ES-401 6 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 77 F 3            N S  78 H 3            B S Used on 2011 NRC exam 79 F 3            B S  80 F 4            N S  81 H 3            N S  82 H 3            N S  83 F 3            N S  84 H 3            B S  85 H 3            N S  86 F 3            N S  87 H 3            B S Used on 2005 NRC exam 88 F 4            N S  89 H 3            B S Used on 2007 NRC exam 90 H 3            B S Used on 2007 NRC exam 91 H 3            N S  92 H 3            N S  93 F 2            N S ES-401 7 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 94 H 3            B S  95 H 3            N S  96 F 2            N S  97 F 2            N S  98 F 3            N S  99 H 3            N S  100 F 3            B S Used on the 1997 NRC exam
 
ES-401 Written Examination Review Worksheet Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 1 H 3            N S  2 H 3            B S Used on 1999 NRC exam 3 H 2            B S  4 H 3            N S  5 H 3            N S  6 H 2            B S  7 F 2            N S  8 F 2            B S  Instructions (Refer to Section D of ES
-401 and Appendix B for additional information regarding each of the following concepts:
: 1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
: 2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2  and 4 are acceptable)
: 3. Check the appropriate box if a psychometric flaw is identified:
  "Stem Focus":  The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).  
   "Cues":  The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).  
   "T/F":  The answer choices are a collection of unrelated true/false statements.  
   "Cred. Dist":  The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.  "Partial":  One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).
: 4. Check the appropriate box if a job content error is identified:
  "Job Link":  The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).
  "Minutia":  The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).
  "# / Units":
The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).
  "Backward":  The question requires reverse logic or application compared to the job requirements.
: 5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO
-only (K/A and license level mismatches are unacceptable)
: 6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES
-401 Section D.2.f.
: 7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).
 
ES-401 2 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 9 H 3            N S  10 H 3            N S  11 F 2            N S  12 H 3            B S Used on 1999 NRC exam 13 H 3            N S  14 H 3            B S  15 H 3            N S  16 H 3            N S  17 H 3            B S  18 F 2            N S  19 H 3            N S  20 H 3            N S  21 H 3            N S  22 H 3            N S  23 F 2            B S  24 H 3            N S  25 F 2            B S ES-401 3 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 26 F 3            N S  27 H 3            B S Used on 2010 NRC exam 28 F 2            N S  29 H 3            N S  30 H 3            M S Used on 2009 NRC exam 31 H 3            B S Used on 2009 NRC exam 32 H 3            B S Used on 2003 NRC exam 33 H 3            N S  34 F 2            N S  35 F 2            N S  36 H 3            B S Used on 2009 NRC exam 37 H 3            N S  38 H 2            B S  39 H 3            B S Used on 2007 NRC exam 40 F 3            B S  41 H 3            N S  42 H 4            N S ES-401 4 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 43 F 3            N S  44 F 3            N S  45 F 2            B S Used on 2005 NRC exam 46 F 3            B S Used on 2010 NRC exam 47 H 3            N S  48 H 3            N S  49 H 3            B S  50 H 3            N S  51 F 3            B S Used on 2007 NRC exam 52 H 3            N S  53 F 2            M S  54 H 4            N S  55 F 3            N S  56 F 2            N S  57 F 2            N S  58 H 3            B S Used on 2005 NRC exam 59 F 2            N S ES-401 5 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 60 H 3            N S  61 H 3            B S Used on 2009 NRC exam 62 F 2            N S  63 F 2            B S  64 H 3            N S  65 H 4            N S Used on 1995 NRC exam 66 F 3            N S  67 F 2            N S  68 F 2            N S  69 F 2            N S  70 F 4            N S  71 F 2            N S  72 H 3            M S  73 F 2            N S  74 H 3            N S  75 F 2            N S  76 H 4            B S Used on 2001 NRC exam
 
ES-401 6 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 77 F 3            N S  78 H 3            B S Used on 2011 NRC exam 79 F 3            B S  80 F 4            N S  81 H 3            N S  82 H 3            N S  83 F 3            N S  84 H 3            B S  85 H 3            N S  86 F 3            N S  87 H 3            B S Used on 2005 NRC exam 88 F 4            N S  89 H 3            B S Used on 2007 NRC exam 90 H 3            B S Used on 2007 NRC exam 91 H 3            N S  92 H 3            N S  93 F 2            N S ES-401 7 Form ES-401-9  Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
: 4. Job Content Flaws
: 5. Other 6. Source (B/ M / N)
: 7. Status (U /E /S)
: 8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 94 H 3            B S  95 H 3            N S  96 F 2            N S  97 F 2            N S  98 F 3            N S  99 H 3            N S  100 F 3            B S Used on the 1997 NRC exam}}

Revision as of 19:58, 30 June 2018

South Texas Project-2015-10 Draft Written Exam Comments
ML15293A556
Person / Time
Site: South Texas  STP Nuclear Operating Company icon.png
Issue date: 10/02/2015
From: Vincent Gaddy
Operations Branch IV
To:
References
Download: ML15293A556 (7)
v • d • e


Text

ES-401 Written Examination Review Worksheet Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 1 H 3 N S 2 H 3 B S Used on 1999 NRC exam 3 H 2 B S 4 H 3 N S 5 H 3 N S 6 H 2 B S 7 F 2 N S 8 F 2 B S Instructions (Refer to Section D of ES

-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)
3. Check the appropriate box if a psychometric flaw is identified:

"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

"Cues": The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).

"T/F": The answer choices are a collection of unrelated true/false statements.

"Cred. Dist": The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. "Partial": One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

"Job Link": The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

"Minutia": The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

"# / Units":

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

"Backward": The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO

-only (K/A and license level mismatches are unacceptable)

6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES

-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 9 H 3 N S 10 H 3 N S 11 F 2 N S 12 H 3 B S Used on 1999 NRC exam 13 H 3 N S 14 H 3 B S 15 H 3 N S 16 H 3 N S 17 H 3 B S 18 F 2 N S 19 H 3 N S 20 H 3 N S 21 H 3 N S 22 H 3 N S 23 F 2 B S 24 H 3 N S 25 F 2 B S ES-401 3 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 26 F 3 N S 27 H 3 B S Used on 2010 NRC exam 28 F 2 N S 29 H 3 N S 30 H 3 M S Used on 2009 NRC exam 31 H 3 B S Used on 2009 NRC exam 32 H 3 B S Used on 2003 NRC exam 33 H 3 N S 34 F 2 N S 35 F 2 N S 36 H 3 B S Used on 2009 NRC exam 37 H 3 N S 38 H 2 B S 39 H 3 B S Used on 2007 NRC exam 40 F 3 B S 41 H 3 N S 42 H 4 N S ES-401 4 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 43 F 3 N S 44 F 3 N S 45 F 2 B S Used on 2005 NRC exam 46 F 3 B S Used on 2010 NRC exam 47 H 3 N S 48 H 3 N S 49 H 3 B S 50 H 3 N S 51 F 3 B S Used on 2007 NRC exam 52 H 3 N S 53 F 2 M S 54 H 4 N S 55 F 3 N S 56 F 2 N S 57 F 2 N S 58 H 3 B S Used on 2005 NRC exam 59 F 2 N S ES-401 5 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 60 H 3 N S 61 H 3 B S Used on 2009 NRC exam 62 F 2 N S 63 F 2 B S 64 H 3 N S 65 H 4 N S Used on 1995 NRC exam 66 F 3 N S 67 F 2 N S 68 F 2 N S 69 F 2 N S 70 F 4 N S 71 F 2 N S 72 H 3 M S 73 F 2 N S 74 H 3 N S 75 F 2 N S 76 H 4 B S Used on 2001 NRC exam

ES-401 6 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 77 F 3 N S 78 H 3 B S Used on 2011 NRC exam 79 F 3 B S 80 F 4 N S 81 H 3 N S 82 H 3 N S 83 F 3 N S 84 H 3 B S 85 H 3 N S 86 F 3 N S 87 H 3 B S Used on 2005 NRC exam 88 F 4 N S 89 H 3 B S Used on 2007 NRC exam 90 H 3 B S Used on 2007 NRC exam 91 H 3 N S 92 H 3 N S 93 F 2 N S ES-401 7 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 94 H 3 B S 95 H 3 N S 96 F 2 N S 97 F 2 N S 98 F 3 N S 99 H 3 N S 100 F 3 B S Used on the 1997 NRC exam

ES-401 Written Examination Review Worksheet Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 1 H 3 N S 2 H 3 B S Used on 1999 NRC exam 3 H 2 B S 4 H 3 N S 5 H 3 N S 6 H 2 B S 7 F 2 N S 8 F 2 B S Instructions (Refer to Section D of ES

-401 and Appendix B for additional information regarding each of the following concepts:

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question a 1(easy) to 5 ( difficult); (questions with a difficulty between 2 and 4 are acceptable)
3. Check the appropriate box if a psychometric flaw is identified:

"Stem Focus": The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

"Cues": The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc.).

"T/F": The answer choices are a collection of unrelated true/false statements.

"Cred. Dist": The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable. "Partial": One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

"Job Link": The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

"Minutia": The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

"# / Units":

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

"Backward": The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those K/As that are designated SRO

-only (K/A and license level mismatches are unacceptable)

6. Enter question's source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES

-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" Status ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 9 H 3 N S 10 H 3 N S 11 F 2 N S 12 H 3 B S Used on 1999 NRC exam 13 H 3 N S 14 H 3 B S 15 H 3 N S 16 H 3 N S 17 H 3 B S 18 F 2 N S 19 H 3 N S 20 H 3 N S 21 H 3 N S 22 H 3 N S 23 F 2 B S 24 H 3 N S 25 F 2 B S ES-401 3 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 26 F 3 N S 27 H 3 B S Used on 2010 NRC exam 28 F 2 N S 29 H 3 N S 30 H 3 M S Used on 2009 NRC exam 31 H 3 B S Used on 2009 NRC exam 32 H 3 B S Used on 2003 NRC exam 33 H 3 N S 34 F 2 N S 35 F 2 N S 36 H 3 B S Used on 2009 NRC exam 37 H 3 N S 38 H 2 B S 39 H 3 B S Used on 2007 NRC exam 40 F 3 B S 41 H 3 N S 42 H 4 N S ES-401 4 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 43 F 3 N S 44 F 3 N S 45 F 2 B S Used on 2005 NRC exam 46 F 3 B S Used on 2010 NRC exam 47 H 3 N S 48 H 3 N S 49 H 3 B S 50 H 3 N S 51 F 3 B S Used on 2007 NRC exam 52 H 3 N S 53 F 2 M S 54 H 4 N S 55 F 3 N S 56 F 2 N S 57 F 2 N S 58 H 3 B S Used on 2005 NRC exam 59 F 2 N S ES-401 5 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 60 H 3 N S 61 H 3 B S Used on 2009 NRC exam 62 F 2 N S 63 F 2 B S 64 H 3 N S 65 H 4 N S Used on 1995 NRC exam 66 F 3 N S 67 F 2 N S 68 F 2 N S 69 F 2 N S 70 F 4 N S 71 F 2 N S 72 H 3 M S 73 F 2 N S 74 H 3 N S 75 F 2 N S 76 H 4 B S Used on 2001 NRC exam

ES-401 6 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 77 F 3 N S 78 H 3 B S Used on 2011 NRC exam 79 F 3 B S 80 F 4 N S 81 H 3 N S 82 H 3 N S 83 F 3 N S 84 H 3 B S 85 H 3 N S 86 F 3 N S 87 H 3 B S Used on 2005 NRC exam 88 F 4 N S 89 H 3 B S Used on 2007 NRC exam 90 H 3 B S Used on 2007 NRC exam 91 H 3 N S 92 H 3 N S 93 F 2 N S ES-401 7 Form ES-401-9 Q 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws
4. Job Content Flaws
5. Other 6. Source (B/ M / N)
7. Status (U /E /S)
8. Explanation Stem Focus Cues T/F Cred. Dist Partial Job-Link Minutia # / Units Back ward Q - K/A SRO Only 94 H 3 B S 95 H 3 N S 96 F 2 N S 97 F 2 N S 98 F 3 N S 99 H 3 N S 100 F 3 B S Used on the 1997 NRC exam
Retrieved from "https://kanterella.com/w/index.php?title=ML15293A556&oldid=772817"