ML17003A282

From kanterella
Jump to navigation Jump to search
NRC Comments on Draft Licensee-Developed Written Exam (Folder 2)
ML17003A282
Person / Time
Site: Beaver Valley
Issue date: 10/24/2016
From:
NRC Region 1
To:
Division of Reactor Safety I
Shared Package
ML16076A416 List:
References
U01922
Download: ML17003A282 (6)
v • d • e


Text

ES-401 BEAVER VALLEY 1 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 1 H 2 N s 2 H 2 N s 3 H 2 N s 4 F 3 N s 5 H 2 x N E Typo in 3rd bullet (Reactor vs React). Add "ONLY" to A 1& B 1.

6 H 2 N s 7 F 2 N s 8 H 2 x 0 B* E Replace o.

Instructions

[Refer to Section O of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOO) of each question using a 1 - 5 (easy- difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:

... The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information) .

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc) .

.. The answer choices are a collection of unrelated true/false statements .

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable .

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem) .

4.

..Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid KIA but, as written, is not operational in content) .

.. The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory) .

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons) .

The question requires reverse logic or application compared to the job requirements .

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section 0.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 9 H 2 x N E Pictures need to be bigger/clearer.

10 F 2 B N E B is a weak distractor. Replace.

11 F 2 x B E Clarify 3rd bullet and reword 2nd Q.

12 H 2 B* s 13 H 2 x N E Delete 3rd bullet.

14 F 2 x N E Add"ONLY"toA1 &C1.

15 F 2 B s 16 H 2 x N E Add IAW FR-H.1 to 2nd Q. Ensure only one possible answer.

17 F 2 B/D B u B1/D1 not plausible for ECA-1.1 condition. Replace.

18 F 3 x N E Replace "reason" with "coincidence and logic".

19 F 2 x N E Reduce wording.

20 F 2 N s 21 F 2 x N E Minimize wording in stem.

22 F 3 c N E Modify first part of C (i.e., perform power reduction).

23 H 3 N s 24 F 2 B s 25 H 2 B s 26 H 2 x M E Remove recirc sptay pumps from choice D.

27 F 2 B s 28 H 2 B s

ES-401 3 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 29 H 2 x B E 1st set of bullets not necessary. Delete.

30 H 2 x B E Minimize/reword for clarity.

31 H 2 x B,C B u Just say blockage occurs in Hx. Replace B & C.

32 F 2 x N E Remove "coincidence/coincident" from Q. Add "signal".

33 H 2 B s 34 H 2 N s 35 F 3 x x N u Eliminate bullets. Address T/F statements. Reword.

36 H 2 x B E Make answer be something other than "Steam is Saturated".

37 H 2 x N E Change Q to diagnose spray valve being open.

38 F 3 B s 39 H 3 B s 40 F 2 N s 41 F 2 B,D B u Eliminate 4th item. Not plausible that that valve would close.

42 F 2 B N E Steam dumps not addressed by TS. Replace with TS-related item.

43 H 2 x B E Delete parenthetical phrase.

44 F 2 x N E Add wording to A to have the same length as other distractors.

45 F 2 N s 46 H 2 B s 47 H 2 N s

,,j8 H 2 N s

ES-401 4 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO 8/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 49 F 3 8 s 50 F 3 x N E Add "at least" to 2nd part to eliminate possible subset issue.

51 H 2 M s 52 F 2 x N E Re-word question for simplicity.

53 H 2 N s 54 H 2 8 s 55 F 2 8 s -

56 H 2 N s 57 H 2 N s 58 H 2 x N E Reword Q for simplicity.

59 F 2 8 s 60 F 2 x N E L vs H. Specify time frame.

61 H 2 N s 62 H 3 x N E Clarify statement about ineffective control room actions. Replace C.

63 F 2 N s 64 H 2 x N u There are 4 or 5 other GFE-type Q's on exam. Reselect.

65 F 2 N s 66 F 2 x 8 E Capitalize a word in each distractor.

67 H 3 N s hJ38 F 2 x 8 E Spell out RPS.

ES-401 5 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #I Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 69 F 2 B s 70 F 2 B s 71 H 2 B s 72 F 2 B E 73 F 2 D N E D not plausible.

74 H 3 B s 75 H 2 N s H=39->52% F=36->48% U=5->6.7%

76 H 2 x N E Provide indications of RCP trip criteria. Minimize wording.

77 H 2 N s 78 H 2 x N u Q can be answer with systems knowledge. Reword or reselect.

79 H 3 B s 80 H 3 x N E Delete "3.5.1". Add "at latest" to specify time frame.

81 H 2 B s 82 H 2 x N E System Q with and EAL twist. Reword or reselect.

83 F 2 N s 84 H 2 B s 85 H 2 B s 86 F 2 N s

ES-401 6 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only 87 F 2 N s 88 H 2 C,D B u Reducing AFW flow not plausible. Replace C & D 89 H 2 N 90 H 2 x N E Plant in Mode 5 (vs 4). Enter Mode 4 (vs 3).

91 F 1 x B u Low discriminatory value. KAMM - no mention of Ni's.

92 H 2 x N u KAMM. Q should be about a LOCA vs SGTR.

93 H 2 x x N E Reword to avoid stating that ASDV is open. Minimize wording.

94 F 2 B s 95 H 2 B s 96 H 2 x B E Replace 3'd bullet with a condition that requires assessment.

97 H 2 x B E Capitalize wording in B.

98 L 2 D N E Replace D - "1 only".

99 L 2 x B E Typo 7th bullet -fuel vs Fuel.

100 H 2 B s

Retrieved from "https://kanterella.com/w/index.php?title=ML17003A282&oldid=2441620"