ML17193A926

Summer (U1) 2016-301 Exam Administrative Items - Delay Release

ML17193A926
Person / Time
Site:	Summer
Issue date:	07/12/2017
From:	NRC/RGN-II
To:	South Carolina Electric & Gas Co
References
Download: ML17193A926 (71)

Text

V.C. Summer 2016

-301 ES-401 Written Examination Review Worksheet Form ES-401-9 Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 1 H 3 X  ? N E K/A 007EK1.02 Question statement needs to be a WOOTF statement like the other questions.

Regarding the SRO only justification for question #76, please explain why this is not SRO only.

4/29/16 - Revised question is satisfactory.

2 H 3 X N E K/A 008AA1.06 Question choices combined with the answer choices are long and difficult to read. Could easily split this up into two questions. Need to ask both questions in accordance with the applicable procedure. Need to discuss this question.

4/29/16 - Revised question is satisfactory. 3 H 3 X B E K/A 009EK3.23 Is it possible for a SBLOCA to cause a 13 psig pressure rise in containment in one minute?

4/29/16 - Revised question is satisfactory.

4 F 3 X M E K/A 011EA2.08 Based on the procedure step, the stem question should read "the CCW train in the active loop," vice "the CCW train with the active loop." 4/29/16 - Revised question is satisfactory.

5 F 3 M S K/A 015AA2.09 Question is Satisfactory.

6 H 3 N S K/A 022AK1.03 Question is Satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 7 F 3 N E K/A 025AA1.11 Are the conditions given operationally valid? If you are below the RWST level where automatic switchover should have automatically occurred, but also below the sump level where ECCS pump suction requirements are not met then it seems to me that this is a very poor plant design or water from the RWST went someplace other than the sump which would lead to choice A also being potentially correct. Verified that conditions would lead to EOP

-2.5 entry.

Question is satisfactory.

8 H 3 B S K/A 026AG2.4.49 Question is Satisfactory.

9 H 2 X X N U K/A 029EK2.06 Since all of the bullets in step 1 of EOP

-1.0 are not met for verifying the reactor tripped, could an applicant argue that the reactor is not tripped? Need to specify in the initial conditions that all or both reactor trip breakers are closed.

Choice B is not a plausible distractor when contrasted with choice A.

Choices D is not plausible. Question is unsatisfactory due to two non-plausible distractors.

5/3/16 - Revised question is satisfactory.

10 H 3 X X N E K/A 038EK1.03 Discussion on page 22 of 34 in the EOP

-4.0 training handout seem s to describe choice D as another correct answer.

4/29/16 - Revised question is satisfactory.

11 F 3 X X B E K/A 040AG2.1.31 Could have XCP 615, 1

-2, RCS TAVG LO in alarm but not yet have a feedwater isolation (1/3 for alarm vice 2/3 for feedwater isolation). I do not believe that both B and C are both plausible distractors since they are at the same setpoint for the same parameter.

4/29/16 - Reviewed revised question. What makes choice A plausible? What caused the reactor trip? Need to further discuss.

5/2/16 - Based on discussion with the licensee, the revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 12 H 3 X X B E K/A 054AK3.01 Question was submitted for preliminary ten sample question review and will not be used for evaluating the quality of the final submittal.

Most issues identified during the sample review were addressed however distractors A and B are still basically the same thing stated with different words.

5/4/2016 - Revised question is satisfactory.

13 H 3 X N E K/A 056AA2.32 The difference between 557 and 559 degrees F is minutia.

4/29/16 - Revised question is satisfactory.

14 H 3 X X  ? N U/E K/A 057AG2.1.19 Need to discuss this question. The first part of the question does not seem to relate to the loss of vital 120VAC power which would mean that the K/A was not matched and would make the question unsatisfactory. Do you have any breakers that are controlled by AC power for plausibility of the distractor? Are the applicants expected to know by memory what APN

-5901 is along with its loads?

K/A 057G.2.1.30 5/5/16 - Revised question is satisfactory.

15 H 3 X X N U K/A 062AK3.03 (This is a replacement K/A.)

What is the significance of reading step 1 of AOP

-117.1? The title of AOP

-117.1 is not TOTAL LOSS OF SERVICE WATER.

Based on step 15 of AOP

-117.1, it appears that cooling to each diesel generator must be manually aligned, therefore the selected answer choice is not fully correct. I understand that the pump starts automatically, but there appear s to be valves that must be manually aligned. The question also does not seem to be worded to ask for a reason, therefore is not a good K/A match. I have a very easy fix that we can discuss.

Question is unsatisfactory due to incorrect answer and K/A mismatch.

4/29/16 - Reviewed the revised question. The question stem asks whether fire service water will be initiated manually or automatically in accordance with the AOP. The AOP directs that it be done manually, but the conditions listed would have caused it to be done automatically. Need to tighten up the question.

5/4/2016 - Revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 16 H 3 X X B E K/A 065AA1.05 Distractor A and C are not plausible

, especially with power at 5%.

5/4/2016 - Revised question is satisfactory.

17 H 3 X M E K/A 077AK2.06 Need a dash in AP

-301.1 in the question statement.

The "a" in the first question statement needs to removed and placed in choices C.1) and D.1).

4/29/16 - Revised question is satisfactory.

18 F 3 X X B E K/A WE05EK2.2 For the plausibility of distractor A with the conditions given, if feed and bleed was not required and sufficient SGWL was available, why would steaming be done through the SG PORVs?

4/29/16 - Revised question is satisfactory.

19 F 3 X X N E K/A 024AK3.01 The note in the Lo

-Lo ARP states that shutdown margin is in question. Choice B.2) plausibility is weak with no AFD indications given. 4/29/16 - Revised question is satisfactory.

20 H 2 N S K/A 036AA2.03 Question is satisfactory.

21 H 3 N S K/A 059AA1.02 Question is satisfactory.

22 F 3 X M U K/A 069AK2.03 This should be K/A 069G2.2.42

. Choice B is not plausible when both questions are considered together. Choice C is not plausible since you have stated that both doors are inoperable due to bad seals.

Question is unsatisfactory due to two non-plausible distractors.

5/5/16 - Revised question is satisfactory.

23 F 3 X B E K/A 074EK3.11 (This is a replacement K/A.)

Should probably complete the sentence and state that a small break LOCA has occurred.

4/29/16 - Revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 24 F 3 X N U K/A WE01EK1.1 Question was submitted for preliminary ten sample question review and will not be used for evaluating the quality of the final submittal.

The ES-401-4 states that this should be K/A WE01EK1.2.

Question does not appear to meet the K/A of the replacement K/A.

Question is unsatisfactory due to K/A mismatch.

5/4/2016 - Revised question is satisfactory.

25 H 3 M S K/A WE03EK2.1 Question is satisfactory.

26 F 3 X N E K/A WE13EA2.1 Choice D is not plausible.

5/4/2016 - Revised question is satisfactory.

27 F 3 N S K/A WE16EK2.2 Question is satisfactory.

28 H 3 X N U K/A 003K6.02 If you lose seal injection, then how could differential pressure remain stable? If you normally have seal injection and thermal barrier cooling and then lose one, how could temperature remain stable.

Question is unsatisfactory due to two non

-plausible distractors.

5/5/16 - Revised question is satisfactory.

29 H 3 B S K/A 003K6.04 Is MVG-9568 TO RB LOAD or LOADS? Question is otherwise satisfactory.

30 H 3 N S K/A 004K3.08 Question is satisfactory.

31 H 2 N S K/A 005K5.01 Question is satisfactory.

32 H 3 X X N U K/A 006A4.07 Is there more to the valve nomenclature than CHG PP?

If not, do you expect the applicants to recognize that it is a mini

-flow isolation.

When you state that transfer to cold leg recirculation is complete, how can suction from the RWST be plausible. Question is unsatisfactory due to two non

-plausible distractors.

4/29/16 - Revised question is satisfactory.

33 H 3 N S K/A 007A3.01 (This is a replacement K/A.)

Question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 34 H 3 X N E K/A 008A1.01 Question was submitted for preliminary ten sample question review and will not be used for evaluating the quality of the final submittal.

Need to discuss whether VCT level being lower would be a better distractor and not possibly have two correct answers for a period of time. 4/29/16 - Revised question is satisfactory.

35 F 3 N S K/A 008K3.03 Question is satisfactory.

36 H 3 N S K/A 010K1.08 Question is satisfactory.

37 H 3 N S K/A 012A2.05 Question is satisfactory.

38 H 3 M S K/A 012A4.02 Question is satisfactory.

39 F 3 N S K/A 013K2.01 Question is satisfactory.

40 H 3 X B U K/A 013K5.02 Choices A and C are not plausible. Question is unsatisfactory due to two non-plausible distractors.

4/29/16 - Reviewed revised question. Do not believe that choices C and D are plausible because they would lead to a 3 of X coincidence. Do you have any logic that requires 3 of X?

5/3/16 - Revised question is satisfactory.

41 H 3 B S K/A 022A3.01 Question is satisfactory.

42 H 3 N S K/A 026G2.2.44 (This is a replacement K/A.)

Question is satisfactory.

43 F 3 N E K/A 039A2.01 It may be confusing that the rate of steam generator pressure

decrease is given in two different units in the conditions.

4/29/16 - Revised question is satisfactory.

44 F 3 X M S K/A 039K4.07 Question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 45 F 3 X M E K/A 059A2.11 Choices B.2) and D.2) are not plausible because the initial conditions state that the valve will not move manually.

Need to specify from the control board.

4/29/16 - Revised question is satisfactory.

46 H 3 X B S/E K/A 061K6.02 What is the difference between IFV

-3531(3541)(3551) and FCV

-3531(3541)(3551)?

Licensee is changing answer choices to FCV.

4/29/16 - Revised question is satisfactory.

47 F 3 X M E K/A 062A1.01 A partial loss of all off

-site power? Some of the choices do not work grammatically with the question stem. Is choice D an or, an and , or and/or? 5/3/16 - Revised question is satisfactory.

48 H 2 B S K/A 062K1.04 Question is satisfactory.

49 F 2  ? B E/U K/A 063A1.01 Need to discuss and ensure that this is not an SRO level bases knowledge question. If this were a license level mismatch, the question would be unsatisfactory.

4/29/16 - Reviewed revised question. The question stem should state contain vice contains.

5/3/16 - Revised question is satisfactory.

50 H 3 X X N E K/A 064G2.4.1 The immediate actions should match the step name in the applicable procedure. The first part of distractors A and B are from the same step in EOP

-13.0. This would make distractor B not plausible.

4/29/16 - Revised question is satisfactory.

51 F 3 X B S K/A 073K4.01 (This is a replacement K/A.)

Question is satisfactory.

52 F 3 B S K/A 076K2.04 Question is satisfactory.

53 H 3 B S K/A 078K1.04 Question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 54 F 3 B S K/A 103A3.01 Are there any sample valves closed by phase A or B that are normally open? Do you expect operators to know the normal position for sample valves listed in chemistry procedures? Is it their job function?

Operations Management representative states that this is expected knowledge.

55 F 3 B S K/A 103G2.4.6 Question is satisfactory.

56 H 3 X B E K/A 002K6.06 Question stem needs to be slightly reworded. Power should be restored not the loss of power.

4/29/16 - Revised question is satisfactory.

57 H 3 X N E K/A 015K5.18 Need to be specific about the time frame for the second part question. Would it be better to ask the direction vice closer to or farther from the upper limit?

4/29/16 - Revised question is satisfactory.

58 H 3 X M E K/A 016A4.01 Do not believe that it is plausible for Tref to be median selected or averaged when the initial conditions state that it is selected to 446. Choice C.2) is not plausible.

5/4/2016 - Revised question is satisfactory.

59 F 3 X N S K/A 033K3.03 (This is a replacement K/A.)

Should the answer choices be 1DA2/1DB2 vice 1DA/1D B? What loads are on buses 1A and 1B (Just need to verify that they are plausible)?

Question is satisfactory.

60 F 3 X M U K/A 034K1.01 This question does not appear to match the K/A. Question is unsatisfactory due to K/A mismatch.

5/3/16 - Revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 61 H 1.5 X N E K/A 035K3.02 Although ensuring the reactor makeup system is in auto is listed in EOP-1.2, I don't believe that it is a plausible distractor for this situation and makes level of difficulty very low.

5/3/16 - Revised question is satisfactory.

62 F 3 M S K/A 071A1.06 Question is satisfactory.

5/1 8/16 - Original question replaced due to overlap with the audit exam. New question is satisfactory.

63 H 3 X B U K/A 072A2.02 Question is unsatisfactory due to K/A mismatch. I do not believe that this is an area radiation monitor. Isn't it a process radiation monitor?

5/4/2016 - Revised question is satisfactory.

64 F 3 B S K/A 079K4.01 Question is satisfactory.

65 F 3 ? N SE K/A 086G2.4.49 Question is satisfactory.

66 H 1 M U K/A G2.1.25 (This is a replacement K/A

.) Question is unsatisfactory due to LOD=1.

5/5/16 - Revised question is satisfactory.

67 F 3 X N E K/A G2.1.34 It would probably be safer to ask if the TS does or does not apply in mode 5. 4/29/16 - Revised question is satisfactory.

68 F 3 M S K/A G2.1.37 Question is satisfactory.

69 H 3  ? B E K/A G2.2.14 Do you expect operators to know the normal position of this particular valve from memory?

4/29/16 - Revised question is satisfactory.

70 F 3 N S K/A G2.2.2 (This is a replacement K/A

.) Question is satisfactory.

71 F 3 M S K/A 033K3.02 This should be K/A 2.3.13 Question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 72 H 3 X M U K/A G2.3.4 This question is unsatisfactory because it overlaps with an administrative JPM.

5/4/2016 - Revised question is satisfactory.

73 H 3 X N E K/A G2.1.20 This should be K/A 2.4.12.

Do you expect operators to remember actions by step number?

4/29/16 - Revised question is satisfactory.

74 H 3 N U K/A G2.4.47 Question is unsatisfactory due to LOD =1. 5/4/2016 - Revised question is satisfactory.

75 F 1.5 X N U K/A G2.4.6 Choices A and D are not plausible. Just because an action is listed in another procedure does not necessarily mean that it is plausible for the given plant conditions.

4/29/16 - Revised question is satisfactory.

76 H 3 X M E K/A 007EA2.02 Need to reword question stem to be in accordance with E

-0, the crew is required to transition to-. The second part of distractor C could be considered a subset of the other second part choices.

You may actually be in the source range with a negative startup rate prior to adding that much borated water. However, it is not really plausible wording for an EOP step. Need to discuss whether or not this is overlap with question #1

. 5/4/2016 - Revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 77 H 1 X X N U K/A 009EG2.4.47 This question is LOD =1 with the references provided. Question is unsatisfactory due to LOD=1.

Yellow and Red Path entry conditions are RO knowledge. When you are to the left of the line that is furthest away from the region of acceptable operation, how could "is" not be the correct answer. Although bases knowledge can be SRO only knowledge, not all information listed in the bases is. Do you have an SRO only learning objective for this information?

4/29/16 - Reviewed revised question. Need to include in the second part question that is in accordance with--

5/5/16 - Revised question is satisfactory.

78 F 3 X X N U K/A 011EA2.05 The valve nomenclature given in the question does not exactly match what is given in EOP

-2.2. Need to further discuss this question. Are two charging pumps running in the initial conditions or not. Is runout ever a concern for your charging pumps?

All answer choices should begin with the same words: C should be "closed while in-"

This does not appear to be SRO only. You have an RO objective in your lesson plan for knowing the bases or reasons for cautions and notes in EOP

-2.2. Question is unsatisfactory due to not being SRO only.

5/4/2016 - Revised question is satisfactory.

79 H 3 M S K/A 026AG2.2.22 Question is satisfactory.

80 H 3 X X N U K/A 054AG2.1.23 WOOTF statement should say "-procedure SRO should implement next?" or "...required to be implemented next?" rather than CRS will, if the CRS waited before making a decision, the CRS will implement E-0 next. Question is unsatisfactory because it is not SRO only. This question can be answered with RO knowledge alone.

5/4/2016 - Revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 81 H 3 X X N E K/A 077AA2.09 (This is a replacement K/A.)

Choice A should say, "Transition to COLD SHUTDOWN is not required with the current conditions."

4/29/16 - Revised question is satisfactory.

82 H 3 X X N U K/A 001AA2.04 WOOTF statement should say words to the effect of what is required or what the procedure requires to be done rather than what the CRS will do.

Does not meet SRO only criteria:

Knowledge of entry conditions for AOPs is RO level. Knowing all entry conditions for AOP 403.3 and AOP 401.7 doesn't meet requirement for 10CFR55.43(b)5 - procedure selection.

Question is unsatisfactory because it is not SRO only.

K/A 001AA2.03 5/4/2016 - New question does not meet the K/A at the SRO level.

5/5/16 - Newer question does not meet the Continuous Rod Withdrawal portion of the K/A and the thermodynamics combined with the feed status and steam generator level status does not seem to be operationally valid.

5/18/16 - Revised question is satisfactory.

83 H 3 N S K/A 060AG2.1.20 E-plan Question is satisfactory.

84 H 3 N S K/A 061AA2.06 TS question Question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 85 F 3 X X M E K/A 069AG2.4.21 (This is a replacement K/A.)

The "The" at the beginning of choice A.2) should not be capitalized

.

4/29/16 - Revised question is satisfactory.

86 H 3 B S K/A 012A2.05 TS question

Question is satisfactory.

87 H 3 M S K/A 039A2.01 Question is satisfactory.

88 H 3 X B S K/A 076AG2.4.6 Question is satisfactory.

89 H 3 X N E K/A 078A2.01 TS question

WOOTF statement should say "-latest time by which the plant must be placed-"

The second bullet at time 0030 does not read well.

4/29/16 - Revised question is satisfactory.

90 H 3 N S K/A 103G2.1.7 E-plan Question is satisfactory.

91 F 3 X N U K/A 028A2.03 For answer choice D, spell out procedures the same as in other choices. Why does the WOOTF say, if any at the end?

Choice A is not plausible in that it is the only choice with SOP

-114 where all the others are parts of SOP

-122. Choice D is not plausible to have the option to choose either procedure.

Question is unsatisfactory due to two non

-plausible distractors.

5/4/2016 - Revised question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 92 H 3 N S K/A 068A2.04 (This is a replacement K/A.)

TS question (ODCM)

Question is satisfactory.

93 H 3 X N E K/A 071G2.2.38 (This is a replacement K/A.)

Question was submitted for preliminary ten sample question review and will not be used for evaluating the quality of the final submittal.

Remove the word "shutdown" from choices A and C to improve plausibility. Shutdown too directly points at answer.

4/29/16 - Revised question is satisfactory.

94 H 1 N U K/A G2.1.13 Question is unsatisfactory due to LOD = 1.

With the reference provided, the applicant only needs to know that the diesel is in a vital area, a diesel part is not special nuclear material, tailgating is not a failure of a safeguard system, and there is more than one diesel. With the reference, this seems to be nearly GET level.

5/3/16 - Revised question is satisfactory.

95 H 3 X M E K/A G2.2.12 TS question

Choice D is not plausible since the reference is given and clearly states that 18 months is for a different surveillance than what is identified in the stem of the question.

4/29/16 - Revised question is satisfactory.

96 F 3 B S K/A G2.2.44 Question is satisfactory.

Q# 1. LOK (F/H) 2. LOD (1-5) 3. Psychometric Flaws

4. Job Content Flaws

5. Other 6. B/M/N 7. U/E/S 8. Explanation Stem Focus Cues T/F Cred. Dist. Partial Job-Link Minutia #/ units Back-ward Q= K/A SRO Only 97 H 3 X N E K/A G2.3.14 Question was submitted for preliminary ten sample question review and will not be used for evaluating the quality of the final submittal.

E-plan The third bullet in the given conditions has an extra "is" in it.

4/29/16- Reviewed revised question. Third bullet should have units.

5/4/2016 - Revised question is satisfactory.

98 H 3 X B U K/A G2.3.6

Does not meet SRO only criteria:

The question requires knowledge on the RO level according to HPP

-709 lesson plan objectives. It is RO level of knowledge that a sample is required prior to releasing a tank that has been in service, and RO knowledge that a tank should not be release with wind from a certain direction. This is considered "system" knowledge.

This question overlaps with question # 62.

Question is unsatisfactory due to not being SRO only.

5/4/2016 - Revised question is satisfactory.

99 F 3 X M U K/A G2.4.18 Two non-plausible distractors:

Contrary to the distractor analysis, the first part of choices A and B is not plausible because the first step of EOP

-4.0 is to check for RCP trip criteria.

This is also a continuous action step.

Question is unsatisfactory due to two non

-plausible distractors.

4/29/16- Reviewed revised question. The word vent at the end of choices B.2) and C.2) should be event. 5/4/2016 - Revised question is satisfactory.

100 H 3 N S K/A G2.4.31 Question is satisfactory.

Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable). 3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content). The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory). The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO

-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES

-401 Section D.2.f.

7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any "U" ratings (e.g., how the Appendix B psychometric attributes are not being met).