Text

Initial Exam 2022-08 Draft Written Exam Comments
	ML22250A450
Person / Time
Site:	South Texas
Issue date:	09/08/2022
From:	Heather Gepford; NRC/RGN-IV/DORS/OB
To:	; South Texas
References
	Download: ML22250A450 (1)
	v • d • e

Form 2.3-5 Written Examination Review Worksheet

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 1

F 3

X B

E NRC:

1) How would opening feeder breakers cause the RTP shunt trip coils to de-energize? Questioning the plausibility of this distractor.

2) Change 10CFR 50.47 reference to (b)(6)

3) Appears to have been used on 2013 NRC Exam (Q11) - remains source B Licensee:

1) Modified explanations for plausibility of shunt trip coils to match original question 1577 from 2013 exam.

2) Changed as requested.

3) Question was Bank Question 1577, created Question 3118 in Bank to reformat to current 2 by 2 format for this test.

NRC:

1) Question, as revised, is SAT.

2 F

3 N

E NRC:

1) Recommend moving Initially in the stem to after will. I.e, the steam dumps will INITIALLY be operated in

2) The applicant has to make the assumption that steam dump controls have not been manipulated prior to this point (would be in Tave mode in auto given the initial conditions), however, the assumption would have to be made that only the procedurally required actions from E-0 to E-1 have been taken.

Licensee:

1) Changed as requested.

2) Placed conditions in stem to show that steam dumps are available and S/G pressures are normal and would require alignment per step 2 of EO10 to depressurize to < 1000 psig.

NRC:

1) Question, as revised, is SAT.

3 H

3 N

S NRC:

1) Change 10CFR 50.47 reference to (b)(3)

Licensee:

1) Changed as requested 4

F 3

N S

NRC:

1) Change 10CFR 50.47 reference to (b)(8)

Licensee:

1) Changed as requested 5

H 3

N E

NRC:

1) Change 10CFR 50.47 reference to (b)(5)

2) Question meets the K/A, but is a tier 2 question (can be answered with system knowledge only).

Need to modify this to meet the tier 1 EOP/AOP.

Licensee:

1) Changed as requested.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

2) Added second part to tie to the off-normal procedure.

NRC:

1) Question, as revised, is SAT.

6 H

F 3

N E

NRC:

1) Add to plausibility justification for part 1 distractors, that letdown is isolated in step 5, but its AFTER cold leg injection valves are isolated in step 3, and the stem asks which valves are isolated first.

Licensee:

1) Changed as requested.

NRC:

1) Question, as revised, is SAT 7

H 3

N M

B E

NRC:

1) Appears to be a modified bank question from the 2015 NRC Exam (Q4).

2) What is the purpose behind part 1 of the question?

The written exam should not be an extension of the operating test. Determining where an indication is monitored in the MCR is an operating test observation and should be minimally applied to the written exam, for indications specifically called out in EOP/AOPs, actuations and/or reset functionality (2/2 switches at 1/2 locations), etc.

Licensee:

1) Replacing with Bank question 2393

2) Replacing with Bank Question 2393 NRC:

1) Question, as revised, is SAT.

2) Bank - 2015 NRC Exam 8

H 3

B E

NRC:

1) Does not appear to be a tier 1 question. Possible tier 4 question. Recommend rejecting K/A and writing new question.

Licensee:

1) Agree. Replaced K/A with APE 027 AK3.03 and used Bank question 2949.with some editorial revisions. Updated RO outline and Rejected K/As.

NRC:

1) Question, as revised, is SAT.

9 F

2 B

S NRC:

1) EOP bases should be referenced rather than the lesson plan that highlights the EOP bases step.

Licensee:

1) Added reference to EOPT-03.26 FRS1 STPEGS EOP Technical Guideline.

10 H

3 X

X N

B U

NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

1) Question does not meet K/A as written. Question should be applying rad levels (mR/hr) to a SGTR, rather than asking what rad monitors are used to determine a SGTR.

2) Question does not appear to have a correct answer. EOP-3 states that a ruptured SG is identified by either SG steamline rad monitors OR SG blowdown line rad monitors; question answer states both are used to identify (as in both must be used).

Licensee:

1) Using Rad Monitor readings would essentially make this an LOD-1 question. Replaced K/A with 038 EA2.01 and replaced with Bank question 2562 with revisions to make it more plausible.

2) Replaced question with question 2562.

NRC:

1) Question, as revised, is SAT.

11 H

3 4

N U

NRC:

1) Not a tier 1 question. Possible tier 4 question.

Recommend rejecting K/A and writing new question. Additionally, question is overly convoluted to meet the K/A. While it meets the K/A, its overly convoluted and could be simplified.

2) What is the IVC abbreviation?

3) How is it operationally valid that MSLs would not be isolated prior to crew diagnosis (due to the isolation on steamline high negative rate) and manual isolation?

Licensee:

1) Agree not a tier 1 question. Wrote new question 3235.

2) Isolation Valve Cubicle. Students would know the terminology but the question has been replaced.

3) High negative rate did not apply in the original question. Low Steamline pressure would. The operators would more than likely have time to diagnose prior to an automatic isolation but the question has been replaced with new question 3235.

4) Based on prep week comment changed EO00 to EO10 in the second part of question 3235.

NRC:

1) Question, as revised, is SAT.

12 H

F 4

3 2

X N

B N

U NRC:

1) Very similar to 2018 NRC Exam Q88.

2) Appears to be SRO-only. Justification states that student must analyze plant conditions and determine the correct EOP step and basis - which is SRO level knowledge.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

3) Change 10CFR 50.47 reference to (b)(4)

Licensee:

1) Replaced question with question 3236.

2) Replaced question with question 3236.

3) Changed 55.41 reference.

4) Changed out part 2 distractor based on NRC comments during review of Q3236.

NRC:

1) Question, as revised, is SAT.

2) Question is a new question, not bank.

13 F

H 2

N E

NRC:

1) Part 2 of the question seems unnecessary - part 1 meets the K/A. Part 2 is an extension of the operating test. Question would be better off as a tradition 1x4 question. Looking at the reference material, 500 gpm seems like a plausible distractor. Is there another AFW flowrate that could be a plausible distractor?

2) Change 10CFR 50.47 reference to (b)(8)

Licensee:

1) Rewrote question as a 1X4 using two other AFW flow numbers.

2) Changed as requested.

3) Capitalized MAXIMUM per prep week comments.

NRC:

1) Question, as revised, is SAT.

14 F

2 M

S NRC:

1) Sequence answers from shortest to longest (ES-4.2 Section D.2)

Licensee:

1) Changed as requested.

15 H

3 N

E NRC:

1) Consider adding a statement in the stem stating that all required TS actions have been taken for NI-

41. Would it still be necessary to state that CP fuses have been removed?

2) Reference TS 3.3.1 in question pedigree (for channel trip requirements)

Licensee:

1) This would require the student to know a greater than one hour action. Pulling the control power fuses is what gives the candidate the knowledge that the channel is tripped.

2) Changed as requested.

NRC:

1) The applicant should know the required actions via the off-normal procedure - TS knowledge > 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months is not solely required. Changes made satisfies this

2) Question, as revised, is SAT.

16 H

3 2

N E

NRC:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

1) Question is not a tier 1 question. Part 1 is tier 2 and part 2 is merely an extension of the operating test.

2) Change 10CFR 50.47 reference to (b)(7).

Licensee:

1) Changed second part of question to tie to Tier 1.

2) Changed as requested NRC:

1) Question, as revised, is SAT.

17 H

3 N

E NRC:

1) Why is a 2x2 question necessary when part 2 is different for every answer choice?

2) Unit 1 is specified - make the equipment in the stem and answer unit specific.

Licensee:

1) Using just 2 things in the second part would have resulted in implausible distractors.

2) Changed as requested.

NRC:

Question, as revised, is SAT.

18 H

F 3

X X

N B

N U

NRC:

1) Question appears to be SRO-only knowledge, requiring detailed procedure knowledge rather than overall mitigative strategy of EO-EC11. Is there justification that the RO should know that since all RCFCs are running, zero CS pumps are necessary?

2) As mentioned above, in the current state, zero CS pumps are necessary, so reduce CS flow would be to secure all CS pumps. Distractor B is partially correct and Answer A is partially correct.

3) Bank question from 2011 NRC Exam (Q63) -

Marked as U instead of E, despite being an NRC approved bank question, due to the license level fit.

Licensee:

1) Agree. Replaced with new question 3237.

2) Replaced with new question 3237.

3) Replaced with new question 3237.

4) Replaced CS Pumps in second part of 3237 with Centrifugal Charging pumps per comment during prep week.

NRC:

1) Question source is new.

Question, as revised, is SAT.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 19 H

3 M

E NRC:

1) Stem should state that no operator actions have been taken (since placing rod control to MAN is an immediate action)

2) Part 2 is LOD=1 (trip Rx with rod control in MAN and outward rod motion). Recommend new part 2 (must ensure question stays tier 1).

3) Change 10CFR 50.47 reference to (b)(6).

Licensee:

1) Changed as requested.

2) Replaced part 2 with new tier 1 related action.

3) Changed as requested.

4) Made changes to question based on prep week comments NRC:

1) If going to CB-A may stop rod motion but is not procedurally allowed, the stem must state that the RO should take action IAW the POP04-RS-0001 procedure (this would make only tripping the Rx a correct answer and would make going to CB-A incorrect, as this is not procedurally allowed).

License:

1) Change made as requested (part 2 of the stem modified to ask procedure action).

NRC:

1) Question, as revised, is SAT.

20 F

3 X

N B

M E

NRC:

1) Bank question that is slightly modified from 2015 NRC Exam (Q7), not enough to be called modified.

2) Part 2 distractors B & D do not appear to be plausible. A SG PORV is going to cycle to maintain pressure. How would setting a setpoint at 1200+

psig prevent depressurizing the ruptured SG (also, this should be leaking SG and not ruptured, since the question stem states a tube leak is in progress, not a rupture - possible cueing).

Recommend using bank question as written,or changing part 2 distractor.

Licensee:

1) Disagree that question isnt at least Modified. The Original question gives the band in the stem and asks why it is adjusted there. The question supplied asks for the band and the reason for the band. Asking for the correct band rather than supplying it in the stem makes it a modified question. Will change to a Modified question.

2) Replaced part 2.

NRC:

1) Agree with modified source classification

2) Question, as revised, is SAT.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 21 H

3 B

S 2013 NRC Exam NRC:

1) Does the sites conduct of ops procedure require backing up the SI & phase A actuations upon automatic actuations in E-0? Should phase A have already been manually actuated (to back up the auto) prior to performing Add 5?.

Licensee:

1) Only if no train actuation has occurred. In this case it is only one train and it would be corrected by performing addendum 5.

22 F

3 X

B U

NRC:

1) Appears to be no correct answer. FRC2 requires depressurizing until all SG pressures are less than 305 psig (all are AT 305psig, not less than) OR two or more RCS HL temps < 435 deg (step is open bulleted).

Licensee:

1) Changed 305 psig to 285 psig NRC:

1) Question, as revised, is SAT 23 H

3 M

S 24 H

3 X

N E

NRC:

1) Stem states SI pump - should reference all SI pumps have been secured.

2) Cueing from part 1 to part 2. Recommend rephrasing to : "In accordance with 0POP05-EO-ES11, subcooling ___(1)____ satisfactory, and the operators are directed to ___(2)___."

Licensee:

1) Changed as requested.

2) Changed as requested.

NRC:

1) Question, as revised, is SAT 25 H

3 2

X N

U E

Reference provided NRC:

1) What is the cause of the high 1B SG level?

Nothing in the stem suggests reasoning for a high SG level (operational validity?).

2) Direct look-up with provided reference. SRO-only knowledge without reference - are ROs expected to know yellow path entry conditions from memory?

Licensee:

1) Added additional bullet to stem to explain high level in 1B S/G.

2) Disagree that the question is a direct lookup. The candidate must use conditions in the stem to

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level eliminate choices and select the highest priority yellow path procedure.

NRC:

1) Agree with licensees assessment that question is not direct look-up. Providing the reference lowers the LOD. LOD 2.

2) Question, as revised, is SAT.

26 H

3 X

X N

M N

U NRC:

1) Question stem asks where can cnmt water level be monitored and answer justification for B states CP-001 has a QDPS plasma display where Containment Water Level can be monitored. The stem does not ask about an indicator, which is the key piece to make choice B incorrect. What is the importance of whether a board indicator exists or a trend recorder? If one was wide range and one narrow range, or one post-accident and the other not, then it would seem pertinent. But a difference between a board indicator and a trend recorder (monitoring the same function), seems to be minutia and an extension of the operating test.

2) Provided references do not support part 1 (panel on which cnmt water level indication exists).

3) Not a tier 1 question - stem provides containment pressure, so the applicant isnt even assessing the conditions.

4) Question appears to be modified from 2010 NRC Exam (Q19)

Licensee:

1) Question replaced with new question 3234.

2) Question replaced with new question 3234.

3) Question replaced with new question 3234.

4) Question replaced with new question 3234.

NRC:

1) Question source is new

2) Question, as revised, is SAT.

27 H

3 N

E S

NRC:

1) Part 2 is an extension of the operating test and not essential to the question being asked. While is does meet the K/A, the question does not have to hit both the operate AND monitoring aspects - its and/or.

2) Question would be far more beneficial providing bearing temps for the RCPs and asking which RCP bearing temp required entry into the off-normal (or simply RCP trip criteria).

Licensee:

1) Discussed with examiner question is SAT.

2) Discussed with examiner question is SAT.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level NRC:

1) Question, as submitted, is SAT.

28 H

3 N

S 29 F

H 3

X N

U S

NRC:

1) Question does not meet K/A. Part 2 does not meet the (b) part of the K/A. While the (a) part is met, the (b) part is not; verifying the pump is tripped is not using a procedure to correct, control, etc. ES-4.2 C.2 - "When selecting or writing questions for K/As that test coupled K/As (e.g., the A.2 K/A statements in Tiers 1 and 2 and a number of generic K/A statements, such as 2.4.1, in Tier 3), try to test both aspects of the K/A statement.

If that is not possible without expending an inordinate amount of resources, limit the scope of the question to that aspect of the K/A statement requiring the highest cognitive level (e.g., the (b) portion of the A.2 K/A statements) or substitute another randomly selected K/A."

Licensee:

1) Discussed with examiner question is SAT.

NRC:

1) Question meets the (b) part of the K/A, as its asking for a annunciator response procedure action

- pump is running below the trip setpoint; applicant must know the requirement to trip the operating pump to prevent damage, IAW the procedure.

2) Question, as submitted, is SAT.

30 F

3 2

N S

NRC:

1) Change 10CFR 50.47 reference to (b)(8).

Licensee:

1) Changed as requested.

31 F

2 N

M E

NRC:

1) Attached references do not clearly delineate the fact that water from the PRT rupture will be first observed in the normal cnmt sump rather than the cnmt water level WR. Answer justification states this, but the references do not.

2) Question appears to be modified from 2014 NRC Exam Q24.

Licensee:

1) Added Tech Manual reference for the level detector.

2) Changed as requested.

NRC:

Question, as revised, is SAT

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 32 H

3 X

N S

NRC:

1) Question has overlap with Q26. Applicants can rule out distractor C & D (#4 in the stem) due to information given in Q26. Recommend changing Q26.

Licensee:

1) Replaced Question 26.

NRC:

1) Q26 replaced, no overlaps concerns exist.

33 F

2 B

S 2018 NRC Exam 34 H

3 2

N S

NRC:

1) Add 10CFR 50.47 reference of (b)(5).

Licensee:

1) Added as requested.

35 H

3 B

S 2018 NRC Exam 36 F

H 2

1 3

N U

NRC:

1) LOD=1 due to 2/4 distractors being 4kV (site has no 4kV fans). Recommend adding some initial conditions and providing the loss of a bus (via indications and asking which RCFC is lost.

Licensee:

1) Rewrote question along lines suggested.

NRC:

1) Question, as revised, is high cog & LOD 3

2) Question, as revised, is SAT.

37 F

3 B

S 38 H

3 N

E NRC:

1) Reference has no labeling or source info (only a slide number). Consider adding additional/improved references to support the answer justification (procedure notes/cautions, systems lessons plans, etc).

Licensee:

1) Added NDR figure and Table for MTC at EOL. Also labeled PowerPoint slide to show where the information came from. This is out of a vendor provided lesson that covered our equivalent of LOT101.25, Reactor Operational Physics.

NRC:

1) Question, as revised, is SAT 39 F

3 B

S 2015 NRC Exam

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 40 H

3 B

S 2018 NRC Exam NRC:

1) Is there a reference to support the statement attached to the question (on the reference) that says AFW flow will be zero on the board meters and the QDPS screens? Im not sure where this statement came from; does not appear that it was in the lesson plan.

Licensee:

1) The reference provided is from the PowerPoint for LP LOT202.44 which is QDPS. Added POP04-AM-0001, Addendum 13 as a reference also, but it just states they are unreliable, not how they actually indicate.

41 F

2 N

E NRC:

1) Revise to make the question a standard 1x4 question asking the power supply for the AFW pump. Part 2 is LOD=1.

Licensee:

1) Rewrote question.

NRC:

1) Question, as revised, is SAT 42 H

3 N

E NRC:

1) Part 1 doesnt appear to be pertinent to the K/A, seems to exist only to provide for 2 distractors for a 2x2 question (written exam should not be an extension of the operating test). Part 1 would be more effective by asking to evaluate the status of the bus grounds (meet the (a) part of the K/A).

Licensee:

1) Rewrote question.

NRC:

1) Question, as revised, is SAT 43 F

3 N

S 44 F

3 2

B S

2019 NRC Exam

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 45 H

3 N

E NRC:

1) In the stem, add and is declared INOPERABLE to the 2nd bullet - dont need RO applicants making operability calls.

2) In part 1 of the stem, add In accordance with Technical Specifications, prior to Unit 1 is required

3) Part 2 has no relevancy to the K/A. While it fits within the question, part 2 is an odd fit in terms of the K/A.

4) Change 10CFR 50.47 reference to (b)(4).

Licensee:

1) Replaced question with question 3159.

2) Replaced question with question 3159.

3) Replaced question with question 3159.

4) Replaced question with question 3159.

NRC:

1) Answer A is a 1-hr LCO action (perform normal &

alt offsite source surv). It is implied that the EDG is OOS and subsequently, the A ECW pump becomes inop, which would not create a new 1-hr LCO action, however, the situation without respect to the timing, does create a 1-hr LCO action and can be considered correct.

Licensee:

1) Modified distractor to state that EDG has been inoperable for 4 hours4.62963e-5 days 0.00111 hours 6.613757e-6 weeks 1.522e-6 months when the ECW pump is declared inoperable.

NRC:

1) Question, as revised, is SAT 46 F

2 B

S 47 F

2 3

B E

2017 NRC Exam NRC:

1) Question could be streamlined by rephrasing to something like: "At 1200, and event requiring containment closure occurred. The hatch must be closed with a minimum of (1) bolts and movement of irradiated fuel must be suspended (2).

Licensee:

1) Revised stem to Containment closure is required to be established.

NRC:

1) Question, as revised, is SAT

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 48 F

2 M

B E

NRC:

1) Question would be more effective as the original bank question (2019 NRC Exam). Asking the relief valve setpoint is a more effective system knowledge objective than where PRT level is monitored at in the MCR (extension of the operating test).

Licensee:

1) Replaced with bank question 2824.

NRC:

1) Question, as revised, is SAT 49 F

2 N

S 50 F

2 X

N U

NRC:

1) Question is SRO knowledge - requires applicant to know TS required actions > 1 hour1.157407e-5 days 2.777778e-4 hours 1.653439e-6 weeks 3.805e-7 months (24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months in this case).

2) Distractor analysis for C states that part 1 is correct, when it is not correct.

Licensee:

1) Replaced with question 3231.

2) Replaced with question 3231.

NRC:

1) Question, as revised, is SAT 51 F

H 3

2 3

X X

N U

NRC:

1) Overlap with Q52.

2) This question is a better match for the K/A associated with Q52.

3) Question does not match K/A - This question is a physical connection between DC distribution and an AC component, not AC distribution (see comment

2)

Licensee:

1) Replaced question with new question (3230).

2) Replaced question with new question (3230)

3) Replaced question with new question (3230)

NRC:

1) No overlap concerns with new questions.

2) Question, as revised, is SAT.

52 F

2 X

B S

2013 NRC Exam NRC:

1) Overlap with Q51.

2) Is there only one non-class IE solenoid? Part 1 is plural, part 2 is singular.

Licensee:

1) Replaced Q51.

2) Yes, there is only one non-safety solenoid.

NRC:

1) No overlap concerns. Question is SAT

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 53 H

3 X

N E

NRC:

1) Sequence answers from shortest to longest (ES-4.2 Section D.2)

2) Distractor D does not appear to be plausible, as the 1B CCW/ECW HX is not in service.

3) 3 of the 4 choices have start to rise, except the correct answer (This could be fixed by replacing distractor D)

Licensee:

1) Changed as requested.

2) Replaced distractor D.

3) Now have two Chillers selections and two trend selections.

NRC:

1) Question, as revised, is SAT 54 F

3 X

N E

S NRC:

1) Part 2 appears to be minutia (for a from memory written exam - much better suited for an in-plant JPM). How is it essential that an applicant knows this from memory? It does not appear to be necessary knowledge per the stated learning objective.

Licensee:

1) Disagree. The operation and location of an important design feature is as to be committed to memory as anything else. This is operation is part of an EOP addendum which may be part of a JPM but we are not overlapping with it.

NRC:

1) Agree with licensees assessment of importance.

No overlap with JPMs.

2) Question, as submitted, is SAT 55 H

3 X

N U

NRC:

1) Distractors C & D are not plausible due to part 1 -

less subcooling post-cooldown is not plausible.

Licensee:

1)

Rewrote question to subcooling requirements during a normal cooldown and means to control it.

NRC:

1) Question, as revised, is SAT 56 F

2 N

M E

NRC:

1) Part 2 of the question is unrelated to the K/A. Can this be written as a traditional 1x4 question with 2 additional plausible power supply distractors?

2) Appears to be a modified bank question from 2015 NRC Exam (Q34)

Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

1) Changed as requested to a 1X4 question over just the power supplies.

2) Change to Modified question from bank question 2372.

NRC:

1) Question, as revised, is SAT 57 H

3 N

E NRC:

1) What purpose does the LOT 506.01 slide 14 serve as a reference?

2) Part 2 A & D would be more plausible to insert all control banks only. Attached references do not highlight or support the claim in A & D distractor justification that > POAH, emergency boration is required, but rod insertion is not.

Licensee:

1) Removed reference.

2) I disagree with the comment that rods would be inserted > POAH. When you are critical TS 3.1.3.6 applies which says if you are below the insertion limits to restore the rods to above the insertion limits within two hours. This is done by borating the rods out.

NRC:

1)

Agree with licensees comments.

2)

Question, as submitted, is SAT.

58 H

3 N

E NRC:

1) Since the stem is establishing a timeline of events, part 2 of the stem should state If Hydrogen Recombiner 2A subsequently trips,

2) Change 10CFR 55.41 reference to (b)(7)

Licensee;

1) Changed as requested.

2) Changed as requested.

NRC:

1) Question, as revised, is SAT 59 H

3 X

M U

S NRC:

1) Appears to be two correct answers. Distractor analysis states 0POP02-HC-0003 is used in modes 1-4 and HC-0002 only in modes 5-6 and defueled.

HC-0003 is used in all modes, and HC-0002 states, "IF directed by 0POP03-ZG-0001, Plant Heatup OR Unit Supervisor/ Shift Manager, THEN Lineup 4, Mode 4 Lineup can be performed in ANY Plant Mode.".

Licensee:

1) Disagree with comment. Supplemental Purge is the only purge available to be used in MODE4.

0POP02-HC-0002 allows the use to perform sections ONLY in MODES 5, 6 or DEFUELED. The note that Lineup 4, MODE 4 Lineup can be

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level performed in ANY PLANT MODE is given so that the MODES 1-4 configuration can be established at any time. This configuration does not allow the use of Normal Purge, conversely it makes it unavailable for use. Added to distractor plausibility.

NRC:

1) Agree with licensees comments. Question, as submitted, is SAT 60 H

F 3

2 X

N U

NRC:

1) Appears to be SRO-only knowledge (below the line level of TS knowledge) knowing how many MSSVs are required to be operable at a given power level in addition to the lift setpoints.

Licensee:

1) Replaced with new question (3228) which asks about UFSAR/TS limits on SG tube leakage.

NRC:

1) Question, as revised, is SAT 61 H

3 X

N E

NRC:

1) Overlap with Scenario 4, Event 5 (same instrument failure and failure mode). Question in SAT by itself, but question must be replaced due to overlap.

Licensee:

1) Replaced question with new question 3227 about a PT-557 failure.

NRC:

1) Question, as revised, is SAT 62 H

3 X

N U

E NRC:

1) Distractors B & D are not plausible with EHC pressure dropping rapidly and an EHC turbine trip occurring at 1300 psig (10 psig margin in this case).

Licensee:

1) Disagree that question is unsat. The turbine trip on LOW Auto Stop Oil Pressure occurs at 1245.8 psig.

The POP09 is a procedural limit of approaching 1300 psig to perform, in this case a manual reactor trip. So there is time to perform either of the other two actions. I will give a specific rate of lowering to make it clearer that there is some time to get this done if they think it is the correct thing to do.

2) Per comment during prep week added the pressure that a reactor trip will occur to the answer explanation.

NRC:

1) Disagree with the licensees assessment that there is time to perform the manual actions - rapidly is subjective and can reasonably imply that manual operator action is not feasible. However, question

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level is considered and E due to the minor nature to resolve the flaw.

2) Question, as revised, is SAT 63 F

3 M

S NRC:

1) Change 10CFR 50.47 reference of (b)(13).

2) Based on the wording of step 4.22 of 0POP02-CW-0001, it seems as through only a single CW pump must be run for 1.5 hrs after securing the release.

Question stem implies all.

Licensee:

1) Changed from 7 to 13 as requested.

2) The stem states three pumps are running. The only time we run with one pump running would be during initial system startup. When shutting down the system the last two pumps are secured together. There are interlocks to ensure this.

Notes and Precaution 4.11.1 states that two pump operation should occur only during the time necessary to start a third pump or the time necessary to shutdown ALL CW pumps.

64 F

2 X

M E

NRC:

1) Distractor A & D not plausible (due to #4 in stem).

Continuous monitoring would seemingly require no other duties (i.e dedicated operator).

Licensee:

1) Removed continuous from stem in #4.

NRC:

1) Question, as revised, is SAT 65 H

3 X

N U

E NRC:

1) Not enough information in the stem to eliminate A (answer justification even states so).

2) Need to add a statement to the stem to treat each case separately.

Licensee:

1) Disagree that this question is unsat. The students are cautioned not to make assumptions and add conditions not stated unless they can be inferred by conditions given in the stem. However, I will incorporate your suggestion in comment 2. I will also modify As plausibility statement to eliminate the ambiguity I apparently created with my clarifying statement.

2) Added statement to treat each case separately NRC:

1) Agree with the licensees assessment that the question should be an E, as submitted

2) Question, as revised, is SAT.

66 F

3 2

N S

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 67 F

2 B

S 2019 NRC Exam 68 H

F 3

2 B

E 2020 NRC Exam Reference Provided NRC:

1) Overlap with Admin JPM A4. Requires the applicant to use survey maps to determine rad area level and PPE requirements. Recommend replacing this question.

Licensee:

1)

Rejected K/A. Selected new K/A 2.3.4 and replaced with new question 3226 on administrative limits.

NRC:

1)

Question, as revised, is SAT 69 F

3 2

B S

2020 NRC Exam 70 H

2 N

E Free Look Submittal Reference provided NRC:

1) Ensure Reference and NRC Reference Reqd on question pedigree page match (Reference says Figure 5, when its Figure 5.5).

2) Why is it necessary to provide excerpt pages from 0PSP-ZG-0003? Doing so makes this more of an Admin JPM. Only Fig 5.5 should be provided.

Licensee:

1) Revised as requested

2) Deleted all of ZG-3 with the exception of Form 2 and filled out required information so that student only has to answer the requested information in the stem.

NRC:

1)

Question, as revised, is SAT 71 H

3 B

S 72 H

3 N

S Free Look Submittal Reference provided (embedded) 73 H

3 N

S Free look Submittal 74 H

2 N

E Free look Submittal NRC:

1) Question should be site specific (OL Feedback FAQ 4.2.1). Could simply provide a specific unit is a given power level in the stem to accomplish this.

2) Recommend underlining or bolding key words in the stem, such as lower.

Licensee:

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level

1) Revised as requested.

2) Revised as requested..

NRC:

1)

Question, as revised, is SAT 75 H

2 B

S Free look Submittal

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 76 H

3 N

S 77 H

3 N

E NRC:

1) TS states..within 24 hours2.777778e-4 days 0.00667 hours 3.968254e-5 weeks 9.132e-6 months restore at least one of the inoperable offsite sources to operable status or apply the requirements of the CRMP. What are the CRMP requirements? Should a statement exist in the stem stating without applying CRMP actions?

Licensee:

1) Added (Assume CRMP will not be applied).

NRC:

1) Question, as revised, is SAT.

78 F

3 X

N U

NRC:

1) Part 2 of the question is not relevant to the K/A (failure modes of air operated equipment).

2) Part 2 of the question is not operationally valid.

Addendum 2 is directed by step 10.h RNO, however, the crew would transition to step 14 at step 10.b, as the stem states that pressure lowered to 70 psig (with no rate). Step 14 simply has the crew monitor IA pressure stable or rising and if its lowering, then perform Add 2. Given the initial conditions in the question, the applicant would not be incorrect assume that pressure is stable at 70 pisg.

Licensee:

1 ) Changed part 2 to question to address the strategy for RHR temperature Control per the procedure as suggested by NRC during prep week review.

2) Added slowly lowering to the stem.

NRC:

1) Question, as revised, is SAT.

79 H

3 X

N E

Free look Submittal NRC:

1) Operability is a term associated with safety-related equipment. Part 2 implies that a minimum SY voltage for operability is for RCP protection. A & C are not plausible

2) References need to support TS 3.8.1 and/or TS bases for qualified offsite sources, as part 1 of the question is asking for an operability determination.

While 0POP04-AE-0005 may require a minimum offsite source voltage to prevent abnormal conditions, this does not support making an operability determination; only the licensing basis does, which must be referenced to support part 1.

Licensee:

1) Disagree with comment. The RCPs are required for Loop operability (TS) and high amps on the

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level RCP can degrade the Containment electrical penetration over time which is a Containment Integrity concern.

2) Added additional references to support question.

3) Per comment during prep week shortened part 2 selections and added additional information to justifications as requested.

NRC:

1) Question, as revised, is SAT.

80 H

3 N

E NRC:

1) Change will to should in the stem Licensee:

1) Changed as requested NRC:

1) Question, as revised, is SAT 81 F

3 X

N E

Reference provided (embedded)

NRC:

1) Distractor B & D not plausible due to part 1 - feed and bleed is initiated due to dry S/Gs; feed and bleed would not be performed to prevent SG dryout.

Licensee:

1) Revised Question.

NRC:

1) Question, as revised, is SAT 82 H

3 X

X N

E NRC:

1) Distractor B & D not plausible due to the cueing in the stem providing a rod bottom alarm - recovery of a misaligned rod is not plausible when the rod is dropped to the bottom. This could be okay IF the applicant were to have to determine the status of the rod (if the rod bottom alarm was not given), but that may hamper the ability to determine the correct procedure. Recommend changing part 1 to better match the K/A (require the applicant to determine the status of the rod). Part 2 makes the question SRO-only.

Licensee:

1) Rewrote question conditions and added DRPI picture per suggestion during prep week.

NRC:

1) Question, as revised, is SAT 83 H

4 3

N S

NRC:

1) Add 10CFR 55.43 reference of (b)(2)

Licensee:

1) Added 55.43 reference. Changed LOD to 3.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 84 H

3 B

S 2019 NRC Exam 85 F

3 N

S NRC:

1) Change 10CFR 55.43 reference of (b)(1)

Licensee:

1) Changed 55.43 reference as requested.

86 H

3 B

E Free look Submittal 2019 NRC Exam NRC:

1) Answer and distractor justification should be improved to discuss procedure entry via the alarm response procedures (is this a direct entry condition to either 0POP04-RC-0002 or 0003?)

2) Change will to should in the stem Licensee:

1) Two of the three POP09s send you to RC-0002.

They do not send you to the Addendum the other POP09 for RCDT Level sends you to RC-0003 but only if you suspect RCS leakage. The indications given support entry into RC-0002.

2) Changed as requested.

NRC:

1) Question, as revised, is SAT.

87 H

3 N

S Free look Submittal 88 H

3 X

N U

NRC:

1) Avoid negative phrasing when possible. Rephrase part 1

2) Question meets the (a) part of the K/A (via part 1),

but not the (b) part. TSs are not procedures used to mitigate, correct, or control a transient. ES-4.2 C.2 - "When selecting or writing questions for K/As that test coupled K/As (e.g., the A.2 K/A statements in Tiers 1 and 2 and a number of generic K/A statements, such as 2.4.1, in Tier 3), try to test both aspects of the K/A statement. If that is not possible without expending an inordinate amount of resources, limit the scope of the question to that aspect of the K/A statement requiring the highest cognitive level (e.g., the (b) portion of the A.2 K/A statements) or substitute another randomly selected K/A.".

Licensee:

1) Rewrote question to address both parts of K/A.

NRC:

1) Question, as revised, is SAT 89 F

3 X

N U

S NRC:

1) Question does not appear to meet the K/A. The question does not address the impact on AFW due

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level to back leakage from MFW. Question is about back leakage from AFW.

Licensee:

1) Disagree that question is unsat. The K/A is about how MFW back leakage affects AFW. The MFW system connects directly to the AFW system via the Preheater Bypass line. The Question asks what the temperature limit of AFW is for back leakage which would show up on the discharge header and what you need to do per the off-normal to correct it.

NRC:

1) Question, as originally written, is SAT based on the licensees response. Question SAT.

90 H

3 X

N E

NRC:

1) Distractors C & D not plausible. Not plausible that a TS would not be satisfied but entry into that TS could be delayed for a period of time.

Licensee:

1) Revised Question.

NRC:

1) Question, as revised, is SAT.

91 H

3 X

N U

E NRC:

1) Change 10CFR 55.43 reference of (b)(7)

2) 0POP08-FH-0001 Add 4 does not appear to state that Core Load Supervisor approval is required, only that notification to them (along with Rx Engineering) is required. Additionally, since the SM has responsibilities over the refueling process, does the SM not ultimately have authority here?

Potentially multiple correct answers.

Licensee:

1) Changed 55.43 reference as requested.

2) Do not agree question is unsat. Per Note and Precaution 4.56 Core Load Supervisor permission is needed to change the LOAD SELECT Switch. If Overload Bypass Switch is used it takes Shift Manager, RX engineering, and Core Load Supervisor permission. Added 4.56 to references.

NRC:

1) Question, as originally written, is not unsat, based on the licensees response, and is an enhancement.

92 F

3 X

N U

NRC:

1) Question does not meet the K/A. It meets the generic K/A, but not for the system (Area Rad Monitors). Question is tier 3, not tier 2.

2) Change 10CFR 55.43 reference of (b)(5)

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level Licensee:

1) Agree with comment. Changed question to new question 3196

2) Changed Reference as requested.

3) Added specific EAL to justifications as requested during prep week.

NRC:

1) Question, as revised, is SAT.

93 H

3 N

S NRC:

1) Change 10CFR 55.43 reference of (b)(5)

Licensee:

1) Changed as requested.

94 F

3 X

N E

NRC:

1) Procedure number in stem is missing a -

2) Distractor justification states that the cognizant operations manager or designee has authority.

This implies there are two correct answers.

0POP01-ZA-0049 does not appear to support that or designee, however. This needs clarification to ensure two correct answers do not exist.

Licensee:

1) Fixed.

2) Removed or designee from distractor justifications.

NRC:

1) Question, as revised, is SAT.

95 F

3 X

B E

2019 NRC Exam NRC:

1) Add 10CFR 55.43 reference of (b)(7)

2) How is A not also correct? Step 3.2.2 states that the CLS "shall be responsible for concurring with any changes to an approved Fuel Transfer Form.

While it may be semantics that the CLS may not necessarily be the one to make the changes, they are required to concur with them. I would recommend seeking a new distractor.

Licensee:

1) Added 55.43 reference.

2) Although we disagreed with this comment, we changed make to prepare which is definitely Rx Engineerings responsibility per 0PEP02-ZM-0005.

NRC:

1) Question, as revised, is SAT.

96 F

2 N

M E

Free look Submittal NRC:

1) Is Ops Director approval required for any procedure revision? Would the procedure SME be a more plausible distractor?

2) Question not written to K/A 2.2.4 on the written outline. The question is sat, as written, but not

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level written to the K/A with no record of rejecting the K/A on the Form 4.1-1.

3) Question appears to be modified from 2020 NRC Exam Q3 (and/or 2017 Exam Q41, 2018 Exam Q77).

Licensee:

1) Changed to Cognizant Manager who does approve procedures.

2) Updated outline and record of rejected K/As.

3) Changed to Modified from Q790.

NRC:

1) Question, as revised, is SAT.

97 H

3 B

E 2020 NRC Exam NRC:

1) Change will to should in stem Licensee:

1) Changed as requested.

NRC:

1) Question, as revised, is SAT.

98 H

3 M

B S

2019 NRC Exam NRC:

1) Could not find the parent question attached (2019 Exam Q8). Question is a bank question and does not meet the requirements to be considered modified (ES-4.2 Section B.4). Only the time duration was changed, which results in the same answer as the parent question.

Licensee:

1) Changed to Bank.

99 H

F H

3 X

X X

N U

NRC:

1) Both parts of the question do not work well together. If an applicant chooses the red path, that eliminates answer C (due to part 2). Similarly, if an applicant chooses based on order (subcriticality),

that eliminates severity as a choice. Additionally, this is RO level knowledge. Question would be SRO-only and higher cog by providing specific plant parameters, such that multiple FR path procedures are met and having the applicant choose the highest priority procedure.

Licensee:

1) Rewrote question as suggested. This results in an H cognitive level rather than an F

2) Changed 562ºF-200ºF to 562ºF to -200ºF in initial conditions of stem per comment during prep week.

NRC:

1) Question, as revised is SAT.

2) Question is High cog as rewritten.

Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

/Units Logic Q-K/A License Level 100 F

3 2

M S

Free look Submittal

Form 2.3-5 Written Examination Review Worksheet Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

/Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.

ML22250A450

Text

Navigation menu

Search