ML22336A126

U2 - NRC Comments on Proposed Licensee-Developed Written Exam

ML22336A126
Person / Time
Site:	Millstone
Issue date:	12/02/2022
From:	Brian Fuller NRC/RGN-I/DORS/OB
To:	Jeffrey Riley Dominion Energy Nuclear Connecticut
References
EPID L-2022-OLL-0007
Download: ML22336A126 (12)

Text

Form 2.3-5 Written Examination Review Worksheet for Millstone U2 Sept 2022 NRC Examination Key: * - presubmitted Q w/ref - question has reference for applicant Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 1

H 3

N S

Remove and and replace with semicolon 2*

H 3

B S

Answer choice C.2 needs to be a complete sentence D.2 is not plausible voiding and accurate level 3

F 2

N E S Overlaps with Q# 33. Resample K/A Make CETs singular in A2 and C2 4 w/ref H

2 N

E S Typo in stem : No. 3 and 4 5

F 2

X N

U E S Backwards logic Overlaps with Q#27 Resample K/A.

Correct the Part 55 reference for the question.

6 H

2 M

E S Add punction marks after answer choice letters.

Explain pump alignment in the justification.

Remove the period at end of D.2. phrase 7

H 3

B S

8 H

3 B

S C only and D all heaters on to make these answer choices more plausible. C. Turn on backup heaters only D. Turn on all heaters.

9 H

3 B

S Justification will be fixed to remove time component 10 H

3 B

S 11*

H 3

X N

S K/A mismatch K/A reselected to AK 2.09 IR 3.7 12 H

3 N

S 13 F

2 N

S 14 H

3 N

E S How is this testing AOP for LOOP? Millstone actions for LOOP are in an EOP. Keep the K/A but note that AOPs are not possible for LOOP. Does initial power affect answer to this question (decay heat diffirences)? Yes will add initial power 100%

15 H

2 N

S 16 H

3 N

E S K/A tested backwards. Test loss of remote/auto not what can be operated remotely

Key: * - presubmitted Q w/ref - question has reference for applicant What is questions source-missing from pedigree (NEW) 17 H

2 N

S Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 18 w/Ref H

3 N

E S Does applicant have to assume that there is a degraded voltage situation on the grid? Millstone will add grid capacity issue.

Need to specify in stem what is the preferred method to raise VARs since A and D could be correct Millstone will fix stem and get rid of You 19 H

2 N

S 20 H

3 N

S Use of you in stem could make more than one answer correct. Action should be singular since only one action is listed in part 1 of answer choice.

Explanation for answer choice C is incorrect. Millstone will adjust all of these.

Fix typo in C justification but cooldown is not power is rising 21*

F 2

N S

22 F

2 M

S Math equation written incorrectly in justification.

Recheck answer choices.

23 H

3 N

S No definition of adverse containment.

24 H

3 X

N U E S Add to stem: A manual valve that is supposed to be closed is found in the open position Too easy-Tier 1 question that is more of a Tier 4 question. Distractors are flawed. New K/A sampled:

G 2.2.22 Knowledge of limiting conditions for operation and safety limits 25 F

3 B

S 26 F

2 X

N U E S Address backwards logic by giving parameters and asking about SDC placed in/out service. Make sure temp/pressures agree Remove periods on all answer choices

Key: * - presubmitted Q w/ref - question has reference for applicant 27 H

3 N

S Do answer choices need to say align and start? YES 28 H

4 N

S Does it meet K/A? Is this part of CVCS. Go through procedure and make sure answer choices are correct.

Avoid use of your actions in stem. Add a K/A justification because procedural controls maintain temperature 29 H

3 N

S Verify SDC Valve 651 is CIV 30 F

3 N

E S Verify correct answer vs. justification. They contradict.

Change so that the question asks: Select the normal electric alignment and valve indication 31*

F 2

N S

Key: * - presubmitted Q w/ref - question has reference for applicant Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 32 F

3 N

S B. could also be correct. Make all answer choices match by using the same terms: move/not move.

33 H

3 N

E Overlaps with Q#3-Plan is to rewrite Q3.

Remove YOU from stem Change answer choices to Rises instead of Rise to match singular noun pressure 34 F

3 B

S Is SDC through shell or tubes. Deconflict answer choice B with justification writeup. Subset problem with C. add procedure to stem 35 F

3 B

E S For K/A match, do any RBCCW loads have TCVs?

Millstone will add K/A justification. Only TCV is letdown system.Add a procedure reference in stem so that answer choice D is not also correct.

36 H

3 M

S 37 H

3 M

E S Dont use describe in stem. Based on this, which of the following is correct? What is def of a partial trip?

No definition of partial trip. Change D to No effect on RPS 38 F

2 N

S 39 H

3 N

S 40 H

3 X

N U E S FIX distractors-c and d never would get picked.

In stem, add automatically in front of actuate in both

1. And 2. Questions. In justification, fix D. correct incorrect it should be correct 41 F

2 N

S This is a new GF question. Change to F level and Change answer choice B to screen clogging 42 F

3 X

N U E S Does not meet K/A for corrosion control Selected a new K/A for this one. Is containment pressure given in stem the normal units for obtaining this reading?

Yes, NR is in inches of water. Millstone will CTMT Pressure reading change to PSIG from safety instrument and convert to 0.5 PSIG 43 H

2 N

S 44 H

3 N

E S Reword question such as Based on these indications, what instrument has failed and HOW 45 F

2 X

M U E S K/A not fully met. What is impact of this improper operations? For example is low speed stop still acceptable. Millstone will add more to justification to explain the IMPACT portion.

Key: * - presubmitted Q w/ref - question has reference for applicant Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 46 H

3 M

S 47 F

2 M

S 48 H

3 N

S 49 F

2 N

S 50 F

3 N

E S Fix OPERATIONs in C and D answer choices to match.

Do not understand the part 2 question and correct answer choices for where to enter the setpoint.

Setpoint entered at RC 14 only. Info for the setpoint is entered into PPC (background, correction factor etc)

Take associated information out of stem and new answer choice is A.

51 H

3 X

N U E S Backward logic, change to WHICH of the following are indications of a failure?

Remove periods on each answer choice 52 H

3 B

E S NEED new K/A because no IA isolations to containment Make answer choices A-D sound the same i.e. with and for or as they are all a little bit different.

53 H

3 X

N S

Is it possible to challenge containment pressure with low pressure? YES Add indicated pressure in stem to be more accurate and take YOU out of stem.

This is technically INDICATED containment pressure and not actual containment pressure. How is containment pressure measured ( PSIG or PSIA)?

54 H

4 N

S We need to talk about this one to understand the distractors and the note at the bottom from B. Bergin. I Is this backwards logic? Given them the bus loss and ask them how the plant responds.

Change stem to have Tcold value

Key: * - presubmitted Q w/ref - question has reference for applicant 55 F

3 M

S 56*

H 3

N S

Fix distractor analysis in D. There is still mention of lower temperatures.

57 H

3 B

E S Is this failure of loop temp to 533F plausible?

Given a failed instrument, test what is Tave and how it is removed.

Fix A2 and C2 to read Automatically and not Automatic In plausibility of D, fix my to may.

58 F

3 N

S Is there a better distractor that could be used? Yes change to core region for C and D. and Above core region A and B.

59 H

3 M

S Fix question terms, dont use expected Change to program band and add bullets for plant conditions.

60 F

3 N

E S Reword A to be more like B. See suggestion Is this RO knowledge? Need a new K/A because not RO knowledge.

Delete pump from B2 and D2 because redundant What actionsnot what action 61 H

3 M

E S Is it unreasonable to expect a min. competent operator to know that the valve fails closed? Is (fail close) necessary in the stem? How is IA being tested if this is provided?

Reselect K/A because there is limited interaction between IA and condensate. Cannot write a discriminating question for this K/A.

In stem (fails closed) placement - it is not clear what valve fails closed. Move to end and state CN-218 fails closed). Answer choice A and B plausibility analysis would not be plausible if we tell them in the stem that the valve fails closed.

Put periods on all answer choices or dont use any at all. Be consistent.

In justification, fix third sentence so it makes sense (valve is missing) 62 F

3 N

S Reword stem question so it is not awkward. What valves close due to resulting ESAS signal?

Key: * - presubmitted Q w/ref - question has reference for applicant Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 63 F

2 X

M U E S A is a subset of B. Change A to Only one 64 F

2 X

B E S Remove - in stem. Answer Choice D is also correct based on justification. Check answer choice C is not also correct. This is NOT a Tier 3 generic question. It is testing system specifics. Ensure that Tier 3 questions focus on plant-wide generic K/As and not system-specific knowledge or knowledge of specific emergency/abnormal plant evolutions.

Reselect a new K/A is for Tier 3 Answer choice A is not lined up with the other choices. Make the answer choices complete sentences so that they are easier to read and match better: for example Walk down the control boards once per hour Answer choices C is not plausible, what are the minimum requirements then No requirement? Use something different or take out the phrase in parenthesis.

65 F

3 B

E S This is not a Tier 3 generic question. It is testing specific system knowledge Millstone will try to find better/generic question.

The referenced OP uses the term RO not CO. Delete the word back after paraphrase. Make all answer choices match (i.e. ROs, perform a verbatim repeatback vs. repeat back verbatim).

Make all ROs 66 H

3 B

E S Put specified time period in front of 92 days, add equipment to first sentence. Remove current status in part 2 of question because it is confusing. Why is 3 months plausible? Remove at anytime because it makes the distractor seem wrong.

Is this RO knowledge? Yes B is a subset of A. Millstone will fix it.

Delete periods after all part 1 answer choices. A2/C2 are misaligned, B2 does not make sense as written, change to: As long as the surveillance is completed SAT, within the next 92 days.

67 F

3 X

X N

U E S A needs to be fixed so that it is correct. B would also be correct because Tags Plus is a tagged component

Key: * - presubmitted Q w/ref - question has reference for applicant plus a device. Tagged component not Tagged Device to make the other answer choices correct.

Change to fundamental.

Answer choice D needs the word valve or tag in it to make it more plausible and not stick out as a distractor.

68 F

3 M

E S Fix part 1 of question to be reliable/not reliable Fix part 2 of question to reference procedure directed action and not your response Remove yes/no in answer choices replace with Rel/Not rel.

Reword stem question and answer choices in part 2 as follows:

The above Annunciator Response Procedures (ARPs) direct 2. RM-8132 purge.

Starting/securing 69 F

3 N

E S Could answer choice B also be correct in some instances? Does applicant have to assume anything in this answer choice?

This is not a Tier 3 question. Millstone will re-try to get a generic question.

Stem is not actually a question. Delete ?

Take always: out of A and C. To make C more plausible state: AOPS can no longer be used.

Delete determine the most immediate need from B 70 H

2 N

S Fix justifications in B and C they are incorrect, equal to is not one of the answer choices, should it be?

Move 1 to answer choice 71 F

2 B

S Add directly from fission to answer choices for fission.

72 H

3 B

S Stem is worded a little awkwardly. Change to As power is 1., CEA insertion limits ensure that CEAs will be positioned higher in the core to compensate for the change in 2.

73 F?

3 B

S Is this Millstone specific CEA #s? change against to to or with Remove group numbers and CEA #s Add not center CEA Remove paratheses and add after insertion to stem Move ASI and Radial Peaking to answer choices.

74*

F 3

N S

75 H

3 N

S Fix grammar in stem. Explain plausibility of answer choice A

Key: * - presubmitted Q w/ref - question has reference for applicant Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. K/A Use Flaws

6. Source (B/M/N)

7. Status (U/E/S)

8. Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job Link Minutia

1. /Units Logic Q-K/A License Level 76 F

3 N

S 77 F

2 N

U Vapor space accident is not tested. Millstone says there are no actions outside the MCR for a vapor space accident. Reselect K/A Fix 55.43 item reference New K/A selected and question written. Question has NO CORRECT answer as revealed during post exam comments. Marked UNSAT per 1021 ES-5.1 G.

78 H

3 N

E S Millstone will fix stem organization and take charging out of it.

Change to feeding with AFW in stem 79 H

3 N

S Do not use describe your actions in the stem 80 H

2 M

E S Reselected G 2.2.44 for Loss of Vital AC to allow writing a discriminatory question.

Split the annunciators that come in from the loss of indications. Numerous annunciators including Loss of indications including What actions will be taken in stem Remove period from part 1 of all answer choices 81*

H 3

N S

This is not the right K/A. Should be AA2.07 Ability to determine and/or interpret the following as they apply to Loss of Instrument Air: Determination of whether backup nitrogen supply is controlling the valve position Reselect AA 2.09 Ask Millstone to explain why this SRO only 82 H

3 X

N U E S More closely aligned with a 55.43(b)(2). Is this SRO-only? There is a typo in the justification writeup (lever).

Millstone will work to make this SRO only using TS.

83 F

3 X

N U E S Not SRO only, check 55.43(b) reference, appendix R details avoid the use of you in the stem. Millstone will work on making this SRO only 84 H

3 X

N U S Initial conditions do not match justifications for correct answer (pressurize pressure is stable in the ICs).

Some editorial errors in the stem to fix see comments.

Key: * - presubmitted Q w/ref - question has reference for applicant Backwards logic. Give them indications and ask them to diagnosis what the plant is doing (leak getting worse or cooldown).

Millstone will fix backwards logic and provide SRO only reason.

85 H

3 B

E S Part 1.of each answer choice should be more specific.

Spelling/grammar errors, is this SRO only, avoid use of You would in stem because it is too subjective Add EOP to stem, make AC match steps in EOP.

Fix title of EOP 2528 in stem.

86 H

3 B

S 87*

H 3

N E S Backwards logic. Yes. Change to given an increase in RBCCW flow then test what happens to power and what procedure to include. Watch cueing with turbine load in steam.

Fix stem questions, they are redundant and awkward to read.

88 F

2 X

N U E S Change Part 2 of question to not test between Mode 4 or 5. What is the other basis for this level? Keep as a two part but test the other bases for the level.

What are temperatures for cold shutdown? More than 1 right answer. Check for subset issues.This is RO level knowledge (basis knowledge of above the line TS)

Millstone rewrote entire question. Question is SAT.

89 F

3 N

S Fix grammatical error in first sentence operating feeding Is this really testing the K/A for DC distro system? Is that control power for breakers?

Removed operating. And will improve the justification to show DC control power and affect on AC buses 90 H

3 N

E S Does RVLMS and PZR level need trends like in Q#85 Avoid use of you in stem because subjective No trend for PZR level at 0% RVLMS changes are in discrete levels and it takes time to show changes.

C and D are not as plausible with pressurizer level at 0%. Is it possible to make it higher?

91 F

3 B

E S Remove extra periods in answer choices A and C.

Key: * - presubmitted Q w/ref - question has reference for applicant 92 H

3 N

E S Flip order of 1 and 2 questions and all the answer choices.

93 H

4 N

E S Switch order of questions for 1 and 2 (In accordance with TS _, or label LCOs as Tech Specs 94 w/ref H

3 N

E S Add acceptable or not acceptable to answer choices.

Is very slowly necessary?

95 F

3 B

S 96*

F 2

N S

97w/ref H

2 N

E S Concern about answer choice D because of conservative decision-making. Could this also be the right answer? Also-what is missing regarding check valve not seating properly in stem? Millstone will fix the justification (no check valve involved.)

Remove periods from answer choices 98 H

3 N

E S Explain plausibility of reducing secondary chemical concentrations in the S/G. Do not use you in the stem. Add a procedure reference for keeping the plant online.

99*

F 2

N E S Typo in answer choice C. In an ALERT, is it plausible that a PAR or exposure authorization would be needed? Change to SAE. Fix AC C personnel Fix a SAE in stem.

100 F

3 N

E S Fix stem: take describe your actions out of it and replace with more objective direction

Form 2.3-5 Written Examination Review Worksheet for Millstone U2 Sept 2022 NRC Examination Key: * - presubmitted Q w/ref - question has reference for applicant Form 2.3-5 Instructions for Written Examination Review Worksheet Refer to ES-4.2 for the definitions of terms used in this worksheet for the written examination.

Review each question (Q) as submitted and as subsequently revised and document the following in the associated worksheet columns:

1. Enter the level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) from 1 (easy) to 5 (difficult); mark direct lookup questions (applicant can directly determine the answer from the provided reference) as LOD 1. A question is (U)nsatisfactory if it is LOD 1 or LOD 5.

3. Check the appropriate box if a psychometric flaw is identified:

Stem Focus: The stem lacks enough focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information). This is an (U)nsatisfactory question.

Cues: The stem or one or more answer choices contains cues (e.g., clues, specific determiners, phrasing, length). This is an (U)nsatisfactory question.

T/F: All of the answer choices are a collection of unrelated true/false statements. This is an (U)nsatisfactory question.

Cred. Dist.: The distractors are not credible; single implausible distractors require (E)nhancement, and more than one noncredible distractor in the same question results in an (U)satisfactory question.

Partial: One or more distractors are partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by the stem). This is an (U)nsatisfactory question.

4. Check the appropriate box if a job content flaw is identified:

Job Link: The question is not linked to the job requirements (i.e., the question has a valid knowledge or ability (K/A) but, as written, is not operational in content). This is an (U)nsatisfactory question.

Minutia: The question requires the recall of knowledge that is too specific for the closed-reference test mode (i.e., it is not required to be known from memory). This is an (U)nsatisfactory question.

1. /Units: The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons). This is an (U)nsatisfactory question.

Logic: The question requires backward or reverse logic or application compared to the job requirements. This is an (U)nsatisfactory question.

5. Check the first box if a K/A mismatch flaw exists. Check the second box if the question is flawed because it is written at the wrong license level. Either condition results in an (U)nsatisfactory question.

6. Enter the questions source: (B)ank, (M)odified, or (N)ew. Verify that (M)odified questions meet the criteria of ES-4.2.

7. Based on the review performed in steps 2-5, mark the question as (U)nsatisfactory, in need of (E)nhancement, or (S)atisfactory.

8. Fully explain the reason for any (U) in column 7 (e.g., how the psychometric attributes are not being met).

9. Save the initial review comments and detail subsequent comment resolution so that each exam-bound question is marked by an (S) on this form.