ML103400280

PV-2010-11-Draft Written Exam Comments

ML103400280
Person / Time
Site:	Palo Verde
Issue date:	11/05/2010
From:	Brian Larson Operations Branch IV
To:	Arizona Public Service Co
References
Download: ML103400280 (9)

Text

ES-401 Written Examination Review Worksheet Form ES-401-9 PV-2010-11 Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 B 5 (easy B difficult) rating scale (questions in the 2 B 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.

7. Based on the reviewer=s judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

8. At a minimum, explain any AU@ ratings (e.g., how the Appendix B psychometric attributes are not being met).

Page 1 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 1

F 3

N S

D/41.7 Modify stem to read Which ONE of the following describes the next appropriate action in accordance with SPTAs. Otherwise answers A and C would trip the reactor and accomplish same as D. Implemented Recommended Change.

2 H

2 B

E B/41.10 Change subcooling from 10F to something closer to the 24F limit (ie 20 or

22) and remove is continuing to degrade since this statement is irrelevant to actions. Implemented Recommended Change.

3 H

3 N

S B/41.6 Will letdown isolate on low PZR level? If so, which occurs first, low level or high regen hx temp. Lesson material not attached to question to verify. Letdown will not isolate on Lowering PZR level. Attached lesson material to support.

4 H

3 N

E B/41.7 Distractor A could be correct if SDC train B was placed in service at a lower

ES-401 2

Form ES-401-9 PV-2010-11 Page 2 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only flowrate. Lowering flowrate is the first step of procedure for vortexing. Changed distracter to place SDC Train B in parallel operation with SDC Train A.

5 H

3 N

S B/41.10 Justification for C & D needs correction to correspond with answers.

Adjusted justification to match question order.

6 F

2 N

S D/41.7 7

H 3

N S

B/41.5 8

F 3

N E

A/41.5 There appears to be some ambiguity in the correct answer. Procedurally RCS pressure is lowered in Step 12 to within 50 psi before actually isolating in Step 15, which would also make C correct. Please advise to this. The Cooldown and Depressurization will occur at the same time. Thot of 540 F will be reached much earlier than the RCS will reach the pressure band. We expect the SG isolation to begin as soon as Thot is less than 540 F without regards to RCS pressure. Bolded the word AND in distracters C and D.

9 F

2 B

S A/41.10 10 F

3 N

S C/41.10 11 F

3 X

B U

B/41.10 Although question does reference LOOP, it is testing procedure use and adherence and doesnt address procedures used for a LOOP. Resampled KA. Replaced with 2.4.46 12 H

3 N

S D/41.7 13 F

2 N

S C/41.7 14 H

3 B

E B/41.4, 41.8 Distractor D informs the applicant to Ensure adequate flow to the Fuel Pool Heat Exchangers if aligned This requires the Applicant to make an assumption of alignment in the question where it is not stated. Could state in the stem whether the other train is aligned or not in order to clarify this distractor. Changed distracter to eliminate assumption.

ES-401 3

Form ES-401-9 PV-2010-11 Page 3 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 15 F

3 N

E C/41.10, 41.8 Distractor D appears as a subset answer to Distractor A, with both ensuring a negative pressure is maintained. Changed Distracter D to remove negative pressure and state maintain ventilation.

16 H

3 N

S B/41.10, 41.8 17 F

3 B

S D/41.10 18 F

3 N

S D/41.7 19 H

3 N

S C/41.2 20 H

4 B

S B/41.5 From NRC 2008 exam 21 F

3 N

S D/41.7 22 H

3 B

S D/41.5 23 F

2 N

S B/41.10 24 F

2 N

S C/41.10 25 H

2 N

S A/41.7 26 F

3 B

E D/41.10 Distractor A, LOAF, is not plausible since the stem contains a bullet that AFB-P01 is controlling S/G levels. Removed AFB controlling SG levels. NRC concurs per phone discussion.

27 F

2 N

S D/41.7 28 F

2 X

N U

A/41.3 K/A Mismatch. Question deals with RCP seal flow parameters and not physical connections or cause/effect relationship of CVCS to the RCPs.

Changed question to diagnose a RCP seal failure per Recommendation.

29 H

3 N

S C/41.7 30 H

3 N

S D/41.7 31 F

2 N

S A/41.7 32 H

3 X

N U

B/41.10 2 part K/A - Question does predict impacts of malfunction but does not use procedures to correct, control, or mitigate. Also, CFR referenced in question is not correct. 41.10 more closely matches since guidance is in abnormal op

ES-401 4

Form ES-401-9 PV-2010-11 Page 4 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only procedure. Made recommended changes, Changed CFR link to 41.10.

33 F

2 B

S B/41.7 34 F

3 N

E B/41.5 Made all distracters odd numbers.

35 H

3 N

S C/41.5 36 F

2 X

N U

D/41.4 K/A concerns effects of loss of CCW on CCW cooled loads. Question as written concerns loss of Turbine Cooling Pumps. Do not see connection between TC and CCW. Changed question to NC vice TC.

37 F

2 N

S A/41.7 38 H

3 B

S C/41.5 39 F

3 B

S A/41.7 40 H

3 B

S B/41.7 41 F

3 B

S A/41.7 42 F

2 N

S D/41.5 43 H

3 N

S C/41.5 44 H

3 N

S A/41.7 Justification is wrong for distractor C. Addresses AWP and not quick open as stated for C. Add noun name for SBCS 1007, or should applicant know from memory? Justification changed to reflect Quick Open block as a distracter. No noun name required as Applicant should know this from memory.

45 F

2 N

S C/41.10 46 F

2 B

S B/41.7 47 H

3 N

S B/41.7 48 F

2 B

S C/41.5 49 F

2 N

S D/41.10 50 F

3 B

S C/41.5

ES-401 5

Form ES-401-9 PV-2010-11 Page 5 of 9 Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation OBDI 202 - IOLE Process Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 51 H

3 N

S B/41.7 52 H

2 X

N U

B/41.5 Question does not utilize PRM system per K/A statement. Changed question to RU-2 (EW Process Rad Monitor) indication is rising.

53 F

3 B

S D/41.7 54 F

2 B

S B/41.7 55 H

3 B

S B/41.9 56 H

3 N

S A/41.7 57 H

2 B

S C/41.5 58 H

2 N

S A/41.7 59 F

2 N

S B/41.7 60 F

2 N

S C/41.7 61 F

2 N

E B/41.7 Distracter D seems implausible to be in Single Element control after an uncomplicated reactor trip and no given instrument failures. Revised Question.

62 H

3 B

E B/41.5 Question is written as 2 part answer however only need to know 1 part of question to get the right answer. Make one part question or revise distracters.

Changed to One out of Two taken twice.

63 F

2 N

S D/41.5 64 F

2 N

S C/41.5 65 F

2 B

S C/41.7 66 F

2 N

S A/41.10 67 F

3 N

E C/41.7 Concerned that Distracter B could be seen as correct. Changed distracter B to state on cubicle door.

68 H

2 B

S A/41.10 69 F

2 N

E A/41.5 Question is written as 2 part answer however only need to know 1 part of

ES-401 6

Form ES-401-9 PV-2010-11 Page 6 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only question to get the right answer. Make one part question or revise distractors.

Changed to One out of Two taken twice.

70 F

3 N

S C/41.10 71 F

2 N

S A/41.11 72 F

2 M

S C/41.12 73 F

2 N

S C/41.6 74 H

3 N

S B/41.10 75 F

3 B

S A/41.10 1

H 2

X N

U A/43.5 Too easy. Based on plant conditions provided in stem, the casualty is easily diagnosed as LOCA. Only one answer includes a procedure for LOCA.

Changed to One out of Two taken twice.

2 H

3 X

N U

A/43.5 Stem says charging is secured due to leak. Not plausible to restore charging flow. B& C not credible based on stem. Changed distracters.

3 H

4 B

S B/43.5 Used on 2007 NRC SRO Exam. Changed LOD to 4 4

H 3

N S

D/43.5 Changed LOD to 3.

5 H

4 B

S B/43.5 Used on 2009 NRC SRO Exam 6

H 3

B E

A/43.5 Needs more clarification as to why C is wrong. Based on stem, SG#2 level is <45% and not being restored.why is HR-2 criteria satisfied? Changed stem to have level at 49% WR and AFW to #2 SG at 150 gpm to make HR-2 satisfied.

7 H

4 B

S C/43.5 Used on 2007 NRC SRO Exam 8

H 3

N S

B/43.5 9

H 4

N S

C/43.5 Provide more procedural guidance for answers. What is procedural guidance for priorities? Where does functional recovery specify cross tie EDG-A to PBB-S04?

Hard to verify answers. Enhanced ref material to support answers.

10 F

3 N

E B/43.5 Provide more specifics in stem (i.e. in accordance with 10CFR50.72,

ES-401 7

Form ES-401-9 PV-2010-11 Page 7 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only Immediate notification requirements for operating nuclear power reactors.

Implemented recommended changes.

11 H

3 B

S A/43.5 Used on 2009 NRC SRO Exam 12 H

4 B

S A/43.5 Used on 2007 NRC SRO Exam 13 H

2 B

S D/43.5 14 H

3 N

S C/43.5 15 H

2 N

E C/43.5 Remove SG pressures are 1170psia, maintained with SBCS in automatic from stem. Subcooling should provide adequate information to choose correct answer. Removing SG pressures will increase LOD. Implemented recommended changes.

16 H

3 B

S C/43.5 17 H

3 N

S B/43.5 18 F

3 N

S D/43.2 Is this question in accordance with the outline for t2/g1? Provided incorrect revision of T2/G1 ES401-2 and rejected KA list should show that 2.4.44 was rejected and replaced with 2.2.40.

19 H

3 N

E D/43.5 B seems a little drastic and maybe not credible. Changed distracter B from 3 hours3.472222e-5 days <br />8.333333e-4 hours <br />4.960317e-6 weeks <br />1.1415e-6 months <br /> to 6 hours6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br />.

20 F

2 N

S A/43.5 21 F

2 B

E D/43.3 Distracters A and B do not seem credible. Revised distractors.

22 F

3 B

E C/43.4 Answer justifications are not correct. Add in accordance with 74RM-9EF20, Gaseous Radioactive Release Permits and Offsite Dose Assessment to stem.

Implemented recommended changes.

23 F

3 N

S B/43.4 Justification needs correction. Split the stem into 2 statements for clarification. Implemented recommended changes.

24 H

3 B

S A/43.5

ES-401 8

Form ES-401-9 PV-2010-11 Page 8 of 9 OBDI 202 - IOLE Process Q#

1.

LOK (F/H)

2.

LOD (1-5)

3. Psychometric Flaws

4. Job Content Flaws

5. Other

6.

B/M/N

7.

U/E/S

8.

Explanation Stem Focus Cues T/F Cred.

Dist.

Partial Job-Link Minutia

1. /

units Back-ward Q=

K/A SRO Only 25 F

3 N

S C/43.5 RO TOTALS:

B= 24 F= 45 E= 11 Additional Notes:

M= 1 H= 30 U = 5 N= 50 S = 59 SRO TOTALS:

B= 11 F= 7 S= 17 Additional Notes:

M= 0 H= 18 E = 6 N= 14 U= 2

GENERAL COMMENT

S:

1. Bank questions are indicated by B; Modified are indicated by M; New questions are indicated by N

2.

Chief Examiner comments are indicated in red.

3.

Average difficulty is __ 2.56__ on the RO exam and 3.0_ on the SRO exam.

4.

The 10CFR55.41/43 distribution is: RO / SRO 41.1 = 0 43.1 = 0 41.2 = 1 43.2 = 1 41.3 = 1 43.3 = 1 41.4 = 2 43.4 = 2 41.5 = 16 43.5 = 21 41.6 = 2 43.6 = 0 41.7 = 30 43.7 = 0 41.8 = 2 41.9 = 1 41.10 = 18 41.11 = 1 41.12 = 1 41.13 = 0 41.14 = 0

5.

The answer distribution is: RO / SRO A = 16 (21%)

/

7 (28%)

B = 24 (32%)

/

6 (24%)

C = 20 (27 %)

/

7 (28%)

ES-401 9

Form ES-401-9 PV-2010-11 Page 9 of 9 OBDI 202 - IOLE Process

6.

T D = 15 (20%)

/

5 (20%)

here are 0 questions with attachments provided.