ML061950377

1-2006 - Draft Written Exam Comments

ML061950377
Person / Time
Site:	Arkansas Nuclear
Issue date:	01/12/2006
From:	Gody A Operations Branch IV
To:	Forbes J Entergy Operations
References
50-313/06-301, ES-401, ES-401-9, NUREG-1021, Rev 9 50-313/06-301
Download: ML061950377 (5)

Text

ES-401 ANO1-12-2005 Written Examination Review Worksheet

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 1B H 2 X - E Need to link post-accident instruments on these panels in Notes section in order to satisfy K/A.

2M H 3 X - S Previous review by Mark Haire - possible GFES question.

3B H 3 X X X E Remove the word best from the stem. For B to be incorrect, need more info in stem to clue operator that RCPs were secured within the 2 minute time frame. Need EOP bases (for ICC?) to validate.

4B H 3 X X - U Remove the words which of the following is true from the stem-true false not allowed.

Restructure distractors so that it isnt a collection of True false statements. D distractor is not credible.

5B F 2 - S 6B H 1 X X - E Substitute question due to error in original question. Reword stem to remove between from the stem.

7 H 3 - S 8B F 2 S Instructions

[Refer to Section D of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either (F)undamental or (H)igher cognitive level.

2. Enter the level of difficulty (LOD) of each question using a 1 - 5 (easy - difficult) rating scale (questions in the 2 - 4 range are acceptable).

3. Check the appropriate box if a psychometric flaw is identified:

@ The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).

@ The stem or distractors contain cues (i.e., clues, specific determiners, phrasing, length, etc).

@ The answer choices are a collection of unrelated true/false statements.

@ One or more distractors is (are) not credible.

@ One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:

@ The question is not linked to the job requirements (i.e., the question has a valid K/A but, as written, is not operational in content).

@ The question requires the recall of knowledge that is too specific for the closed reference test mode (i.e., it is not required to be known from memory).

@ The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

@ The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved K/A and those that are designated SRO-only (K/A and license level mismatches are unacceptable).

6. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?

7. At a minimum, explain any U ratings (e.g., how the Appendix B psychometric attributes are not being met).

33 of 34

ES-401 2

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 9B H 4 X X - U Need inches for units added to stem. Need reference for SGs for isolation to validate distractor B.

10B F 2 X X - E Modify stem to make distractors more credible, stem more operationally based with alarms.

11 F 4 X - U Startup and low load block valves also close on reactor trip. Remove from stem.

12B H 3 - S Minor edit to stem to increase understanding of battery discharge rate.

13B F 2 - S 14 H 3 X - U C is not credible since instrument does not fail midscale. Remove mid-scale.

15B H 2 X - U Need references to validate distractors. Change B distractor to read...D01" to make it credible.

16 H 2 X - E Minor stem edit to include HPI P-36B is aligned to A4 bus. This is a swing pump that can be aligned to two diff. buses.

17M H 3 X - E Need references for C and D distractors.

18M F 2 - S System Title on exam incorrect (says excessive, should be inadequate),

point value is also wrong on outline, should be 3.4.

19B H 3 X - U D is a poor distractor. Consider Align RPI with API as substitute.

20B H 3 - E Minor distractor edit for D for clarity on stabilizing power.

21B H 3 - S 22B H 3 X - U B is a poor distractor. Consider Reset RE-4830 and Notify Nuclear Chem.

23 H 4 - S 24 F 2 - S 34 of 34

ES-401 2

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 25B H 2 - A distractor-should Tcold have leg after it? Outline is incorrect X E (changed KA from EA1.17 to EK1.03 in exam, not outline).

26 H 3 X - U Modify distractors and answer to include total feed flow because flow is re-ratioed on 1 RCP trip. Also consider removing temps from distractors because too similar to question 52.

27 H 3 - S 28B H 3 - S 29M H 3 - S 30 H 3 X - E Put equipment/valve numbers with noun descriptions.

31B H 3 - E Need description of connectivity of buses, preferential trips, etc. Label pump B P-34B in stem.

32B H 2 X - E reference is weak in supporting only 1 answer.

33B H 3 X - U D distractor is not credible-change from maximize to minimize letdown flow.

34 H 3 - S 35 H 3 X - E Answer D is actually incorrect for plant conditions in stem. Heaters would be off because of the cooldown in progress.

36 H 3 - S 37B F 2 - S 38B H 3 - S 39B F 2 - S 40B F 2 X - E(S) references weak in supporting the correct answer.

41 H 3 - E references weak in supporting the correct answer.

42B F 2 X E Wrong K/A was described for question. Recommend modifying the stem to include the procedure step where MSR stm valves are closed.

34 of 34

ES-401 2

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 43B F 3 - E Minor edit of B and D distractors-remove HI load limit and change setting values in same distractors to increase credibility.

44B H 4 - S 45B H 4 X - E Minor edit to stem to lower temp/pressure to ensure only answer is correct and distractors credible without providing SCM graph.

46M H 3 X - U Edit stem to include degraded power such that RCPs are secured (reqd for thumb rule in question).

47M H 4 X - U Is B6 downstream of A4, no guidance on B6 restoration in the provided reference? Yes, auto pickup of B6 when A4 renergized.

48B H 4 X - E Move if the auto transformer...service to stem for all four selections.

Place is between equipment and operable in the stem.

49 H 3 X - E Need references to confirm all distractors will not cause a trip.

50B F 4 - S 51B H 3 - S 52M F 2 - E Change distractor B since it is also correct. Also need references to confirm distractors.

53B F 3 X - E Need references to tie B62 to answer.

54M H 3 X - E Change stem pressure from 48psig to 50 psig (actual setpoint inprocedure for action). Also need to review entire procedure for distractor credibility.

55B F 2 X - E Outline pt value is wrong at 2.8* (it is actually 3.9).

56B F 2 - S 57B H 3 X - U(S) Too similar to question 26 - modify distractors accordingly.

58B H 3 X - U Need more references for answer completeness and distractors.

59M H 4 - S 60B H 4 - S 61B H 2 - S 62B H 4 - S 34 of 34

ES-401 2

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 63B F 2 - S 64M F 2 - S 65M F 2 - S 66B H 3 - S 67B F 2 X - U Distractors A and D do not seem credible. Are cell phones available for other comm used in CR extension?

68B H 3 - U Substitute question because....Also need references for distractors.

Outline pt value is wrong-should be 3.4, not 2.8. Remove the word for in distractor C.

69B F 2 X - U Distractor A symmetry - It should read verification of the associated trip actions of all logic elements for the channel. B distractor-replace known values with a known standard. For the D distractor - replace proper response with Operability.

70B F 3 - S 71B F 3 - S 72B H 3 - S 73B H 3 X - U Why cant Bdistractor also be correct?

74B H 2 - S 75B F 2 X X X - U Poor question for discriminatory validity. Stem focus on distractor makes it too easy. Outline pt value is wrong-should be 2.9, not 3.3.

34 of 34