ML102160274

From kanterella
Jump to navigation Jump to search

Comments on Exam NRC Comments on Draft Licensee-Developed Exam (Written & Operating Tests) (Folder 2)
ML102160274
Person / Time
Site: Indian Point Entergy icon.png
Issue date: 05/03/2010
From: Caruso J
Operations Branch I
To: Ferrick J
Entergy Nuclear Operations
Hansell S
Shared Package
ML092470061 List:
References
TAC U01787
Download: ML102160274 (12)
v • d • e


Text

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LaD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Dist. Link units ward KIA Only

9. H 3 M S
10. H 3 M S
11. H 3 M S
12. H 3 B S
13. H 3 N E Added Mode 1 to stem. Added Service Water Header to Item 6 in stem for clarification.

14 H 3 M S 15 H 3 B S 16 H 3 B S 17 F 2 B S 18 F 2 N S 19 H 2 N S 20 H 2 M S 21 H 4-5 N S The licensee was asked and verified that Q is fair from memory for an RO.

22 H 2 B E Stem add procedure. The licensee was asked and verified that Q is fair from memory for an RO.

23 H 3 M E Added reference 24 F 2 N S 25 H 3 B S 26 H 3 B E Changed TSC in stem to CRS directs. Clarified wording on "0" 27 H 3 B S 28 F 2 M E Too easy simple set point Q action if CCW lost greater than or equal to 2 minutes. Modified stem to give alarm condition now have to analyze loss of CCW improved Q.

ES-401 2 Form ES-401-9

1. 2. 3. PsychometriC Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only

29. H 2 X N S Backward logic but ok since higher order Q.
30. H 3 X N E 1) RCS is at mid-loop with al S/G primary man ways removed is in all 4 answer choices put this statement n the stem.; 2) No reference provided? Items 1 and 2 complete 3) Explained why other answer choices are plausible
31. F 2 N S
32. F 2 M S
33. F 2 X X B E 1) Stem awkward wording; 2) "A" obviously wrong not credible what does conserving water have to do with PTS? Revised stem to clarify Q and rewrote "A" to Improve plausibility 34 F 2 B S 35 H 3 N E Provided better explan.ation for correct answer and distractors 36 H 3 M S 37 H 2 B S 38 H 3 X E Should you specify in stem raised power from 80-100% to match answer justification? Incorporated comment.

39 F 2 X X B U 1) KIA match What design feature or interlock being tested? 2) "N not plausible why even have fans if not needed? 3) 1"0" implausible not physically possible less sub-cooling reduces voids? Q replaced with Q 55 and selected KIA that matches and fits outline.

40 F 3 N S 41 H 3 N S Raised a concern regarding clarity of answer choices and alignment with the reference wording. Licensee indicated that the Q validated well without confusion 42 F 2 B S 43 H 3 B S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

00 LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N UlE/S Explanation Focus Dist. Link units ward KIA Only 44 H 3 N E Check switch nomendature is correct for "A" Added word "Reset" to switich 45 H 3 N S 46 H 3 X B E "0" implausible talks about 22 Recirc pump and in stem starting 21 Recirc pump? Fixed "D" to make more plausible also modify stem to make "D" more plausible?

47 H 4 N E Justifications for "C&O" the same? What are the power supplies to PORV 456C and irs block valve? Modified the answer choices and provided improved justification statements.

48 F 2 N S ES-401 2 Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N U/E/S Explanation Focus Oist. Link units ward KIA Only

49. F 3 X B E Provided better explanations for correct answer and distractors. Also revised "A" wording to make more plausible
50. F 3 N S
51. H 2 X B U KIA match How does loss of sw affect sw Ops? Re.placed with a Bank Q using the same KIA.
52. F 3 X X N E 1) Stem why not use term starting limitations as stated in SOP rather than "starting duty requirements"; 2) "C" not correct as stated two starts allowed if motor allowed to coast to stop between starts; 3) "D" more correct as worded Revised stem and answer to address comments
1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOO (F/H) (1-5) Stem Cues T/F CreeL Partial Job- Minutia #1 Back- Q= SRO B/MIN U/E/S Explanation Focus Oist. Link units ward KIA Only

53. F 3 N S 54 H 3 B S 55 F 3 X M U 1) KIA match is isolating FW considered part of containment isolation? 2) the reference [provided doesn't appear to support the answer; 3) Pzr press low causes Rx trip and Rx trip causes closure of FRVs? Q replaced with Bank Q 56 F 3 M E Explain why "on is not correct it looks like it is equivalent to WAn in net affect? Justication explains difference.

57 F 2 N S 58 F 2 N S 59 H 2 X N U KIA match - what design feature or interlock provides for maintaining cleanliness in this situation? Revised the stem answer choices to better match KIA 60 F 1-2 X X B U 1) KIA match - this is a manual action not an auto function of fuel handling? 2) LOO=1 answer is obvious given conditions; 3) *c&on don't appear to be plausible 4) I think the question is memory level. Question replaced with bank Q using different KIA.

61 H 4 N E 1) Do you have a reference to support the answer 2) why wouldn't power change due to negative reactivity from the increase in temperature? Revised two of the columns for Turbine and Reactor tripped or not tripped so that answer choices more straight forward AND objective.

62 H 2 B S 63 F 2 N S 64 F 2 M S 65 H 3 N E For the last bulleted Item In the stem added Actual rod position was 200 steps on CB D "at the start of the power accession" to clarify 66 F 2 B S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

00 LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #/ Back- Q:: SRO B/M/N utE/S Explanation Focus Dis!. Link units ward KIA Only 67 F 2 B E Added A TC RO to the stem to remove ambiguity then the Q becomes too easy.

68 F 2 N S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Focus cr~'lpartial Dis!.

Job- Minutia #1 Link Back-units ward Q= SRO B/MIN utEtS KIA Only Explanation

69. F 3 M E Explained basis for why fair Q for RO from memory.
70. F 1-2 N U LOD=1, too easy answer is obvious KIA tested on Admin JPM involving identifying blocking and tagging points Replaced Q with a TS bank Q
71. H 3 B S
72. F 3 X N E 1) Is this required knowledge from memory for an RO? 2) 0 as written is not completely correct move to an ALARA area or exit is what the procedure says? The site indicated that this is expected knowledge "0" was modified to reflect procedure verbiage.
73. F 3 M U GET level what makes this license level and in particular an RO responsibility. 00 the ROs need to oversee or approve this activity?

Q replaced with Bank Q testing different KIA 74 H 3 B S 75 F 3 M S 76 H 4 N S 77 H 3 M E "A&C" add "E-O" immediate actions are complete to answer choices for clarity Revised "A&C" to add E-O.

78 H 3 N S 79 H 3 M S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO BtM/N utEtS Explanation Focus Oist. Link units ward KIA Only 80 H 3 X? N E Add expected affected alarms to answer improving KIA match also provided better justifications for distractors 81 H 3 X B E Given that 235 gpm is the answer why wouldn't more SI flow be correct making "C&O" also correct? Added word minimum to answer choices so only one correct answer 82 H 2 B S 83 H 3 X N E The correct answer "N is the only choice with MOV-333 in it can we balance answer choices by adding this valve to another choice? liD" revised to add MOV-333 to balance answer choices.

Better explanations were added for why distracters are plausible 84 H 2 X N E 1) "A&B" state RM 55B has failed high ans stem says is at 'Warn" point; 2) Explanation and justifications talk about RM61A-0 and don't mention RM 55B? Revised A&B to improve plausibility additionally revised justification 85 H 3 X N E "0" does not seem plausible that there is no timit? Distractors and answer revised and reassigned question to question 98.

Replacement bank Q.

86 H 3 N S 87 H 3 N S 88 H 3 N S 5*401 2 Form E5*401*9

2. 3. Psychometric Flaws 4. Job Content Flaws I 5. Other I 6. I 7. I 8.

LOO (1-5) Explanation 3 N S 3 N S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Credo Partial Job- Minutia #1 Back- Q= SRO B/M/N utEtS Explanation Focus Dist. Link units ward KIA Only 91 H 3 X N E Added words "Main Turbine" governor oil pressure for clarity in stem. Furlher clarified justifications 92 H 3 N E Reworded stem to ensure compliance with TS and added explanation of how compliance with TS maintained 93 H 3 N E 94 H 3 B E Fix "C" justification. Done 95 H 3 B E Capitalize the word "ONE" Done 96 H 2 X N U Low operational validity borderline minutia. TS Bases not really SRO level are the RO expected to know TS bases? Replaced question using same KIA 97 H 2 M S 98 H 2 X N U Not SRO level really GET level non-license level. This question was replaced with former question 85 also had to further adjust answer choice to more accurately reflect wording in reference.

99 H 4 B S 100 H 3 M S

Operating Exam Comments General Comment for all JPMs - JPM formatting doesn't fit NUREG 1021, Appendix "C" see quality checklist ex. format listing tools and references, restrictions on step sequencing, task standard listed on tear off sheets for applicants, initial conditions and initiating cue listed together, etc.

Administrative JPMs:

SRO ADMIN 1 Review WCR-1

  • Difficult to grade
  • Make all computations critical
  • Assign a tolerance for calculations
  • Added "Candidate should determine that the Boron Addition to Compensate PWR Defect less Rods value for 10% is in error"
  • Label "Key" SRO ADMIN 2 Identify the Location for Spent Fuel Assemblies
  • Format
  • No additional comments during Validation SRO ADMIN - 2
  • This JPM was designed during Validation Week. Replaced JPM to initiate a Temporary Procedure Change.

Prepare a TPC for 2-AOP-ROD-1 Rod Control and Indication Malfunctions

  • Format
  • Move step 7 CUE to Initial Conditions
  • This JPM was rep/aced during Validation Week too simplistic with Review a Tagout SRO ADMIN - 3 Review a Liquid Radioactive Release Permit for 14 Liquid Waste Distillate Storage Tank
  • Format
  • Missing Units on Initial Conditions
  • Remove CUE step 5
  • Move CUE step 7 to Initial Conditions
  • Move handout form to last page
  • No Additional Comments during Validation SRO ADMIN -4 Classify Emergency Events requiring Emergency Plan Implementation (Time Critical)
  • Keep E-Plan declaration idea. Replace security event which is equivalent to a direct look-up with a challenging integrated plant event.
  • Replaced
  • Added Reference for IP-EP-410 Protective Action Recommendations RO ADMIN 1 Calculate Shutdown Margin with Untrippable Rod
  • Format
  • Add COLR to references
  • Adjust numbers such that calculated SDM is < COLR if possible
  • Check correct numbers on Answer Key
  • Add "using WCR-1 Page 2 Steps 1-11." to Initiating Cue.
  • Add Cue to JPM Step 17. "Examiner should hand candidate 2-GRAPH-RPC-6 Core Operating Limits Report (COLR) when asked for."

Perform CR portion of IR NIS COL

  • Replaced to simplistic did not test operator understanding
  • This JPM was replaced with Perform a Manual QPTR Calculation (3 Detectors 100% ) prior to validation week.

RO ADMIN - 1*2 Perform a Manual QPTR Calculation (3 Detectors 100%)

  • No additional comments during validation ROADMIN -2 Prepare a Tagout
  • Format
  • Typo "filter" step 6
  • Check references on step 6
  • List each component individually as critical including desired and restoration position
  • Revised Answer Key to reflect Common Nomenclature for Components ROADMIN -3 Calculate a Liquid Radioactive Release Permit for #14 Liquid Waste Distillate Storage Tank
  • Format
  • Revise "Your are directed to ... " statement
  • Missing Units on Initial Conditions
  • Remove CU Estep 5
  • Move CUE step 7 to Initial Conditions
  • Move handout form to last page
  • Added "The Discharge Flowmeter and Recorder are operable."to Initial Conditions
  • Revised Initial Condition for Chemistry Report to state less than the limit of SMM-CY 001.

Simulator JPMs:

SIM -1 Respond to a Misaligned Rod

  • Format
  • Added to JPM step 7
  • Added cue step 9 rod group position 174 steps SIM - 8 Restore Power to Bus 3 A using 22 EDG
  • Revise JPM to restore power to Bus 2A since procedure has typos SIM -4 Transfer AFW feed to Low Flow Bypass valves
  • Step 10 list valve numbers
  • End JPM at step 11 original JPM too difficult and not fair for one operator - 2 person operation In Plant JPMs:

Perform Required Actions to Dump Steam Locally

  • Add steps to clarify alternate path
  • During validation changed from 21 SIG to 23 SIG because of accessibility to valves Change Summary of Scenarios following Validation Week The first change to scenarios was the numbering. The scenarios are numbered 1,2,3,and 4(spare). The current Scenario 1 was not renamed. The new Scenario 2 was the old Scenario 4.

The current Scenario 3 was the old Scenario 5(spare). The current Scenario 4(spare) was the old Scenario 3. Due to a reduction in candidates taking the exam, there are only three scenarios needed plus a spare. All the scenarios required: 1) more detailed scripting to be added to list all required operator actions; 2) add more detailed turnovers; 3) add scripting to have the SRO applicants classify the event at the end of each scenario. Note: The spare scenario 4 and the original draft scenario 2 were not used on the exam so those scenarios are not included in the master exam file to allow the licensee to reuse these scenarios for a future exam.

Scenario 1:

  • Added malfunction for blowdown valve not going closed and noted it is a tech spec call forCRS.
  • Added a lot of detail to the expected operator actions section.

Scenario 2:

  • Renamed - was Scenario 4.
  • Added MSIV failure to 0-1.
  • Added BOP normal operations action to 0-1 and expected operator actions.
  • Added a lot of detail to the expected operator actions section.

Scenario 3:

  • Renamed - was Scenario 5(spare)
  • Added malfunction to 0-1 for MTG valve pair not closing requiring manual closing of MSIVs
  • Added a lot of detail to the expected operator actions section.

ES-401 Written Examination Review Worksheet Form ES-401-9

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7. 8.

Q# LOK LOD (F/H) (1-5) Stem Cues TtF Credo Partial Job- Minutia #1 Back- Q= SRO BtM/N UlEtS Explanation Focus Dist. Link units ward KIA Only

1. F 2 N S
2. H 3 N S Licensee made some revisions to the distractors based on high miss rate during validation
3. H 3 N S
4. H 3 N S
5. F 2 B S
6. H 3 N S
7. F 2 B S
8. H 3 X B U KIA Mismatch. Quest/on choices were rewritten to include manual actions required to restore AFW flow to all SIGs Instructions

[Refer to Section 0 of ES-401 and Appendix B for additional information regarding each of the following concepts.]

1. Enter the level of knowledge (LOK) of each question as either {F)undamental or (H)igher cognitive level.
2. Enter the level of difficulty (LOD) of each question using a 1 5 (easy difficult) rating scale (questions in the 2 - 4 range are acceptable).
3. Check the appropriate box if a psychometric flaw is identified:
  • The stem lacks sufficient focus to elicit the correct answer (e.g., unclear intent, more information is needed, or too much needless information).
    • The stem or distractors contain cues (I.e., clues, specific determiners, phrasing, length, etc).

The answer choices are a collection of unrelated true/false statements.

The distractors are not credible; single implausible distractors should be repaired, more than one is unacceptable.

One or more distractors is (are) partially correct (e.g., if the applicant can make unstated assumptions that are not contradicted by stem).

4. Check the appropriate box if a job content error is identified:
    • The question is not linked to the job requirements (I.e., the question has a valid KIA but, as written, is not operational in content).

The question requires the recall of knowledge that is too specific for the closed reference test mode (I.e., it is not required to be known from memory).

The question contains data with an unrealistic level of accuracy or inconsistent units (e.g., panel meter in percent with question in gallons).

The question requires reverse logic or application compared to the job requirements.

5. Check questions that are sampled for conformance with the approved KIA and those that are designated SRO-only (KIA and license level mismatches are unacceptable).
6. Enter question source: (B)ank, (M)odified, or (N)ew. Check that (M)odified questions meet criteria of ES-401 Section D.2.f.
7. Based on the reviewer's judgment, is the question as written (U)nsatisfactory (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
8. At a minimum, explain any *U* ratings (e.g., how the Appendix B psychometric attributes are not being met).

ES*401 2 Form ES*401*9

Retrieved from "https://kanterella.com/w/index.php?title=ML102160274&oldid=2556512"