ML023510157

From kanterella
Jump to navigation Jump to search
Init EXAM-08/2002 - NRC Draft Exam Comments
ML023510157
Person / Time
Site: Cooper Entergy icon.png
Issue date: 02/25/2002
From: Gody A
Division of Reactor Safety IV
To: Denise Wilson
Nebraska Public Power District (NPPD)
References
50-298/02-301 50-298/02-301
Download: ML023510157 (13)
v • d • e


Text

COOPER - 8/2002 ES-401 Written Examination Form ES-401-9 (R8, S1)

Review Worksheet

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only B1 H 3 Y E For answer d, remove loss of busses 1A and 1B to make it consistent with answer C.

B2 F 2 Y S B3 F 2 X Y E Change either answer a or c to use the startup transformer only 1 time. I believe with startup transformer in both answers, you dont have 2 plausible distinct distractors.

B4 F 2 Y S B5 H 2 Y S B6 H 2 Y S B7 H 2 X Y E This does not qualify as a modified question - see ES-401, D.2.f.

Delete How will DEH respond to the power increase? - modify what you are asking the candidate to decide. You are asking for the response of the bypass valves and DEH. Unable to verify answers with the given reference.

B8 H 2 Y S B9 H 2 Y S Ensure a Mollier diagram is included with the steam tables.

B10 H 3 X N U Question does not address K/A - somewhat related, but no inadvertent reactivity addition. Are answers b & d credible distractors?

B11 H 2 Y S

ES-401 2 Form ES-401-9 (R8, S1)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only B12 H 2 X Y E Answers a and c are not credible distractors.

B13 H 3 Y E Change M0 (Mzero) to MO.

B14 H 4 Y  ? Was training done on this type of event? Can they calculate the change in Rx vessel water level and come up with high level trip?

B15 H 2 Y S B16 H 3 Y S Verify no REC realignment. I thought REC got a signal for a Group 6 isolation.

B17 H 3 Y S B18 H 3 Y S B19 H 2 Y S Should the question read possible consequence?

B20 H 3 Y S B21 H 2 Y S Does not qualify as a modified question. The stem did not have a pertinent condition changed.

B22 H 2 Y E Try and balance answer lengths. May want to modify answer d - remove place Containment Cooling 2/3 Core Valve Control Permissive Switches in OVERRIDE. Each answer gets successively longer.

B23 H 3 Y S B24 F 2 X Y E Remove the 15 minutes from the question. Doesnt add anything.

B25 H 2 Y S B26 H 2 Y S B27 H 2 Y S B28 H 3 Y S B29 H 3 Y S B30 H 3 Y S B31 H 3 Y S

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only B32 H 2 X Y E Remove causing the core void fraction to rapidly rise. This leads the candidate to the correct answer. Let the candidate determine the impact of the rapidly opening bypass valves on voids.

B33 H 3 Y S B34 F 2 Y S Does not qualify as a modified question. The stem did not have a pertinent condition changed.

B35 H 2 Y S B36 H 2 Y S B37 H 2 Y S B38 H 2 Y/N U The question only addresses the Part a of the K/A. Part b is NOT addressed.

B39 H 3 Y S B40 H 3 Y S B41 H 3 S B42 F 2 S Correct KA?

B43 F 2 S (1)

B44 F 2 S (1)

B45 F 2 S (1)

B46 F 2 S (1)

B47 H 3 S (1)

B48 H 3 S B49 H 3 S B50 H 3 S B51 H 3 S B52 F 2 S B53 H 3 S B54 H 3 S Foil b.does not fit w/distractor b. Foil d. does not agree with L/P.

B55 F 3 S Foil d. should be "...radiation level above its max normal operating value.

Note (1)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only B56 F 3 S B57 F 3 S B58 F 2 S B59 F 3 S B60 F 2 S B61 F 3 S (1)

B62 H 3 S B63 F 2 S (1)

B64 F 3 S (1)

B65 F 3 S B66 F 3 S KA should also list EA1.01 (H2).

B67 f 2 S B68 F 2 U No correct answer. Requires editing.

B69 F 2 S See question for Foil a. on exam sheet for Q69 B70 F 2 S B71 F 2 S Direct lookup if T. S. pages provided.

B72 F 3 S R73 F 3 S R74 H 2 S R75 H 2 S R76 F 2 S R77 H 3 S R78 H 3 S R79 H 3 S "Skills" listed are not appropriate.

R80 F 3 S R81 H 3 S ^

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only R82 H 3 E ^Reference indicates ARI rod insertion is prevented. This is not consistent with answer.

R83 F 3 S R84 H 2 S R85 F 3 S R86 F 2 S R87 F 2 S R88 F 1 S R89 F 2 S R90 F 2 S R91 F 3 S R92 H 3 S R93 H 3 S R94 H 3 S R95 F 3 S R96 H 3 S R97 H 3 S R98 H 2 S R99 F 2 S R100 H 3 S SRO ONLY QUESTIONS S73 H 3 U U ^Not responsive to 10CFR43.5 and not SRO only 274 H 4 U U ^Not SRO only and not responsive to 10CFR55.43 S75 H 3 S ^Responsive to 10CFR55.43(2)

S76 H 3 U U ^No 10 CFR55.43 reference. Not an SRO only question.

S77 F 3 U U ^No 10 CFR55.43 reference. Not an SRO only question.

S78 F 2 S ^Reference should be to 10CFR55.43(7)

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only S79 F 2 S ^Reference should be to 10CFR55.43(7)

S80 F 3 S ^

S81 F 2 U E ^Should provide applicant more than TS 3.7.7 to make it more than direct lookup.

S82 H 4 U U ^No 10 CFR55.43 reference. Not an SRO only question.

S83 H 3 S ^More appropriate reference is 10CFR55.43(3)

S84 F 1 U ^Does not discriminate. Applicant only needs to look at one number in a handout table to answer.

S85 H 2 S ^

S86 H 1 U U ^Not discriminatory with handout. No 10 CFR55.43 reference. Not an SRO only question.

S87 H 1 E ^Item is not discriminatory if applicant is given the procedure to look up limits. Why is this not required memory knowledge?

S88 F 3 U U ^No 10 CFR55.43 reference. Not an SRO only question.

S89 H 3 S ^

S90 H 4 U U ^No 10 CFR55.43 reference. Not an SRO only question.

S91 F 3 S ^Reference should be 10CFR55.43(4)

S92 H 3 E ^Handouts should not be required S93 F 2 S ^

S94 H 3 S ^

S95 F 2 S ^

S96 F 2 U U ^No 10 CFR55.43 reference. Not an SRO only question.

S97 F 2 S ^

S98 F 2 S ^

S99 H 3 S ^

S100 F 2 S ^

^Part of initial sample checked for K/A concurrence General comment: Extensive handouts are provided in this exam. An analysis should be performed to determine if they give answers to questions designed as closed book.

COOPER 8/2002 C:\MyFiles\Checkout\CNS 08 2002 NRC DRAFT EXAM COMMENTS.wpd APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job 5. 6.

JPM# Dyn LOD Content U/E Explanation (D/ (1-5) Errors /S (See below for instructions)

S)

IC Cue Critic Scop Ove Jo Minuti Focu s al e r- b- a s Step (N/B lap Link s )

RO A1.1 S 2 E Cue sheet refers to ADMIN -A1.2 SRO S 2 E Cue sheet refers to ADMIN -A1.2 A1.1 RO/SRO S 3 S A1.2 RO A2 S 2 E Cue sheet refers to ADMIN-A1.2 SRO A2 S 1 N U U This task does not discriminate for SRO level. It may be acceptable if applicant were given data to make operability determination and then make appropriate log entry.

- Page 7 -

COOPER 8/2002 Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column. Provide conclusion on whether JPM SET criteria satisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).

- Page 8 -

C:\MyFiles\Checkout\CNS 08 2002 NRC DRAFT EXAM COMMENTS.wpd COOPER 8/02 APPENDIX F - REGION IV OPERATING TEST QUESTION QUALITY REVIEW MATRIX

1. 2. 3. Psychometric Errors 4. Job Content Errors 5. 6.

Q# LOK LOD Explanation (F/H (1- Ste Dire One Min Sco Job- Min SR Bac U/E/ (See below for instructions)

) 5) m ct Ans Res pe Link u-tia O k- S Foc L/U p (TS Only war us  ?) ds RO- H 2 S Satisfactory only as CLOSED A.3- REFERENCE 1

RO- H 3 U Almost identical to Question 95 A.3- in written exam.

1 SR F 1 U Does not discriminate.

O- Applicant only must know to A.3- assign highest dose job to 1 applicant with lowest accumulative does, etc., for satisfactory response.

SR H 3 S Satisfactory only as CLOSED O- REFERENCE A.3-2 RO- F 2 U E Job link is not clear. Only A.4- satisfactory if CLOSED 1 REFERENCE and there is a learning objective.

RO- F 2 S Satisfactory only as CLOSED A.4- REFERENCE 2

- Page 9 -

C:\MyFiles\Checkout\CNS 08 2002 NRC DRAFT EXAM COMMENTS.wpd COOPER 8/02 APPENDIX F - REGION IV OPERATING TEST QUESTION QUALITY REVIEW MATRIX

1. 2. 3. Psychometric Errors 4. Job Content Errors 5. 6.

Q# LOK LOD Explanation (F/H (1- Ste Dire One Min Sco Job- Min SR Bac U/E/ (See below for instructions)

) 5) m ct Ans Res pe Link u-tia O k- S Foc L/U p (TS Only war us  ?) ds SR H 2 U E Stem is unnecessarily focused.

O- Just say determine reporting A.4- requirements and time allowed 1 for reports to be made.

SR H 3 E Provide proposed key for each O- scenario.

A.4-2

- Page 10 -

COOPER 8/02 APPENDIX F - REGION IV OPERATING TEST QUESTION QUALITY REVIEW MATRIX

1. 2. 3. Psychometric Errors 4. Job Content Errors 5. 6.

Q# LOK LOD Explanation (F/H (1- Ste Dire One Min Sco Job- Min SR Bac U/E/ (See below for instructions)

) 5) m ct Ans Res pe Link u-tia O k- S Foc L/U p (TS Only war us  ?) ds Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it.

The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Classify level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when a psychometric error is identified:
  • Stem lacks sufficient focus to solicit only the answers listed (e.g., unclear on intent, answer needed, or unnecessarily negatively phrased)
  • Direct lookup (e.g., desired answer contained in obvious reference),
  • Question does NOT solicit single demonstrably correct answer,
  • Minimum response for passing credit NOT described in key,
  • Scope of question outside guidance of NUREG (e.g., why not how, TS bases not system, emergency not emergency plan),
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job, invalid K/A).
  • Recall of too specific knowledge,
  • RO test items test at the SRO job level or vice versa,
  • Reverse logic or application compared to job.
5. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column.

- Page 11 -

COOPER 8/02 C:\MyFiles\Checkout\CNS 08 2002 NRC DRAFT EXAM COMMENTS.wpd APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job 5. 6. 7.

JPM# Dyn LOD Content Set U/E Explanation (D/ (1-5) Errors /S (See below for instructions)

S)

IC Cue Critic Scop Ove Jo Minuti Focu s al e r- b- a s Step (N/B lap Link s )

A.1-1 D 3 S Start and Load the Diesel Generators A.1-2 D 1 S Respond to one RFP in Track and Hold A.1-3 D 3 S Perform SBGT System Decay Heat Removal A.1-4 D 3 S Perform Alternate Control Rod Insertion A.1-5 D 4 S Manually Initiate HPCI A.1-6 S 1 X U Perform D/W Calculation (this is not a JPM)

A.1-7 S 3 S Xfr H2/O2 Monitors B.1-8 S 3 x7 E Xfr an MCC to Alternate Power Supply B.1-9 S 2 S Install and Remove PTMs B.1-10 S 2 1 N E Reset RCIC from Overspeed Trip (may need more steps suchas as isol. Steam admission valve)

Notes: All JPMs should designate whether they are time critical or not. Per ES-C-1 Some Jpms state whether it is performed as a faulted or Non-faulted JPM (inconsistent)

General Conditions should be stated as Initial conditions and should be included on the applicants handout sheet.

- Page 12 -

COOPER 8/02 Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. JPM SET criteria sateisfied (i.e., number/distribution of safety functions, A.3 and A.4 integrated with parts B/C, Admin topics per section meet ES).
5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column.

- Page 13 -

Retrieved from "https://kanterella.com/w/index.php?title=ML023510157&oldid=2681752"