ML020940828

From kanterella
Jump to navigation Jump to search
ANO2-2-2002-Draft Outline Comments
ML020940828
Person / Time
Site: Arkansas Nuclear Entergy icon.png
Issue date: 01/17/2002
From:
NRC Region 4
To:
Entergy Operations
References
50-368/02301
Download: ML020940828 (10)
v • d • e


Text

Page 1 of 6 ES-401 Form ES-401-9 FACILITY: ANO 1 Unit 2 EXAM DATE: Feb 2002 CHIEF EXAMINER: P Gage REVIEWER: R Lantz

  • Indicates part of initial sample
1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (ID #) (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 377 H 3 S from memory?

378 H 2 S 379 H 3 S R381* H 3 U S 382* F 3 U S 383 H 3 S R384 H 2 S 385 F 3 S should name procedure in stem 386* F 3 U S 387* H 3 U S 388 F 2 S 389* F 3 U S 390 F 2 S 391 H 4 S 392 F 3 S R393 H 3 U E switch a, b... change d to A-running unloaded, b-standby 394* H 3 U S R395 F 1 U U not licensed level... unless unique to the CR?

S396 F 2 S 397 H 2 S 398 F 3 E dist d seems incomplete

Page 2 of 6

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (ID #) (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 399 F 2 S R400 F 2 S R401 F 3 S R405* F 2 U U U U remove yellow from 2 dists, add some initial conditions to stem R406 F 2 S 407* F 2 U S S409* H 3 U U S S412 H 3 S S413 H 4 S S415 F 4 S S416 F 4 S S417 H 3 S S418 H 3 S S419 H 3 S S420 F 3 E edit typo in stem?, NRC notification? Time?

R421* F 3 U S S422 F 3 U U reword a and c... suggest c read that approval can be granted without completion of the 50.59 Evaluation S423* H 2 U U S S425 F 2 S S428 H 3 U U very similar to prior question, see 409 S429 F 2 S S430 H 2 S notes are in error?

R431 F 3 S R432* H 4 U S

Page 3 of 6

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (ID #) (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only R433 H 4 S R434* H 3 U S R435 H 3 S S436 H 2 U U E d not credible and cues answer verify others are incorrect S438* H 2 U U S R439 F 2 S S441 H 2 U E c does not make sense: suggest: b trip, d out of bypass, b bypass, d to trip 442 F 3 S S443 F 3 U E b- remove licensed limit (cue) 444 H 3 S notes have typo S445 H 2 S 037* H 2 U S S070* F 2 U N U 55.43 ref? Good Q, but not SRO only lvl.

S136* H 3 U U S required from memory?

S161 H 3 S is a manual SIAS performed as backup to auto?

219 F 3 S S296 H 2 S R298* F 2 U S 300 H 3 S are each a valid makeup source when power is available?

308 H 2 E d - delete remain at power and (cue) 309 F 3 S 310 F 3 S 311 F 3 U E make up another vent location for a, and or b

Page 4 of 6

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (ID #) (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 312* H 3 U S 313 H 3 S 314* H 3 U S 315 H 3 S 316* H 3 U S S318 F 3 S 319 H 3 S 320 H 3 S 321 F 4 E add auto (close) to A, B- add containment (press/temp) 322* H 3 U S R323* H 2 U S 324 H 2 S 325 H 2 S 326 H 2 S 327 H 2 U E SRO lvl? Pick 2 other distractors.. Answer does not challenge the Q 328 F 3 S 329 H 2 S S330* H 4 U N U SRO lvl needs action to be taken after diagnosis...

R331* H 3 U S 332 F 2 U U U similar concept asked in prior Q.. 327 S333* H 3 U U S R334 F 2 U S c- is it still incorrect if maximze.. MTS?

336 H 2 S S337 H 2 U N U similar prior Q, see 363... k/a not good match

Page 5 of 6

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (ID #) (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only 338 H 3 S R339* F 3 U S 340 F 2 U U QID 418 already test when to go the the FRP for SRO 341* H 3 U S any magnitude clarifiers needed?

342 H 2 U E R343 F 2 S R344* H 3 U S 345* F 3 U S 346 F 3 S from memory OK?

R347 F 2 S 348* H 2 U S R349* H 2 U S 350 F 3 S 351 H 2 S 352* F 4 U S R353 F 3 S 354* F 3 U S 355 F 4 S from memory OK?

356 H 3 S 357 F 4 S 358 H 3 S 359 H 2 S 360 F 3 S 361* H 4 U S

Page 6 of 6

1. 2. 3. Psychometric Flaws 4. Job Content Flaws 5. Other 6. 7.

Q# LOK LOD (ID #) (F/H) (1-5) Stem Cues T/F Cred. Partial Job- Minutia #/ Back- Q= SRO U/E/S Explanation Focus Dist. Link units ward K/A Only R362 H 3 S 363 H 2 U U similar to 337, and 362 364 H 2 E no handout? Better to add action required based on a reading in plant compared to graph reading 365 H 2 S no handout?

R366 F 1 U U GFES 367 F 3 S 368 F 3 S R369 F 3 S 370 H 3 S 371 F 3 S 372 H 2 S 373 F 4 S 374 H 3 S NUREG-1021

Dec 12, 2001 S:\DRS\OB\EXAMS\ANO\UNIT2\February 2002 Exam\Draft comments\DRAFT COMMENTS-AppE_F.wpd Ano1 unit 2, Feb 2002 Reviewer: Ryan Lantz Chief Examiner: Pual Gage Author: Facility APPENDIX E - REGION IV OPERATING TEST JOB PERFORMANCE MEASURE QUALITY REVIEW MATRIX

1. 2. 3. Attributes 4. Job Content 5. 6.

JPM# Dyn LOD Errors Explanation (D/S) (1-5) U/E/S (See below for instructions)

IC Cues Critical Scope Over- Job- Minutia Focus Steps (N/B) lap Link ROA1.1 S 2 S RO A2 S 3 S SRO A1.1 S 2 S SRO A1.2 S 3 S SRO A2 S 3 S B.1.a S 2 S B.1.b D 3 T E Need to have a failure criteria for stopping the EDG... ie, prior to load reaching 2000kw.

B.1.c D 2 N E weak alt task, only recognize not open then open. Swithc lineup B.1.d D 3 S B.1.e D 3 S consider adding a pressure band to establish, then observe applicant lowering pressure to establish using the aux spray.

B.1.f D 2 S simple switch lineup B.1.g D 2 S B.2.a S 2 S B.2.b S 3 S B.2.c S 2 S

- Page 1 -

Dec 12, 2001 S:\DRS\OB\EXAMS\ANO\UNIT2\February 2002 Exam\Draft comments\DRAFT COMMENTS-AppE_F.wpd APPENDIX F - REGION IV OPERATING TEST QUESTION QUALITY REVIEW MATRIX

1. 2. 3. Psychometric Errors 4. Job Content Errors 5. 6.

Q# LO LOD Explanation K (1-5) U/E/S (See below for instructions)

(F/H Stem Direct One Min Scope Job- Minu- SRO Back-

) Focus L/U Ans Resp (TS?) Link tia Only wards RA1.2a H 3 T E Specify that a reference or explanation is required within the response.

RA1.2b H 4 T T E Descriptive verbage OK (valve #'s not required); specify SAT acceptance criteria RA3.1 F 2 S RA3.2 F 3 S RA4.1 F 2 S RA4.2 F 3 S SA3.1 F 2 S SA3.2 F 3 S SA4.1 H 3 S SA4.2 H 3 S

- Page 2 -

APPENDIX F - REGION IV OPERATING TEST QUESTION QUALITY REVIEW MATRIX

1. 2. 3. Psychometric Errors 4. Job Content Errors 5. 6.

Q# LO LOD Explanation K (1-5) U/E/S (See below for instructions)

(F/H Stem Direct One Min Scope Job- Minu- SRO Back-

) Focus L/U Ans Resp (TS?) Link tia Only wards Instructions for Completing Matrix (Complete for either all items or all items requiring comment):

1. Classify level of knowledge (LOK) as either (F)undamental or (H)igher cognitive level.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when a psychometric error is identified:
  • Stem lacks sufficient focus to solicit only the answers listed (e.g., unclear on intent, answer needed, or unnecessarily negatively phrased)
  • Direct lookup (e.g., desired answer contained in obvious reference),
  • Question does NOT solicit single demonstrably correct answer,
  • Minimum response for passing credit NOT described in key,
  • Scope of question outside guidance of NUREG (e.g., why not how, TS bases not system, emergency not emergency plan),
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job, invalid K/A).
  • Recall of too specific knowledge,
  • RO test items test at the SRO job level or vice versa,
  • Reverse logic or application compared to job.
5. Based on the reviewers judgment, is the question as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column.

Operating Test Review Comments Admin Job Performance Measures Dynamic Scenarios Op test 1

- Page 3 -

Instructions for Completing Matrix (Complete for either all items or all items requiring comment):

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically an system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewers judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of problem in the explanation column.

- Page 4 -

Retrieved from "https://kanterella.com/w/index.php?title=ML020940828&oldid=2707360"