ML16028A021

From kanterella
Jump to navigation Jump to search
2015-12 Draft Op Test Comments
ML16028A021
Person / Time
Site: Grand Gulf  Entergy icon.png
Issue date: 12/14/2015
From: Vincent Gaddy
Operations Branch IV
To:
Entergy Operations
References
50-416/15-012, 50-417/15-012
Download: ML16028A021 (12)


Text

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link General comments:

1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
4. Noun and numerical names are required for pumps, valves, and controllers.
5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion.
6. JPM standards should include a statement that all critical steps are completed satisfactorily.
7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM.
8. Your numbering system for the JPMs is not very easy to follow. It should be S-1, S-2, S-3, and in-plants should be P-1, P-2, etc.

RO (AR1) S 2 X U

  • Label on title page 2015 AR1
  • No KA assigned to this JPM you are required to assign a K/A to each JPM
  • Task standard is incorrect (Need to use something like applicant determines that the time to 200 F in the reactoris 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> plus or minus 10 minutes or something like that.
  • No key was submitted.
  • The one critical step is about four hours which is not bounded as required by the NUREG
  • Need to bound the acceptable range in the calculation.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since S

this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.

  • Sat with Rev 2 as submitted in folder RO (AR2) S 3 X U
  • Label on title page 2015 AR2
  • No key was submitted.
  • No safety function in JPM.
  • No KA assigned to the JPM.
  • The task standard does not match critical steps.
  • The brief standard overview has more info in it than the actual task standard for this JPM.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since S

this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA. This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO-only to know fire door operability requirements. 0 Page 1 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Sat with Rev 2 as submitted in folder RO (AR3) S 3 X E
  • Label on title page 2015 AR3
  • No safety function in JPM.
  • The task overview states it is a detailed description of the task and yet it has less than the actual task standard for this JPM.
  • Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?

E

  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Recommended replacing the JPM S
  • Sat with Rev 2 as submitted in folder RO (AR4) S 3 X U
  • Label on title page 2015 AR4
  • No key was submitted.
  • No Safety Function in JPM
  • The task standard does not match critical steps. Is the battery charger needed or not?

If needed then the key would show the correct answer for this JPM.

  • Need error range on the calculation for the JPM S
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Sat with Rev 2 as submitted in folder SRO S 3 X E
  • Label on title page 2015 AS1 (AS1)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Do not provide them every reference that they could possibly need to complete the JPM.

They are expected to find some things on their own.

  • What are the handouts that are to be given? The file only has the section of the UFSAR E while the JPM has a list of several. Not sure what has changed.
  • Sat with Rev 2 as submitted in folder S

SRO S 3 X E

  • Label on title page 2015 AS2 (AS2)
  • The task standard does not match critical step. 0 Page 2 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Task standard should stateThe risk is yellow because of the EDG and the SSW pump..

just like to critical step states.

  • Need the References attached to the JPM S
  • Sat with Rev 2 as submitted in folder SRO S 3 X U
  • Label on title page 2015 AS3 (AS3)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Need a clearly marked key
  • Need to clearly mark handout.
  • I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn E out tail spin. We should consider getting rid of a couple of the non-critical errors that they need to catch.
  • Change Task Standard to read The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN-OP-102 (noun Name) and Identify all of the critical components listed in the JPM
  • Do not have the references attached to the handout.

S

  • Sat with Rev 2 as submitted in folder SRO S 2 E
  • Label on title page 2015 AS4 (AS4)
  • This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
  • It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
  • At least you have the dilution flow issue, which is good. It is probably okay though.

E

  • Revise Task Standard to Read The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM
  • Key still looks like 4 errors S
  • Sat with Rev 2 as submitted in folder SRO S X E
  • Label on title page 2015 AS5 (AS5)
  • Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
  • Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesnt just give the applicant the minimum info to get to the correct EAL call.
  • Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time. 0 Page 3 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 4 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link General Notes:

1. All of the JPMs need to be titled in sequence given on the 301-2 (S1, S2, S3P1, P2)
2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
3. The Procedure that will be handed to the applicant needs to be included in the JPM package
4. Task Standards need to be more detailed for example Applicant will start the SSW A pump in accordance with SOI by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW A pump or closing valve. Within XXX minutes of receiving alarm XXX)

S1 D 2 X X B U

  • Label on title page 2015 S1
  • Need to provide the procedure that will be given to the applicant
  • There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33-F067B).
  • Need to have the alternate path marked on the JPM where it begins
  • The cue to tell the applicant to take action is not allowed it cues the alternate path
  • The cue to tell applicant that fast detent can be used should not be in the standard.
  • Use both noun names and valve numbers in the cue to student and throughout the JPM
  • Task needs to be bounded, how long is the power increase allowed to occur?

Specific power level they should not reach pump runout conditions etc U

  • Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1 S
  • Sat with Rev 2 as submitted in folder S2 (CR2) S 2 B E
  • Label Title page 2015 S2/CR2
  • Need to provide the handout that will be given to the applicant.
  • Need to ensure that the work week schedule will not interfere with the performance of the JPM
  • Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
  • In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there. Needs to match the critical 0 Page 5 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link steps S

  • Sat with Rev 2 as submitted in folder S3 D 2 X N E
  • Label Title page 2015 S3
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Steps 4 and 5 should also be marked as critical
  • Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
  • Step 4 should be recognize a valid group 1 isolation signal occurs
  • It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs S
  • Sat with Rev 2 as submitted in folder S4 D 2 B E
  • Label Title page 2015 S4
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Task Overview section change third bullet to experience a failure to initiate and a failure to inject
  • Step 7. What is the consequence of not achieving the flow rate in the task standard? If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
  • Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student
  • What is the procedure 04-1-E22-1? Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
  • Need to include in the standard what would cause the HPCS start signal written out.
  • Task Standard needs to provide the procedure transition to the alternate path S
  • Sat with Rev 2 as submitted in folder S5 S 2 X B E
  • Label Title page 2015 S5/CR5
  • Need to provide the procedure that will be given to the applicant
  • I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13-P870.
  • If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
  • Step 1 Notes says to give the applicant attachment 6 should be attachment 14 0 Page 6 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 2 should be marked as critical
  • Step 5 should be marked as critical
  • Task standard is not written well. Just venting containment is not good enough.

Either state that and and all critical steps in the JPM completed satisfactorily or State it as Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans.

E

  • Changed location of the JPM and decided to use the V-panel to simulate the fan banks S
  • Sat with Rev 2 as submitted in folder S6 D 2 B U
  • Label Title page 2015 S6
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Do two or nine rods scram initially?
  • It is not clear on what is going on in the task standard. When depress the manual scram switch (step 5.4) the JPM std states that the standard Depress MAN SCRAM Switch and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
  • Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesnt match this either. It states that when the first scram switch is depressed, the nine rods scram
  • There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam
  • Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
  • How long is acceptable to wait to scram?
  • If we keep it then you will need to fix all the issues with the JPM E
  • Number of Critical steps need changed to 3
  • Fix Task Standard to The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05-1-02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily.
  • Step 4 it needs to indicate that 9 rods scram not 4 0 Page 7 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
  • Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM S
  • Sat with Rev 2 as submitted in folder S7 D 2 X B E
  • Label Title page 2015 S7
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • JPM step 1 What happens if the switch is not taken to TEST? Would this be a critical step to be able to accomplish the task?
  • There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path. No credit is given to the applicant for items that are cued
  • How long is it acceptable to take to isolate the leak? Needs included in task standard.
  • Step after step 12 should be step 13 not step 8
  • Task standard needs to be more detailed form a legal perspective E
  • Revise Task Standard to Applicant is to perform a manual start of SSW pump A in accordance with 04 01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily.
  • On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)

S

  • Sat with Rev 2 as submitted in folder S8 D 2 B E
  • Label Title page 2015 S8
  • Need to provide the procedure that will be given to the applicant
  • The task standard should be something like completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM.
  • Step 20 of JPM What happens if they hold it long enough to get a high alarm? Is this a failure criteria?
  • Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed
  • Step after step 25 should be step 26 not step 8
  • Minor Edit to make the JPM flow with little down time on unverifiable actions is to E

have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed. 0 Page 8 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder P1 S B E
  • Label Title page 2015 P1
  • Need to provide the procedure that will be given to the applicant
  • What happens if the applicant tries to reset an overcurrent really? The standard says that they should not do this.
  • Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps
  • In-plant JPM cues need to be correct for in-plant JPMs. You cant use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
  • The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
  • I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week
  • Sat with Rev 2 as submitted in folder S

P2 S B E

  • Label Title page 2015 P2
  • Need to provide the procedure that will be given to the applicant
  • I do not understand how this is an EN JPM, Please explain
  • Task Standard is not up to legal standards as explained in other JPM comments
  • Sat with Rev 2 as submitted in folder S

P3 S N E

  • Label Title Page 2015 P3
  • Need to provide the procedure that will be given to the applicant
  • JPM was altered during validation to more accurately mirror actual method of completion.

S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in 0 Page 9 of 12 OBDI 202 - IOLE Process

reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:

$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

$ The JPM does not contain sufficient cues that are objective (not leading).

$ All critical steps (elements) have not been properly identified.

$ Scope of the task is either too narrow (N) or too broad (B).

$ Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 10 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS Scenario 1. 2. 3. 4. 5. 6. 7. 8. 9.

10. Explanation (See below for instructions)

Set ES TS Crit IC Pred TL L/C Eff U/E/S 1 S No comments prior to validation E Validation Week Comments to incorporate

  • Added applicable tech spec entry to event 3
  • Added applicable tech spec condition to event 4
  • On D-1 corrected the order of events
  • Added bounding criteria to Critical Tasks
  • Changed Critical task 2 due to a change in event 8
  • Fixed simulator Setup information on page 2 of the D-2 from 7.7 to 7.5
  • Fixed turnover sheet to reflect step 7.5 as well
  • Added condition A to Tech Spec call in event 3 on D-2
  • Removed potential tech spec (tracking LCO) from event 3
  • Added condition A to LCO call for event 4
  • Added Critical Task bounding criteria to event 6 on D-2
  • Add direction to booth to bring completed attachments to the control room as would be done in the plant
  • Added critical Task bounding criterial to event 7
  • Made event 7 a NRC trigger S
  • Sat with Rev 2 as submitted in folder 2 S No Comments prior to validation week E Validation week comments to incorporate
  • Tech Specs on D-1 for events 1,2,4and 5 were incomplete
  • In Scenario Activities sections added appropriate Tech Specs and conditions to match D-2
  • Event 2 and 3 needs NRC trigger symbol
  • Event 4 is TR 3.1.5 condition A applicable?
  • Missing TS on event 5
  • Fix bullets on page 6
  • Fix typo on page 12 of D-2 for B RFPT
  • Add noun names on event 7 of D-2
  • Provide details for examiners on event 7 verification of ECCS systems
  • Add noun names for SRVs S
  • Sat with Rev 2 as submitted in folder 3 S No Comments prior to validations week E Validation week comments to incorporate 0 Page 11 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS

  • Page 1 od D-1 add the SBLOCA to event 9 as it is in the D-2
  • Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2
  • D-1 page 2, put TR in front of 3.3.2.1 on event 5
  • Change critical tasks 3 to 2 one of the critical tasks is not EOP based
  • D-2 page 5, list the alarm noun names
  • D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform
  • D-2 page 7, give the noun name for the alarm that is required to be addressed
  • D-2 page 10, add bounding criteria to Critical task
  • D-2 page 12, add the conditions for TRM 3.3.2.1
  • D-2 page 13, list that alarm that will cue applicant with noun name S
  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. ES: ES-301 checklists 4, 5, & 6 satisfied.
2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5. Pred: Scenario sequence and other factors avoid predictability issues.
6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9. Based on the reviewer=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
10. Provide a brief description of problem in the explanation column.
11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 12 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link General comments:

1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
4. Noun and numerical names are required for pumps, valves, and controllers.
5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion.
6. JPM standards should include a statement that all critical steps are completed satisfactorily.
7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM.
8. Your numbering system for the JPMs is not very easy to follow. It should be S-1, S-2, S-3, and in-plants should be P-1, P-2, etc.

RO (AR1) S 2 X U

  • Label on title page 2015 AR1
  • No KA assigned to this JPM you are required to assign a K/A to each JPM
  • Task standard is incorrect (Need to use something like applicant determines that the time to 200 F in the reactoris 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> plus or minus 10 minutes or something like that.
  • No key was submitted.
  • The one critical step is about four hours which is not bounded as required by the NUREG
  • Need to bound the acceptable range in the calculation.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since S

this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.

  • Sat with Rev 2 as submitted in folder RO (AR2) S 3 X U
  • Label on title page 2015 AR2
  • No key was submitted.
  • No safety function in JPM.
  • No KA assigned to the JPM.
  • The task standard does not match critical steps.
  • The brief standard overview has more info in it than the actual task standard for this JPM.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since S

this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA. This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO-only to know fire door operability requirements. 0 Page 1 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Sat with Rev 2 as submitted in folder RO (AR3) S 3 X E
  • Label on title page 2015 AR3
  • No safety function in JPM.
  • The task overview states it is a detailed description of the task and yet it has less than the actual task standard for this JPM.
  • Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?

E

  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Recommended replacing the JPM S
  • Sat with Rev 2 as submitted in folder RO (AR4) S 3 X U
  • Label on title page 2015 AR4
  • No key was submitted.
  • No Safety Function in JPM
  • The task standard does not match critical steps. Is the battery charger needed or not?

If needed then the key would show the correct answer for this JPM.

  • Need error range on the calculation for the JPM S
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Sat with Rev 2 as submitted in folder SRO S 3 X E
  • Label on title page 2015 AS1 (AS1)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Do not provide them every reference that they could possibly need to complete the JPM.

They are expected to find some things on their own.

  • What are the handouts that are to be given? The file only has the section of the UFSAR E while the JPM has a list of several. Not sure what has changed.
  • Sat with Rev 2 as submitted in folder S

SRO S 3 X E

  • Label on title page 2015 AS2 (AS2)
  • The task standard does not match critical step. 0 Page 2 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Task standard should stateThe risk is yellow because of the EDG and the SSW pump..

just like to critical step states.

  • Need the References attached to the JPM S
  • Sat with Rev 2 as submitted in folder SRO S 3 X U
  • Label on title page 2015 AS3 (AS3)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Need a clearly marked key
  • Need to clearly mark handout.
  • I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn E out tail spin. We should consider getting rid of a couple of the non-critical errors that they need to catch.
  • Change Task Standard to read The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN-OP-102 (noun Name) and Identify all of the critical components listed in the JPM
  • Do not have the references attached to the handout.

S

  • Sat with Rev 2 as submitted in folder SRO S 2 E
  • Label on title page 2015 AS4 (AS4)
  • This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
  • It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
  • At least you have the dilution flow issue, which is good. It is probably okay though.

E

  • Revise Task Standard to Read The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM
  • Key still looks like 4 errors S
  • Sat with Rev 2 as submitted in folder SRO S X E
  • Label on title page 2015 AS5 (AS5)
  • Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
  • Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesnt just give the applicant the minimum info to get to the correct EAL call.
  • Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time. 0 Page 3 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 4 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link General Notes:

1. All of the JPMs need to be titled in sequence given on the 301-2 (S1, S2, S3P1, P2)
2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
3. The Procedure that will be handed to the applicant needs to be included in the JPM package
4. Task Standards need to be more detailed for example Applicant will start the SSW A pump in accordance with SOI by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW A pump or closing valve. Within XXX minutes of receiving alarm XXX)

S1 D 2 X X B U

  • Label on title page 2015 S1
  • Need to provide the procedure that will be given to the applicant
  • There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33-F067B).
  • Need to have the alternate path marked on the JPM where it begins
  • The cue to tell the applicant to take action is not allowed it cues the alternate path
  • The cue to tell applicant that fast detent can be used should not be in the standard.
  • Use both noun names and valve numbers in the cue to student and throughout the JPM
  • Task needs to be bounded, how long is the power increase allowed to occur?

Specific power level they should not reach pump runout conditions etc U

  • Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1 S
  • Sat with Rev 2 as submitted in folder S2 (CR2) S 2 B E
  • Label Title page 2015 S2/CR2
  • Need to provide the handout that will be given to the applicant.
  • Need to ensure that the work week schedule will not interfere with the performance of the JPM
  • Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
  • In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there. Needs to match the critical 0 Page 5 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link steps S

  • Sat with Rev 2 as submitted in folder S3 D 2 X N E
  • Label Title page 2015 S3
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Steps 4 and 5 should also be marked as critical
  • Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
  • Step 4 should be recognize a valid group 1 isolation signal occurs
  • It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs S
  • Sat with Rev 2 as submitted in folder S4 D 2 B E
  • Label Title page 2015 S4
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Task Overview section change third bullet to experience a failure to initiate and a failure to inject
  • Step 7. What is the consequence of not achieving the flow rate in the task standard? If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
  • Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student
  • What is the procedure 04-1-E22-1? Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
  • Need to include in the standard what would cause the HPCS start signal written out.
  • Task Standard needs to provide the procedure transition to the alternate path S
  • Sat with Rev 2 as submitted in folder S5 S 2 X B E
  • Label Title page 2015 S5/CR5
  • Need to provide the procedure that will be given to the applicant
  • I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13-P870.
  • If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
  • Step 1 Notes says to give the applicant attachment 6 should be attachment 14 0 Page 6 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 2 should be marked as critical
  • Step 5 should be marked as critical
  • Task standard is not written well. Just venting containment is not good enough.

Either state that and and all critical steps in the JPM completed satisfactorily or State it as Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans.

E

  • Changed location of the JPM and decided to use the V-panel to simulate the fan banks S
  • Sat with Rev 2 as submitted in folder S6 D 2 B U
  • Label Title page 2015 S6
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Do two or nine rods scram initially?
  • It is not clear on what is going on in the task standard. When depress the manual scram switch (step 5.4) the JPM std states that the standard Depress MAN SCRAM Switch and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
  • Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesnt match this either. It states that when the first scram switch is depressed, the nine rods scram
  • There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam
  • Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
  • How long is acceptable to wait to scram?
  • If we keep it then you will need to fix all the issues with the JPM E
  • Number of Critical steps need changed to 3
  • Fix Task Standard to The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05-1-02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily.
  • Step 4 it needs to indicate that 9 rods scram not 4 0 Page 7 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
  • Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM S
  • Sat with Rev 2 as submitted in folder S7 D 2 X B E
  • Label Title page 2015 S7
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • JPM step 1 What happens if the switch is not taken to TEST? Would this be a critical step to be able to accomplish the task?
  • There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path. No credit is given to the applicant for items that are cued
  • How long is it acceptable to take to isolate the leak? Needs included in task standard.
  • Step after step 12 should be step 13 not step 8
  • Task standard needs to be more detailed form a legal perspective E
  • Revise Task Standard to Applicant is to perform a manual start of SSW pump A in accordance with 04 01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily.
  • On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)

S

  • Sat with Rev 2 as submitted in folder S8 D 2 B E
  • Label Title page 2015 S8
  • Need to provide the procedure that will be given to the applicant
  • The task standard should be something like completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM.
  • Step 20 of JPM What happens if they hold it long enough to get a high alarm? Is this a failure criteria?
  • Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed
  • Step after step 25 should be step 26 not step 8
  • Minor Edit to make the JPM flow with little down time on unverifiable actions is to E

have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed. 0 Page 8 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder P1 S B E
  • Label Title page 2015 P1
  • Need to provide the procedure that will be given to the applicant
  • What happens if the applicant tries to reset an overcurrent really? The standard says that they should not do this.
  • Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps
  • In-plant JPM cues need to be correct for in-plant JPMs. You cant use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
  • The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
  • I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week
  • Sat with Rev 2 as submitted in folder S

P2 S B E

  • Label Title page 2015 P2
  • Need to provide the procedure that will be given to the applicant
  • I do not understand how this is an EN JPM, Please explain
  • Task Standard is not up to legal standards as explained in other JPM comments
  • Sat with Rev 2 as submitted in folder S

P3 S N E

  • Label Title Page 2015 P3
  • Need to provide the procedure that will be given to the applicant
  • JPM was altered during validation to more accurately mirror actual method of completion.

S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in 0 Page 9 of 12 OBDI 202 - IOLE Process

reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:

$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

$ The JPM does not contain sufficient cues that are objective (not leading).

$ All critical steps (elements) have not been properly identified.

$ Scope of the task is either too narrow (N) or too broad (B).

$ Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 10 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS Scenario 1. 2. 3. 4. 5. 6. 7. 8. 9.

10. Explanation (See below for instructions)

Set ES TS Crit IC Pred TL L/C Eff U/E/S 1 S No comments prior to validation E Validation Week Comments to incorporate

  • Added applicable tech spec entry to event 3
  • Added applicable tech spec condition to event 4
  • On D-1 corrected the order of events
  • Added bounding criteria to Critical Tasks
  • Changed Critical task 2 due to a change in event 8
  • Fixed simulator Setup information on page 2 of the D-2 from 7.7 to 7.5
  • Fixed turnover sheet to reflect step 7.5 as well
  • Added condition A to Tech Spec call in event 3 on D-2
  • Removed potential tech spec (tracking LCO) from event 3
  • Added condition A to LCO call for event 4
  • Added Critical Task bounding criteria to event 6 on D-2
  • Add direction to booth to bring completed attachments to the control room as would be done in the plant
  • Added critical Task bounding criterial to event 7
  • Made event 7 a NRC trigger S
  • Sat with Rev 2 as submitted in folder 2 S No Comments prior to validation week E Validation week comments to incorporate
  • Tech Specs on D-1 for events 1,2,4and 5 were incomplete
  • In Scenario Activities sections added appropriate Tech Specs and conditions to match D-2
  • Event 2 and 3 needs NRC trigger symbol
  • Event 4 is TR 3.1.5 condition A applicable?
  • Missing TS on event 5
  • Fix bullets on page 6
  • Fix typo on page 12 of D-2 for B RFPT
  • Add noun names on event 7 of D-2
  • Provide details for examiners on event 7 verification of ECCS systems
  • Add noun names for SRVs S
  • Sat with Rev 2 as submitted in folder 3 S No Comments prior to validations week E Validation week comments to incorporate 0 Page 11 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS

  • Page 1 od D-1 add the SBLOCA to event 9 as it is in the D-2
  • Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2
  • D-1 page 2, put TR in front of 3.3.2.1 on event 5
  • Change critical tasks 3 to 2 one of the critical tasks is not EOP based
  • D-2 page 5, list the alarm noun names
  • D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform
  • D-2 page 7, give the noun name for the alarm that is required to be addressed
  • D-2 page 10, add bounding criteria to Critical task
  • D-2 page 12, add the conditions for TRM 3.3.2.1
  • D-2 page 13, list that alarm that will cue applicant with noun name S
  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. ES: ES-301 checklists 4, 5, & 6 satisfied.
2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5. Pred: Scenario sequence and other factors avoid predictability issues.
6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9. Based on the reviewer=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
10. Provide a brief description of problem in the explanation column.
11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 12 of 12 OBDI 202 - IOLE Process