ML16028A021: Difference between revisions

From kanterella
Jump to navigation Jump to search
(Created page by program invented by StriderTol)
(Created page by program invented by StriderTol)
 
Line 16: Line 16:


=Text=
=Text=
{{#Wiki_filter:Attachment 10 Page 1 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
{{#Wiki_filter:GG - 2015 - 12                                               DRAFT OPERATING TEST COMMENTS                                                                           ADMIN JPMS
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 4. Job Content
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia            General comments:
: 1. 2.           3. Attributes                                                                                     6.
Errors 5.
JPM#    Dyn  LOD                                                                                                            Explanation U/E/S (D/S) (1-5)  IC  Cues Critical  Scope  Over- Job-    Minutia                                              (See below for instructions)
Focus       Steps     (N/B) lap Link General comments:
: 1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
: 1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
: 2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
: 2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
: 3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
: 3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
: 4. Noun and numerical names are required for pumps, valves, and controllers.
: 4. Noun and numerical names are required for pumps, valves, and controllers.
: 5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion. 6. JPM standards should include a statement that "all critical steps are completed satisfactorily."
: 5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion.
: 7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM. 8. Your numbering system for the JPMs is not very easy to follow. It should be S
: 6. JPM standards should include a statement that all critical steps are completed satisfactorily.
-1, S-2, S-3, and in-plants should be P
: 7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM.
-1, P-2, etc. RO (A R 1) S 2   X     U Label on title page 2015 AR1 No KA assigned to this JPM you are required to assign a K/A to each JPM Task standard is incorrect (Need to use something like "applicant determines that the time to 200 F in the reactor--is 4 hours plus or minus 10 minutes" or something like that.
: 8. Your numbering system for the JPMs is not very easy to follow. It should be S-1, S-2, S-3, and in-plants should be P-1, P-2, etc.
No key was submitted.
RO (AR1)   S     2                 X                                     U
The one critical step is "about four hours" which is not bounded as required by the NUREG Need to bound the acceptable range in the calculation
* Label on title page 2015 AR1
. With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.
* No KA assigned to this JPM you are required to assign a K/A to each JPM
Sat with Rev 2 as submitted in folder RO (A R 2) S 3   X     U
* Task standard is incorrect (Need to use something like applicant determines that the time to 200 F in the reactoris 4 hours plus or minus 10 minutes or something like that.
 
* No key was submitted.
Label on title page 2015 AR2 No key was submitted.
* The one critical step is about four hours which is not bounded as required by the NUREG
No safety function in JPM.
* Need to bound the acceptable range in the calculation.
No KA assigned to the JPM.
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since S
The task standard does not match critical steps.
this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.
The brief standard overview has more info in it than the actual task standard for this JPM.
* Sat with Rev 2 as submitted in folder RO (AR2)   S     3                 X                                     U
With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA.
* Label on title page 2015 AR2
This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO
* No key was submitted.
-only to know fire door operability requirements.
* No safety function in JPM.
 
* No KA assigned to the JPM.
Attachment 10 Page 2 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
* The task standard does not match critical steps.
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
* The brief standard overview has more info in it than the actual task standard for this JPM.
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  Sat with Rev 2 as submitted in folder RO (A R3)  S 3  X    E E
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since S
S  Label on title page 2015 AR3 No safety function in JPM.
this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA. This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO-only to know fire door operability requirements. 0                                                            Page 1 of 12                                                             OBDI 202 - IOLE Process
The task overview states it is a detailed description of the task and yet it has less than th e actual task standard for this JPM.
Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?
 
With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
Recommended replacing the JPM Sat with Rev 2 as submitted in folder RO (A R 4) S 3  X    U S  Label on title page 2015 AR4  No key was submitted.
No Safety Function in JPM The task standard does not match critical steps. Is the battery charger needed or not?


GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                                                                        ADMIN JPMS
: 4. Job Content
: 1. 2.            3. Attributes                                                                                  6.
Errors 5.
JPM#    Dyn  LOD                                                                                                        Explanation U/E/S (D/S) (1-5)  IC  Cues Critical  Scope  Over- Job-    Minutia                                          (See below for instructions)
Focus        Steps    (N/B)  lap  Link
* Sat with Rev 2 as submitted in folder RO (AR3)    S    3                  X                                    E
* Label on title page 2015 AR3
* No safety function in JPM.
* The task overview states it is a detailed description of the task and yet it has less than the actual task standard for this JPM.
* Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?
E
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
* Recommended replacing the JPM S
* Sat with Rev 2 as submitted in folder RO (AR4)    S    3                  X                                    U
* Label on title page 2015 AR4
* No key was submitted.
* No Safety Function in JPM
* The task standard does not match critical steps. Is the battery charger needed or not?
If needed then the key would show the correct answer for this JPM.
If needed then the key would show the correct answer for this JPM.
Need error range on the calculation for the J PM  With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
* Need error range on the calculation for the JPM S
Sat with Rev 2 as submitted in folder SRO (A S1) S 3   X   E
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
 
* Sat with Rev 2 as submitted in folder SRO     S     3                           X                           E
E  S  Label on title page 2015 AS1 This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
* Label on title page 2015 AS1 (AS1)
Do not provide them every reference that they could possibly need to complete the JPM.
* This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
* Do not provide them every reference that they could possibly need to complete the JPM.
They are expected to find some things on their own.
They are expected to find some things on their own.
What are the handouts that are to be given? The file only has the section of the UFSAR while the JPM has a list of several. Not sure what has changed.
* What are the handouts that are to be given? The file only has the section of the UFSAR E        while the JPM has a list of several. Not sure what has changed.
Sat with Rev 2 as submitted in folder SRO (AS2) S 3   X     E   Label on title page 2015 AS2 The task standard does not match critical step.  
* Sat with Rev 2 as submitted in folder S
 
SRO     S     3                 X                                     E
Attachment 10 Page 3 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
* Label on title page 2015 AS2 (AS2)
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
* The task standard does not match critical step. 0                                                            Page 2 of 12                                                           OBDI 202 - IOLE Process
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia    S  Task st andard should state-"The risk is yellow because of the EDG and the SSW pump-.." just like to critical step states.
Need the References attached to the JPM Sat with Rev 2 as submitted in folder SRO (AS3) S 3  X    U E        S  Label on title page 2015 AS3 This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
Need a clearly marked key Need to clearly mark handout.
I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn out tail spin. We should consider getting rid of a couple of the non
-critical errors that they need to catch.
Change Task Standard to read "The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN
-OP-102 (noun Name) and Identify all of the critical components listed in the JPM" Do not have the references attached to the handout.
Sat with Rev 2 as submitted in folder SRO (AS4) S 2        E   
 
 
S  Label on title page 2015 AS4 This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
At least you have the dilution flow issue, which is good. It is probably okay though.
Revise Task Standard to Read
 
"The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM" Key still looks like 4 errors


Sat with Rev 2 as submitted in folder SRO (AS5) S   X     E Label on title page 2015 AS5 Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                                                                          ADMIN JPMS
Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesn't just give the applicant the minimum info to get to the correct EAL call.
: 4. Job Content
Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time.
: 1. 2.            3. Attributes                                                                                    6.
Errors 5.
JPM#    Dyn  LOD                                                                                                        Explanation U/E/S (D/S) (1-5)  IC  Cues Critical  Scope  Over- Job-    Minutia                                            (See below for instructions)
Focus        Steps    (N/B)  lap  Link
* Task standard should stateThe risk is yellow because of the EDG and the SSW pump..
just like to critical step states.
* Need the References attached to the JPM S
* Sat with Rev 2 as submitted in folder SRO      S    3                  X                                    U
* Label on title page 2015 AS3 (AS3)
* This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
* Need a clearly marked key
* Need to clearly mark handout.
* I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn E        out tail spin. We should consider getting rid of a couple of the non-critical errors that they need to catch.
* Change Task Standard to read The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN-OP-102 (noun Name) and Identify all of the critical components listed in the JPM
* Do not have the references attached to the handout.
S
* Sat with Rev 2 as submitted in folder SRO     S    2                                                        E
* Label on title page 2015 AS4 (AS4)
* This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
* It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
* At least you have the dilution flow issue, which is good. It is probably okay though.
E
* Revise Task Standard to Read The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM
* Key still looks like 4 errors S
* Sat with Rev 2 as submitted in folder SRO      S                X                                             E
* Label on title page 2015 AS5 (AS5)
* Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
* Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesnt just give the applicant the minimum info to get to the correct EAL call.
* Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time. 0                                                            Page 3 of 12                                                            OBDI 202 - IOLE Process


Attachment 10 Page 4 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
GG - 2015 - 12                                                               DRAFT OPERATING TEST COMMENTS                                                             ADMIN JPMS
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 4. Job Content
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  S   Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG
: 1.       2.               3. Attributes                                                                               6.
-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
Errors 5.
JPM#      Dyn      LOD                                                                                                        Explanation U/E/S (D/S)    (1-5)    IC    Cues Critical  Scope    Over-  Job-    Minutia                                    (See below for instructions)
Focus         Steps     (N/B)   lap   Link S
* Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 2. Determine level of difficulty (LOD) using established 1
: 2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
: 3. Check the appropriate box when an attribute weakness is identified:
: 3. Check the appropriate box when an attribute weakness is identified:
The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
* The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
The JPM does not contain sufficient cues that are objective (not leading).
* The JPM does not contain sufficient cues that are objective (not leading).
All critical steps (elements) have not been properly identified.
* All critical steps (elements) have not been properly identified.
Scope of the task is either too narrow (N) or too broad (B).
* Scope of the task is either too narrow (N) or too broad (B).
Excessive overlap with other part of operating test or written examination.
* Excessive overlap with other part of operating test or written examination.
: 4. Check the appropriate box when a job content error is identified:
: 4. Check the appropriate box when a job content error is identified:
Topics not linked to job content (e.g., disguised task, not required in real job).
* Topics not linked to job content (e.g., disguised task, not required in real job).
Task is trivial and without safety significance.
* Task is trivial and without safety significance.
: 5. Based on the reviewer
: 5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 6. Provide a brief description of any U or E rating in the explanation column.
: 6. Provide a brief description of any U or E rating in the explanation column.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0                                                                            Page 4 of 12                                                    OBDI 202 - IOLE Process


Attachment 10 Page 5 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
GG - 2015 - 12                                               DRAFT OPERATING TEST COMMENTS                               CONTROL ROOM/IN-PLANT SYSTEMS JPMS
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 1. 2.           3. Attributes                                                                                 6.
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia            General Notes:
Errors       5.
: 1. All of the JPMs need to be titled in sequence given on the 301
JPM#    Dyn  LOD                                                                                                      Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                            (See below for instructions)
-2 (S1, S2, S3-P1, P2-) 2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
Focus       Steps     (N/B) lap Link General Notes:
: 1. All of the JPMs need to be titled in sequence given on the 301-2 (S1, S2, S3P1, P2)
: 2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
: 3. The Procedure that will be handed to the applicant needs to be included in the JPM package
: 3. The Procedure that will be handed to the applicant needs to be included in the JPM package
: 4. Task Standards need to be more detailed for example "Applicant will start the SSW "A" pump in accordance with SOI- by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW "A" pump or closing valve-. Within XXX minutes of receiving alarm XXX)"
: 4. Task Standards need to be more detailed for example Applicant will start the SSW A pump in accordance with SOI by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW A pump or closing valve. Within XXX minutes of receiving alarm XXX)
S 1 D 2 X X B   U   
S1      D     2         X       X         B                           U
 
* Label on title page 2015 S1
U
* Need to provide the procedure that will be given to the applicant
 
* There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33-F067B).
Label on title page 2015 S1 Need to provide the procedure that will be given to the applicant There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33
* Need to have the alternate path marked on the JPM where it begins
-F067B). Need to have the alternate path marked on the JPM where it begins The cue to tell the applicant to take action is not allowed it cues the alternate path The cue to tell applicant that fast detent can be used should not be in the standard. Use both noun names and valve numbers in the cue to student and throughout the JPM Task needs to be bounded, how long is the power increase allowed to occur? Specific power level they should not reach pump runout conditions et c-  Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1
* The cue to tell the applicant to take action is not allowed it cues the alternate path
 
* The cue to tell applicant that fast detent can be used should not be in the standard.
Sat with Rev 2 as submitted in folder S 2 (CR2) S 2   B   E
* Use both noun names and valve numbers in the cue to student and throughout the JPM
 
* Task needs to be bounded, how long is the power increase allowed to occur?
Label Title page 2015 S2/CR2 Need to provide the handout that will be given to the applicant.
Specific power level they should not reach pump runout conditions etc U
Need to ensure that the work week schedule will not interfere with the performance of the JPM Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
* Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1 S
In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there.
* Sat with Rev 2 as submitted in folder S2 (CR2)   S     2                           B                           E
Needs to match the critical Attachment 10 Page 6 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
* Label Title page 2015 S2/CR2
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
* Need to provide the handout that will be given to the applicant.
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
* Need to ensure that the work week schedule will not interfere with the performance of the JPM
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  S steps  Sat with Rev 2 as submitted in folder S 3 D 2  X N    E   
* Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
 
* In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there. Needs to match the critical 0                                                            Page 5 of 12                                                         OBDI 202 - IOLE Process
S  Label Title page 2015 S3 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM Steps 4 and 5 should also be marked as critical Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
Step 4 should be recognize a valid group 1 isolation signal occurs It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs Sat with Rev 2 as submitted in folder S 4 D 2    B    E 
 
S  Label Title page 2015 S4 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM Task Overview section change third bullet to "experience a failure to initiate and a failure to inject" Step 7. What is the consequence of not achieving the flow rate in the task standard?  If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student What is the procedure 04 E22-1?  Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
Need to include in the standard what would cause the HPCS start signal written out. Task Standard needs to provide the procedure transition to the alternate path Sat with Rev 2 as submitted in folder S 5 S 2  X B    E Label Title page 2015 S5/CR5 Need to provide the procedure that will be given to the applicant I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13
-P870. If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
Step 1 Notes says to give the applicant attachment 6 should be attachment 14
 
Attachment 10 Page 7 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia E  S  Step 2 should be marked as critical Step 5 should be marked as critical Task standard is not written well. Just venting containment is not good enough. Either state that and "and all critical steps in the JPM completed satisfactorily" or State it as "Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans."    Changed location of the JPM and decided to use the V
-panel to simulate the fan banks  Sat with Rev 2 as submitted in folder S 6 D 2    B    U   
 
 
Label Title page 2015 S6 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM Do two or nine rods scram initially?  It is not clear on what is going on in the task standard.
When depress the manual scram switch (step 5.4) the JPM std states that the standard "Depress MAN SCRAM Switch" and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesn't match this either. It states that when the first scram switch is depressed, the nine rods scram There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
How long is acceptable to wait to scram?
If we keep it then you will need to fix all the issues with the JPM Number of Critical steps need changed to 3 Fix Task Standard to
 
"The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06
-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05 02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily."
Step 4 it needs to indicate that 9 rods scram not 4
 
Attachment 10 Page 8 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia      S  Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM Sat with Rev 2 as submitted in folder S 7 D 2  X  B    E 
 
 
S  Label Title page 2015 S7 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM JPM step 1 What happens if the switch is not taken to TEST?  Would this be a critical step to be able to accomplish the task?
There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path.
No credit is given to th e applicant for items that are cue d  How long is it acceptable to take to isolate the leak?  Needs included in task standard. Step after step 12 should be step 13 not step 8 Task standard needs to be more detailed form a legal perspective Revise Task Standard to "Applicant is to perform a manual start of SSW pump A in accordance with 04 01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily."
On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)
Sat with Rev 2 as submitted in folder S 8 D 2    B    E 
 
E    Label Title page 2015 S8 Need to provide the procedure that will be given to the applicant The task standard should be something like "completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM."
Step 20 of JPM What happens if they hold it long enough to get a high alarm?  Is this a failure criteria?
Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed Step after step 25 should be step 26 not step 8 Minor Edit to make the JPM flow with little down time on unverifiable actions is to


have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed.
GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                            CONTROL ROOM/IN-PLANT SYSTEMS JPMS
: 4. Job Content
: 1. 2.            3. Attributes                                                                              6.
Errors      5.
JPM#    Dyn  LOD                                                                                                    Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                          (See below for instructions)
Focus        Steps    (N/B)  lap  Link steps S
* Sat with Rev 2 as submitted in folder S3      D    2                  X        N                          E
* Label Title page 2015 S3
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* Steps 4 and 5 should also be marked as critical
* Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
* Step 4 should be recognize a valid group 1 isolation signal occurs
* It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs S
* Sat with Rev 2 as submitted in folder S4      D    2                            B                          E
* Label Title page 2015 S4
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* Task Overview section change third bullet to experience a failure to initiate and a failure to inject
* Step 7. What is the consequence of not achieving the flow rate in the task standard? If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
* Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student
* What is the procedure 04-1-E22-1? Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
* Need to include in the standard what would cause the HPCS start signal written out.
* Task Standard needs to provide the procedure transition to the alternate path S
* Sat with Rev 2 as submitted in folder S5      S    2                 X        B                          E
* Label Title page 2015 S5/CR5
* Need to provide the procedure that will be given to the applicant
* I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13-P870.
* If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
* Step 1 Notes says to give the applicant attachment 6 should be attachment 14 0                                                            Page 6 of 12                                                        OBDI 202 - IOLE Process


Attachment 10 Page 9 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
GG - 2015 - 12                                               DRAFT OPERATING TEST COMMENTS                             CONTROL ROOM/IN-PLANT SYSTEMS JPMS
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 1. 2.           3. Attributes                                                                               6.
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  S   Sat with Rev 2 as submitted in folder P1 S     B   E      
Errors       5.
JPM#    Dyn  LOD                                                                                                    Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                          (See below for instructions)
Focus       Steps     (N/B) lap Link
* Step 2 should be marked as critical
* Step 5 should be marked as critical
* Task standard is not written well. Just venting containment is not good enough.
Either state that and and all critical steps in the JPM completed satisfactorily or State it as Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans.
E
* Changed location of the JPM and decided to use the V-panel to simulate the fan banks S
* Sat with Rev 2 as submitted in folder S6      D     2                            B                           U
* Label Title page 2015 S6
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* Do two or nine rods scram initially?
* It is not clear on what is going on in the task standard. When depress the manual scram switch (step 5.4) the JPM std states that the standard Depress MAN SCRAM Switch and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
* Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesnt match this either. It states that when the first scram switch is depressed, the nine rods scram
* There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam
* Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
* How long is acceptable to wait to scram?
* If we keep it then you will need to fix all the issues with the JPM E
* Number of Critical steps need changed to 3
* Fix Task Standard to The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05-1-02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily.
* Step 4 it needs to indicate that 9 rods scram not 4 0                                                            Page 7 of 12                                                        OBDI 202 - IOLE Process


S  Label Title page 2015 P1 Need to provide the procedure that will be given to the applicant What happens if the applicant tries to reset a n overcurrent really? The standard say s that they should not do this.
GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                              CONTROL ROOM/IN-PLANT SYSTEMS JPMS
Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps In-plant JPM cues need to be correct for in
: 4. Job Content
-plant JPMs. You can't use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
: 1. 2.            3. Attributes                                                                              6.
The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
Errors      5.
I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week Sat with Rev 2 as submitted in folder P2 S    B    E 
JPM#    Dyn  LOD                                                                                                    Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                          (See below for instructions)
Focus        Steps    (N/B)  lap Link
* Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
* Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM S
* Sat with Rev 2 as submitted in folder S7      D    2          X                B                          E
* Label Title page 2015 S7
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* JPM step 1 What happens if the switch is not taken to TEST? Would this be a critical step to be able to accomplish the task?
* There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path. No credit is given to the applicant for items that are cued
* How long is it acceptable to take to isolate the leak? Needs included in task standard.
* Step after step 12 should be step 13 not step 8
* Task standard needs to be more detailed form a legal perspective E
* Revise Task Standard to Applicant is to perform a manual start of SSW pump A in accordance with 04                                                                                      01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily.
* On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)
S
* Sat with Rev 2 as submitted in folder S8      D    2                            B                          E
* Label Title page 2015 S8
* Need to provide the procedure that will be given to the applicant
* The task standard should be something like completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM.
* Step 20 of JPM What happens if they hold it long enough to get a high alarm? Is this a failure criteria?
* Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed
* Step after step 25 should be step 26 not step 8
* Minor Edit to make the JPM flow with little down time on unverifiable actions is to E
have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed. 0                                                            Page 8 of 12                                                        OBDI 202 - IOLE Process


S  Label Title page 2015 P2 Need to provide the procedure that will be given to the applicant I do not understand how this is an EN JPM, Please explain Task Standard is not up to legal standards as explained in other JPM comments Sat with Rev 2 as submitted in folder P3 S     N   E
GG - 2015 - 12                                                      DRAFT OPERATING TEST COMMENTS                            CONTROL ROOM/IN-PLANT SYSTEMS JPMS
: 4. Job Content
: 1. 2.              3. Attributes                                                                              6.
Errors      5.
JPM#        Dyn  LOD                                                                                                      Explanation IC    Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                            (See below for instructions)
Focus          Steps    (N/B)  lap  Link S
* Sat with Rev 2 as submitted in folder P1          S                                      B                          E
* Label Title page 2015 P1
* Need to provide the procedure that will be given to the applicant
* What happens if the applicant tries to reset an overcurrent really? The standard says that they should not do this.
* Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps
* In-plant JPM cues need to be correct for in-plant JPMs. You cant use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
* The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
* I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week
* Sat with Rev 2 as submitted in folder S
P2          S                                      B                          E
* Label Title page 2015 P2
* Need to provide the procedure that will be given to the applicant
* I do not understand how this is an EN JPM, Please explain
* Task Standard is not up to legal standards as explained in other JPM comments
* Sat with Rev 2 as submitted in folder S
P3         S                                     N                           E
* Label Title Page 2015 P3
* Need to provide the procedure that will be given to the applicant
* JPM was altered during validation to more accurately mirror actual method of completion.
S
* Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in 0                                                                    Page 9 of 12                                                      OBDI 202 - IOLE Process


S  Label Title Page 2015 P3 Need to provide the procedure that will be given to the applicant JPM was altered during validation to more accurately mirror actual method of completion.
reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG
-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in Attachment 10 Page 10 of 12 OBDI 202 - I OLE Process reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 2. Determine level of difficulty (LOD) using established 1
: 2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested. 3. Check the appropriate box when an attribute weakness is identified:
: 3. Check the appropriate box when an attribute weakness is identified:
$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
      $ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
$ The JPM does not contain sufficient cues that are objective (not leading). $ All critical steps (elements) have not been properly identified.
      $ The JPM does not contain sufficient cues that are objective (not leading).
$ Scope of the task is either too narrow (N) or too broad (B).
      $ All critical steps (elements) have not been properly identified.
$ Excessive overlap with other part of operating test or written examination.
      $ Scope of the task is either too narrow (N) or too broad (B).
      $ Excessive overlap with other part of operating test or written examination.
: 4. Check the appropriate box when a job content error is identified:
: 4. Check the appropriate box when a job content error is identified:
Topics not linked to job content (e.g., disguised task, not required in real job).
* Topics not linked to job content (e.g., disguised task, not required in real job).
Task is trivial and without safety significance.
* Task is trivial and without safety significance.
: 5. Based on the reviewer
: 5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 6. Provide a brief description of any U or E rating in the explanation column.
: 6. Provide a brief description of any U or E rating in the explanation column.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0                                                                      Page 10 of 12                                                   OBDI 202 - IOLE Process
 
Attachment 10 Page 11 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIO S Scenario Set 1. ES 2. TS 3. Crit 4. IC 5. Pred 6. TL 7. L/C 8. Eff 9. U/E/S 10. Explanation (See below for instructions) 1        S  E 
 
S No comments prior to validation
 
Validation Week Comments to incorporate Replaced Event 8 with a feedwater pump trip Added applicable tech spec entry to event 3 Added applicable tech spec condition to even t 4  On D-1 corrected the order of events Added bounding criteria to Critical Task's Changed Critical task 2 due to a change in event 8 Fixed simulator Setup information on page 2 of the D
-2 from 7.7 to 7.5 Fixed turnover sheet to reflect step 7.5 as well Added condition A to Tech Spec call in event 3 on D
-2  Removed potential tech spec (tracking LCO) from event 3 Added condition A to LCO call for event 4 Added Critical Task bounding criteria to event 6 on D
-2  Add direction to booth to bring completed attachments to the control room as would be done in the plant Added critical Task bounding criterial to event 7 Made event 7 a NRC trigger Sat with Rev 2 as submitted in folder 2        S  E 
 
S No Comments prior to validation week
 
Validation week comments to incorporate Tech Specs on D
-1 for events 1,2, 4and 5 were incomplete In Scenario Activities sections added appropriate Tech Spec's and conditions to match D
-2  Event 2 and 3 needs NRC trigger symbol Event 4 is TR 3.1.5 condition A applicable?
Missing TS on event 5 Fix bullets on page 6 Fix typo on page 12 of D
-2 for B RFPT Add noun names on event 7 of D
-2  Provide details for examiners on event 7 verification of ECCS systems Add noun names for SRV's Sat with Rev 2 as submitted in folder 3        S  E No Comments prior to validations week Validation week comments to incorporate


Attachment 10 Page 12 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIO S      
GG - 2015 - 12                         DRAFT OPERATING TEST COMMENTS                                                         SCENARIOS Scenario    1. 2. 3. 4. 5. 6. 7. 8. 9.
: 10. Explanation (See below for instructions)
Set      ES TS Crit IC Pred TL L/C    Eff  U/E/S 1                                            S  No comments prior to validation E  Validation Week Comments to incorporate
* Replaced Event 8 with a feedwater pump trip
* Added applicable tech spec entry to event 3
* Added applicable tech spec condition to event 4
* On D-1 corrected the order of events
* Added bounding criteria to Critical Tasks
* Changed Critical task 2 due to a change in event 8
* Fixed simulator Setup information on page 2 of the D-2 from 7.7 to 7.5
* Fixed turnover sheet to reflect step 7.5 as well
* Added condition A to Tech Spec call in event 3 on D-2
* Removed potential tech spec (tracking LCO) from event 3
* Added condition A to LCO call for event 4
* Added Critical Task bounding criteria to event 6 on D-2
* Add direction to booth to bring completed attachments to the control room as would be done in the plant
* Added critical Task bounding criterial to event 7
* Made event 7 a NRC trigger S
* Sat with Rev 2 as submitted in folder 2                                            S  No Comments prior to validation week E  Validation week comments to incorporate
* Tech Specs on D-1 for events 1,2,4and 5 were incomplete
* In Scenario Activities sections added appropriate Tech Specs and conditions to match D-2
* Event 2 and 3 needs NRC trigger symbol
* Event 4 is TR 3.1.5 condition A applicable?
* Missing TS on event 5
* Fix bullets on page 6
* Fix typo on page 12 of D-2 for B RFPT
* Add noun names on event 7 of D-2
* Provide details for examiners on event 7 verification of ECCS systems
* Add noun names for SRVs S
* Sat with Rev 2 as submitted in folder 3                                            S  No Comments prior to validations week E  Validation week comments to incorporate 0                                  Page 11 of 12                                                    OBDI 202 - IOLE Process


Page 1 od D
GG - 2015 - 12                                                        DRAFT OPERATING TEST COMMENTS                                                                SCENARIOS
-1 add the SBLOCA to event 9 as it is in the D
* Page 1 od D-1 add the SBLOCA to event 9 as it is in the D-2
-2 Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2 D-1 page 2, put TR in front of 3.3.2.1 on event 5 Change critical tasks 3 to 2 one of the critical tasks is not EOP based D-1 page 3, add Tech spec 3.7.2 condition B to MCC 17B01 Trip D-2 page 5, list the alarm noun names D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform D-2 page 7, give the noun name for the alarm that is required to be addressed D-2 page 10, add bounding criteria to Critical task D-2 page 12, add the conditions for TRM 3.3.2.1 D-2 page 13, list that alarm that will cue applicant with noun name Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG
* Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2
-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
* D-1 page 2, put TR in front of 3.3.2.1 on event 5
* Change critical tasks 3 to 2 one of the critical tasks is not EOP based
* D-1 page 3, add Tech spec 3.7.2 condition B to MCC 17B01 Trip
* D-2 page 5, list the alarm noun names
* D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform
* D-2 page 7, give the noun name for the alarm that is required to be addressed
* D-2 page 10, add bounding criteria to Critical task
* D-2 page 12, add the conditions for TRM 3.3.2.1
* D-2 page 13, list that alarm that will cue applicant with noun name S
* Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
: 1. ES: ES-301 checklists 4, 5, & 6 satisfied.
: 1. ES: ES-301 checklists 4, 5, & 6 satisfied.
: 2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
: 2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
: 3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES
: 3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
-D-2. 4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
: 4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
: 5. Pred: Scenario sequence and other factors avoid predictability issues.
: 5. Pred: Scenario sequence and other factors avoid predictability issues.
: 6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
: 6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
: 7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
: 7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
: 8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions. 9. Based on the reviewer
: 8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
: 9. Based on the reviewer=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
: 10. Provide a brief description of problem in the explanation column.
: 10. Provide a brief description of problem in the explanation column.
: 11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.
: 11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0                                                                  Page 12 of 12                                                        OBDI 202 - IOLE Process


Attachment 10 Page 1 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
GG - 2015 - 12                                               DRAFT OPERATING TEST COMMENTS                                                                           ADMIN JPMS
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 4. Job Content
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia            General comments:
: 1. 2.           3. Attributes                                                                                     6.
Errors 5.
JPM#    Dyn  LOD                                                                                                            Explanation U/E/S (D/S) (1-5)  IC  Cues Critical  Scope  Over- Job-    Minutia                                              (See below for instructions)
Focus       Steps     (N/B) lap Link General comments:
: 1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
: 1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
: 2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
: 2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
: 3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
: 3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
: 4. Noun and numerical names are required for pumps, valves, and controllers.
: 4. Noun and numerical names are required for pumps, valves, and controllers.
: 5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion. 6. JPM standards should include a statement that "all critical steps are completed satisfactorily."
: 5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion.
: 7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM. 8. Your numbering system for the JPMs is not very easy to follow. It should be S
: 6. JPM standards should include a statement that all critical steps are completed satisfactorily.
-1, S-2, S-3, and in-plants should be P
: 7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM.
-1, P-2, etc. RO (A R 1) S 2   X     U Label on title page 2015 AR1 No KA assigned to this JPM you are required to assign a K/A to each JPM Task standard is incorrect (Need to use something like "applicant determines that the time to 200 F in the reactor--is 4 hours plus or minus 10 minutes" or something like that.
: 8. Your numbering system for the JPMs is not very easy to follow. It should be S-1, S-2, S-3, and in-plants should be P-1, P-2, etc.
No key was submitted.
RO (AR1)   S     2                 X                                     U
The one critical step is "about four hours" which is not bounded as required by the NUREG Need to bound the acceptable range in the calculation
* Label on title page 2015 AR1
. With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.
* No KA assigned to this JPM you are required to assign a K/A to each JPM
Sat with Rev 2 as submitted in folder RO (A R 2) S 3   X     U
* Task standard is incorrect (Need to use something like applicant determines that the time to 200 F in the reactoris 4 hours plus or minus 10 minutes or something like that.
 
* No key was submitted.
Label on title page 2015 AR2 No key was submitted.
* The one critical step is about four hours which is not bounded as required by the NUREG
No safety function in JPM.
* Need to bound the acceptable range in the calculation.
No KA assigned to the JPM.
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since S
The task standard does not match critical steps.
this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.
The brief standard overview has more info in it than the actual task standard for this JPM.
* Sat with Rev 2 as submitted in folder RO (AR2)   S     3                 X                                     U
With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA.
* Label on title page 2015 AR2
This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO
* No key was submitted.
-only to know fire door operability requirements.
* No safety function in JPM.
 
* No KA assigned to the JPM.
Attachment 10 Page 2 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
* The task standard does not match critical steps.
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
* The brief standard overview has more info in it than the actual task standard for this JPM.
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  Sat with Rev 2 as submitted in folder RO (A R3)  S 3  X    E E
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since S
S  Label on title page 2015 AR3 No safety function in JPM.
this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA. This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO-only to know fire door operability requirements. 0                                                            Page 1 of 12                                                             OBDI 202 - IOLE Process
The task overview states it is a detailed description of the task and yet it has less than th e actual task standard for this JPM.
Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?
 
With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
Recommended replacing the JPM Sat with Rev 2 as submitted in folder RO (A R 4) S 3  X    U S  Label on title page 2015 AR4  No key was submitted.
No Safety Function in JPM The task standard does not match critical steps. Is the battery charger needed or not?


GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                                                                        ADMIN JPMS
: 4. Job Content
: 1. 2.            3. Attributes                                                                                  6.
Errors 5.
JPM#    Dyn  LOD                                                                                                        Explanation U/E/S (D/S) (1-5)  IC  Cues Critical  Scope  Over- Job-    Minutia                                          (See below for instructions)
Focus        Steps    (N/B)  lap  Link
* Sat with Rev 2 as submitted in folder RO (AR3)    S    3                  X                                    E
* Label on title page 2015 AR3
* No safety function in JPM.
* The task overview states it is a detailed description of the task and yet it has less than the actual task standard for this JPM.
* Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?
E
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
* Recommended replacing the JPM S
* Sat with Rev 2 as submitted in folder RO (AR4)    S    3                  X                                    U
* Label on title page 2015 AR4
* No key was submitted.
* No Safety Function in JPM
* The task standard does not match critical steps. Is the battery charger needed or not?
If needed then the key would show the correct answer for this JPM.
If needed then the key would show the correct answer for this JPM.
Need error range on the calculation for the J PM  With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
* Need error range on the calculation for the JPM S
Sat with Rev 2 as submitted in folder SRO (A S1) S 3   X   E
* With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
 
* Sat with Rev 2 as submitted in folder SRO     S     3                           X                           E
E  S  Label on title page 2015 AS1 This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
* Label on title page 2015 AS1 (AS1)
Do not provide them every reference that they could possibly need to complete the JPM.
* This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
* Do not provide them every reference that they could possibly need to complete the JPM.
They are expected to find some things on their own.
They are expected to find some things on their own.
What are the handouts that are to be given? The file only has the section of the UFSAR while the JPM has a list of several. Not sure what has changed.
* What are the handouts that are to be given? The file only has the section of the UFSAR E        while the JPM has a list of several. Not sure what has changed.
Sat with Rev 2 as submitted in folder SRO (AS2) S 3   X     E   Label on title page 2015 AS2 The task standard does not match critical step.  
* Sat with Rev 2 as submitted in folder S
 
SRO     S     3                 X                                     E
Attachment 10 Page 3 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
* Label on title page 2015 AS2 (AS2)
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
* The task standard does not match critical step. 0                                                            Page 2 of 12                                                           OBDI 202 - IOLE Process
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia    S  Task st andard should state-"The risk is yellow because of the EDG and the SSW pump-.." just like to critical step states.
Need the References attached to the JPM Sat with Rev 2 as submitted in folder SRO (AS3) S 3  X    U E        S  Label on title page 2015 AS3 This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
Need a clearly marked key Need to clearly mark handout.
I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn out tail spin. We should consider getting rid of a couple of the non
-critical errors that they need to catch.
Change Task Standard to read "The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN
-OP-102 (noun Name) and Identify all of the critical components listed in the JPM" Do not have the references attached to the handout.
Sat with Rev 2 as submitted in folder SRO (AS4) S 2        E   
 
 
S  Label on title page 2015 AS4 This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
At least you have the dilution flow issue, which is good. It is probably okay though.
Revise Task Standard to Read
 
"The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM" Key still looks like 4 errors


Sat with Rev 2 as submitted in folder SRO (AS5) S   X     E Label on title page 2015 AS5 Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                                                                          ADMIN JPMS
Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesn't just give the applicant the minimum info to get to the correct EAL call.
: 4. Job Content
Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time.
: 1. 2.            3. Attributes                                                                                    6.
Errors 5.
JPM#    Dyn  LOD                                                                                                        Explanation U/E/S (D/S) (1-5)  IC  Cues Critical  Scope  Over- Job-    Minutia                                            (See below for instructions)
Focus        Steps    (N/B)  lap  Link
* Task standard should stateThe risk is yellow because of the EDG and the SSW pump..
just like to critical step states.
* Need the References attached to the JPM S
* Sat with Rev 2 as submitted in folder SRO      S    3                  X                                    U
* Label on title page 2015 AS3 (AS3)
* This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
* Need a clearly marked key
* Need to clearly mark handout.
* I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn E        out tail spin. We should consider getting rid of a couple of the non-critical errors that they need to catch.
* Change Task Standard to read The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN-OP-102 (noun Name) and Identify all of the critical components listed in the JPM
* Do not have the references attached to the handout.
S
* Sat with Rev 2 as submitted in folder SRO     S    2                                                        E
* Label on title page 2015 AS4 (AS4)
* This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
* It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
* At least you have the dilution flow issue, which is good. It is probably okay though.
E
* Revise Task Standard to Read The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM
* Key still looks like 4 errors S
* Sat with Rev 2 as submitted in folder SRO      S                X                                             E
* Label on title page 2015 AS5 (AS5)
* Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
* Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesnt just give the applicant the minimum info to get to the correct EAL call.
* Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time. 0                                                            Page 3 of 12                                                            OBDI 202 - IOLE Process


Attachment 10 Page 4 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
GG - 2015 - 12                                                               DRAFT OPERATING TEST COMMENTS                                                             ADMIN JPMS
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 4. Job Content
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  S   Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG
: 1.       2.               3. Attributes                                                                               6.
-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
Errors 5.
JPM#      Dyn      LOD                                                                                                        Explanation U/E/S (D/S)    (1-5)    IC    Cues Critical  Scope    Over-  Job-    Minutia                                    (See below for instructions)
Focus         Steps     (N/B)   lap   Link S
* Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 2. Determine level of difficulty (LOD) using established 1
: 2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
: 3. Check the appropriate box when an attribute weakness is identified:
: 3. Check the appropriate box when an attribute weakness is identified:
The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
* The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
The JPM does not contain sufficient cues that are objective (not leading).
* The JPM does not contain sufficient cues that are objective (not leading).
All critical steps (elements) have not been properly identified.
* All critical steps (elements) have not been properly identified.
Scope of the task is either too narrow (N) or too broad (B).
* Scope of the task is either too narrow (N) or too broad (B).
Excessive overlap with other part of operating test or written examination.
* Excessive overlap with other part of operating test or written examination.
: 4. Check the appropriate box when a job content error is identified:
: 4. Check the appropriate box when a job content error is identified:
Topics not linked to job content (e.g., disguised task, not required in real job).
* Topics not linked to job content (e.g., disguised task, not required in real job).
Task is trivial and without safety significance.
* Task is trivial and without safety significance.
: 5. Based on the reviewer
: 5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 6. Provide a brief description of any U or E rating in the explanation column.
: 6. Provide a brief description of any U or E rating in the explanation column.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0                                                                            Page 4 of 12                                                    OBDI 202 - IOLE Process


Attachment 10 Page 5 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
GG - 2015 - 12                                               DRAFT OPERATING TEST COMMENTS                               CONTROL ROOM/IN-PLANT SYSTEMS JPMS
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 1. 2.           3. Attributes                                                                                 6.
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia            General Notes:
Errors       5.
: 1. All of the JPMs need to be titled in sequence given on the 301
JPM#    Dyn  LOD                                                                                                      Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                            (See below for instructions)
-2 (S1, S2, S3-P1, P2-) 2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
Focus       Steps     (N/B) lap Link General Notes:
: 1. All of the JPMs need to be titled in sequence given on the 301-2 (S1, S2, S3P1, P2)
: 2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
: 3. The Procedure that will be handed to the applicant needs to be included in the JPM package
: 3. The Procedure that will be handed to the applicant needs to be included in the JPM package
: 4. Task Standards need to be more detailed for example "Applicant will start the SSW "A" pump in accordance with SOI- by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW "A" pump or closing valve-. Within XXX minutes of receiving alarm XXX)"
: 4. Task Standards need to be more detailed for example Applicant will start the SSW A pump in accordance with SOI by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW A pump or closing valve. Within XXX minutes of receiving alarm XXX)
S 1 D 2 X X B   U   
S1      D     2         X       X         B                           U
 
* Label on title page 2015 S1
U
* Need to provide the procedure that will be given to the applicant
 
* There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33-F067B).
Label on title page 2015 S1 Need to provide the procedure that will be given to the applicant There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33
* Need to have the alternate path marked on the JPM where it begins
-F067B). Need to have the alternate path marked on the JPM where it begins The cue to tell the applicant to take action is not allowed it cues the alternate path The cue to tell applicant that fast detent can be used should not be in the standard. Use both noun names and valve numbers in the cue to student and throughout the JPM Task needs to be bounded, how long is the power increase allowed to occur? Specific power level they should not reach pump runout conditions et c-  Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1
* The cue to tell the applicant to take action is not allowed it cues the alternate path
 
* The cue to tell applicant that fast detent can be used should not be in the standard.
Sat with Rev 2 as submitted in folder S 2 (CR2) S 2   B   E
* Use both noun names and valve numbers in the cue to student and throughout the JPM
 
* Task needs to be bounded, how long is the power increase allowed to occur?
Label Title page 2015 S2/CR2 Need to provide the handout that will be given to the applicant.
Specific power level they should not reach pump runout conditions etc U
Need to ensure that the work week schedule will not interfere with the performance of the JPM Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
* Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1 S
In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there.
* Sat with Rev 2 as submitted in folder S2 (CR2)   S     2                           B                           E
Needs to match the critical Attachment 10 Page 6 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
* Label Title page 2015 S2/CR2
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
* Need to provide the handout that will be given to the applicant.
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
* Need to ensure that the work week schedule will not interfere with the performance of the JPM
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  S steps  Sat with Rev 2 as submitted in folder S 3 D 2  X N    E   
* Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
 
* In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there. Needs to match the critical 0                                                            Page 5 of 12                                                         OBDI 202 - IOLE Process
S  Label Title page 2015 S3 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM Steps 4 and 5 should also be marked as critical Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
Step 4 should be recognize a valid group 1 isolation signal occurs It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs Sat with Rev 2 as submitted in folder S 4 D 2    B    E 
 
S  Label Title page 2015 S4 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM Task Overview section change third bullet to "experience a failure to initiate and a failure to inject" Step 7. What is the consequence of not achieving the flow rate in the task standard?  If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student What is the procedure 04 E22-1?  Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
Need to include in the standard what would cause the HPCS start signal written out. Task Standard needs to provide the procedure transition to the alternate path Sat with Rev 2 as submitted in folder S 5 S 2  X B    E Label Title page 2015 S5/CR5 Need to provide the procedure that will be given to the applicant I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13
-P870. If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
Step 1 Notes says to give the applicant attachment 6 should be attachment 14
 
Attachment 10 Page 7 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia E  S  Step 2 should be marked as critical Step 5 should be marked as critical Task standard is not written well. Just venting containment is not good enough. Either state that and "and all critical steps in the JPM completed satisfactorily" or State it as "Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans."    Changed location of the JPM and decided to use the V
-panel to simulate the fan banks  Sat with Rev 2 as submitted in folder S 6 D 2    B    U   
 
 
Label Title page 2015 S6 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM Do two or nine rods scram initially?  It is not clear on what is going on in the task standard.
When depress the manual scram switch (step 5.4) the JPM std states that the standard "Depress MAN SCRAM Switch" and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesn't match this either. It states that when the first scram switch is depressed, the nine rods scram There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
How long is acceptable to wait to scram?
If we keep it then you will need to fix all the issues with the JPM Number of Critical steps need changed to 3 Fix Task Standard to
 
"The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06
-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05 02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily."
Step 4 it needs to indicate that 9 rods scram not 4
 
Attachment 10 Page 8 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia      S  Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM Sat with Rev 2 as submitted in folder S 7 D 2  X  B    E 
 
 
S  Label Title page 2015 S7 Need to provide the procedure that will be given to the applicant Need to mark where the alternate path begins in the JPM JPM step 1 What happens if the switch is not taken to TEST?  Would this be a critical step to be able to accomplish the task?
There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path.
No credit is given to th e applicant for items that are cue d  How long is it acceptable to take to isolate the leak?  Needs included in task standard. Step after step 12 should be step 13 not step 8 Task standard needs to be more detailed form a legal perspective Revise Task Standard to "Applicant is to perform a manual start of SSW pump A in accordance with 04 01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily."
On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)
Sat with Rev 2 as submitted in folder S 8 D 2    B    E 
 
E    Label Title page 2015 S8 Need to provide the procedure that will be given to the applicant The task standard should be something like "completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM."
Step 20 of JPM What happens if they hold it long enough to get a high alarm?  Is this a failure criteria?
Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed Step after step 25 should be step 26 not step 8 Minor Edit to make the JPM flow with little down time on unverifiable actions is to


have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed.
GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                            CONTROL ROOM/IN-PLANT SYSTEMS JPMS
: 4. Job Content
: 1. 2.            3. Attributes                                                                              6.
Errors      5.
JPM#    Dyn  LOD                                                                                                    Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                          (See below for instructions)
Focus        Steps    (N/B)  lap  Link steps S
* Sat with Rev 2 as submitted in folder S3      D    2                  X        N                          E
* Label Title page 2015 S3
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* Steps 4 and 5 should also be marked as critical
* Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
* Step 4 should be recognize a valid group 1 isolation signal occurs
* It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs S
* Sat with Rev 2 as submitted in folder S4      D    2                            B                          E
* Label Title page 2015 S4
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* Task Overview section change third bullet to experience a failure to initiate and a failure to inject
* Step 7. What is the consequence of not achieving the flow rate in the task standard? If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
* Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student
* What is the procedure 04-1-E22-1? Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
* Need to include in the standard what would cause the HPCS start signal written out.
* Task Standard needs to provide the procedure transition to the alternate path S
* Sat with Rev 2 as submitted in folder S5      S    2                 X        B                          E
* Label Title page 2015 S5/CR5
* Need to provide the procedure that will be given to the applicant
* I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13-P870.
* If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
* Step 1 Notes says to give the applicant attachment 6 should be attachment 14 0                                                            Page 6 of 12                                                        OBDI 202 - IOLE Process


Attachment 10 Page 9 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN
GG - 2015 - 12                                               DRAFT OPERATING TEST COMMENTS                             CONTROL ROOM/IN-PLANT SYSTEMS JPMS
-PLANT SYSTEM S JPM S JPM# 1. Dyn (D/S) 2. LOD (1-5) 3. Attributes
: 4. Job Content
: 4. Job Content Errors 5. U/E/S 6. Explanation (See below for instructions)
: 1. 2.           3. Attributes                                                                               6.
IC Focus Cues  Critical Steps Scope (N/B) Over- lap Job-Link Minutia  S   Sat with Rev 2 as submitted in folder P1 S     B   E      
Errors       5.
JPM#    Dyn  LOD                                                                                                    Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                          (See below for instructions)
Focus       Steps     (N/B) lap Link
* Step 2 should be marked as critical
* Step 5 should be marked as critical
* Task standard is not written well. Just venting containment is not good enough.
Either state that and and all critical steps in the JPM completed satisfactorily or State it as Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans.
E
* Changed location of the JPM and decided to use the V-panel to simulate the fan banks S
* Sat with Rev 2 as submitted in folder S6      D     2                            B                           U
* Label Title page 2015 S6
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* Do two or nine rods scram initially?
* It is not clear on what is going on in the task standard. When depress the manual scram switch (step 5.4) the JPM std states that the standard Depress MAN SCRAM Switch and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
* Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesnt match this either. It states that when the first scram switch is depressed, the nine rods scram
* There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam
* Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
* How long is acceptable to wait to scram?
* If we keep it then you will need to fix all the issues with the JPM E
* Number of Critical steps need changed to 3
* Fix Task Standard to The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05-1-02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily.
* Step 4 it needs to indicate that 9 rods scram not 4 0                                                            Page 7 of 12                                                        OBDI 202 - IOLE Process


S  Label Title page 2015 P1 Need to provide the procedure that will be given to the applicant What happens if the applicant tries to reset a n overcurrent really? The standard say s that they should not do this.
GG - 2015 - 12                                                DRAFT OPERATING TEST COMMENTS                              CONTROL ROOM/IN-PLANT SYSTEMS JPMS
Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps In-plant JPM cues need to be correct for in
: 4. Job Content
-plant JPMs. You can't use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
: 1. 2.            3. Attributes                                                                              6.
The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
Errors      5.
I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week Sat with Rev 2 as submitted in folder P2 S    B    E 
JPM#    Dyn  LOD                                                                                                    Explanation IC  Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                          (See below for instructions)
Focus        Steps    (N/B)  lap Link
* Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
* Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM S
* Sat with Rev 2 as submitted in folder S7      D    2          X                B                          E
* Label Title page 2015 S7
* Need to provide the procedure that will be given to the applicant
* Need to mark where the alternate path begins in the JPM
* JPM step 1 What happens if the switch is not taken to TEST? Would this be a critical step to be able to accomplish the task?
* There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path. No credit is given to the applicant for items that are cued
* How long is it acceptable to take to isolate the leak? Needs included in task standard.
* Step after step 12 should be step 13 not step 8
* Task standard needs to be more detailed form a legal perspective E
* Revise Task Standard to Applicant is to perform a manual start of SSW pump A in accordance with 04                                                                                      01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily.
* On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)
S
* Sat with Rev 2 as submitted in folder S8      D    2                            B                          E
* Label Title page 2015 S8
* Need to provide the procedure that will be given to the applicant
* The task standard should be something like completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM.
* Step 20 of JPM What happens if they hold it long enough to get a high alarm? Is this a failure criteria?
* Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed
* Step after step 25 should be step 26 not step 8
* Minor Edit to make the JPM flow with little down time on unverifiable actions is to E
have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed. 0                                                            Page 8 of 12                                                        OBDI 202 - IOLE Process


S  Label Title page 2015 P2 Need to provide the procedure that will be given to the applicant I do not understand how this is an EN JPM, Please explain Task Standard is not up to legal standards as explained in other JPM comments Sat with Rev 2 as submitted in folder P3 S     N   E
GG - 2015 - 12                                                      DRAFT OPERATING TEST COMMENTS                            CONTROL ROOM/IN-PLANT SYSTEMS JPMS
: 4. Job Content
: 1. 2.              3. Attributes                                                                              6.
Errors      5.
JPM#        Dyn  LOD                                                                                                      Explanation IC    Cues Critical  Scope  Over- Job-    Minutia U/E/S (D/S) (1-5)                                                                                            (See below for instructions)
Focus          Steps    (N/B)  lap  Link S
* Sat with Rev 2 as submitted in folder P1          S                                      B                          E
* Label Title page 2015 P1
* Need to provide the procedure that will be given to the applicant
* What happens if the applicant tries to reset an overcurrent really? The standard says that they should not do this.
* Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps
* In-plant JPM cues need to be correct for in-plant JPMs. You cant use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
* The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
* I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week
* Sat with Rev 2 as submitted in folder S
P2          S                                      B                          E
* Label Title page 2015 P2
* Need to provide the procedure that will be given to the applicant
* I do not understand how this is an EN JPM, Please explain
* Task Standard is not up to legal standards as explained in other JPM comments
* Sat with Rev 2 as submitted in folder S
P3         S                                     N                           E
* Label Title Page 2015 P3
* Need to provide the procedure that will be given to the applicant
* JPM was altered during validation to more accurately mirror actual method of completion.
S
* Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in 0                                                                    Page 9 of 12                                                      OBDI 202 - IOLE Process


S  Label Title Page 2015 P3 Need to provide the procedure that will be given to the applicant JPM was altered during validation to more accurately mirror actual method of completion.
reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG
-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in Attachment 10 Page 10 of 12 OBDI 202 - I OLE Process reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
: 2. Determine level of difficulty (LOD) using established 1
: 2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested. 3. Check the appropriate box when an attribute weakness is identified:
: 3. Check the appropriate box when an attribute weakness is identified:
$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
      $ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
$ The JPM does not contain sufficient cues that are objective (not leading). $ All critical steps (elements) have not been properly identified.
      $ The JPM does not contain sufficient cues that are objective (not leading).
$ Scope of the task is either too narrow (N) or too broad (B).
      $ All critical steps (elements) have not been properly identified.
$ Excessive overlap with other part of operating test or written examination.
      $ Scope of the task is either too narrow (N) or too broad (B).
      $ Excessive overlap with other part of operating test or written examination.
: 4. Check the appropriate box when a job content error is identified:
: 4. Check the appropriate box when a job content error is identified:
Topics not linked to job content (e.g., disguised task, not required in real job).
* Topics not linked to job content (e.g., disguised task, not required in real job).
Task is trivial and without safety significance.
* Task is trivial and without safety significance.
: 5. Based on the reviewer
: 5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
: 6. Provide a brief description of any U or E rating in the explanation column.
: 6. Provide a brief description of any U or E rating in the explanation column.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.
: 7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0                                                                      Page 10 of 12                                                   OBDI 202 - IOLE Process
 
Attachment 10 Page 11 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIO S Scenario Set 1. ES 2. TS 3. Crit 4. IC 5. Pred 6. TL 7. L/C 8. Eff 9. U/E/S 10. Explanation (See below for instructions) 1        S  E 
 
S No comments prior to validation
 
Validation Week Comments to incorporate Replaced Event 8 with a feedwater pump trip Added applicable tech spec entry to event 3 Added applicable tech spec condition to even t 4  On D-1 corrected the order of events Added bounding criteria to Critical Task's Changed Critical task 2 due to a change in event 8 Fixed simulator Setup information on page 2 of the D
-2 from 7.7 to 7.5 Fixed turnover sheet to reflect step 7.5 as well Added condition A to Tech Spec call in event 3 on D
-2  Removed potential tech spec (tracking LCO) from event 3 Added condition A to LCO call for event 4 Added Critical Task bounding criteria to event 6 on D
-2  Add direction to booth to bring completed attachments to the control room as would be done in the plant Added critical Task bounding criterial to event 7 Made event 7 a NRC trigger Sat with Rev 2 as submitted in folder 2        S  E 
 
S No Comments prior to validation week
 
Validation week comments to incorporate Tech Specs on D
-1 for events 1,2, 4and 5 were incomplete In Scenario Activities sections added appropriate Tech Spec's and conditions to match D
-2  Event 2 and 3 needs NRC trigger symbol Event 4 is TR 3.1.5 condition A applicable?
Missing TS on event 5 Fix bullets on page 6 Fix typo on page 12 of D
-2 for B RFPT Add noun names on event 7 of D
-2  Provide details for examiners on event 7 verification of ECCS systems Add noun names for SRV's Sat with Rev 2 as submitted in folder 3        S  E No Comments prior to validations week Validation week comments to incorporate


Attachment 10 Page 12 of 12 OBDI 202 - I OLE Process GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIO S      
GG - 2015 - 12                         DRAFT OPERATING TEST COMMENTS                                                         SCENARIOS Scenario    1. 2. 3. 4. 5. 6. 7. 8. 9.
: 10. Explanation (See below for instructions)
Set      ES TS Crit IC Pred TL L/C    Eff  U/E/S 1                                            S  No comments prior to validation E  Validation Week Comments to incorporate
* Replaced Event 8 with a feedwater pump trip
* Added applicable tech spec entry to event 3
* Added applicable tech spec condition to event 4
* On D-1 corrected the order of events
* Added bounding criteria to Critical Tasks
* Changed Critical task 2 due to a change in event 8
* Fixed simulator Setup information on page 2 of the D-2 from 7.7 to 7.5
* Fixed turnover sheet to reflect step 7.5 as well
* Added condition A to Tech Spec call in event 3 on D-2
* Removed potential tech spec (tracking LCO) from event 3
* Added condition A to LCO call for event 4
* Added Critical Task bounding criteria to event 6 on D-2
* Add direction to booth to bring completed attachments to the control room as would be done in the plant
* Added critical Task bounding criterial to event 7
* Made event 7 a NRC trigger S
* Sat with Rev 2 as submitted in folder 2                                            S  No Comments prior to validation week E  Validation week comments to incorporate
* Tech Specs on D-1 for events 1,2,4and 5 were incomplete
* In Scenario Activities sections added appropriate Tech Specs and conditions to match D-2
* Event 2 and 3 needs NRC trigger symbol
* Event 4 is TR 3.1.5 condition A applicable?
* Missing TS on event 5
* Fix bullets on page 6
* Fix typo on page 12 of D-2 for B RFPT
* Add noun names on event 7 of D-2
* Provide details for examiners on event 7 verification of ECCS systems
* Add noun names for SRVs S
* Sat with Rev 2 as submitted in folder 3                                            S  No Comments prior to validations week E  Validation week comments to incorporate 0                                  Page 11 of 12                                                    OBDI 202 - IOLE Process


Page 1 od D
GG - 2015 - 12                                                        DRAFT OPERATING TEST COMMENTS                                                                SCENARIOS
-1 add the SBLOCA to event 9 as it is in the D
* Page 1 od D-1 add the SBLOCA to event 9 as it is in the D-2
-2 Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2 D-1 page 2, put TR in front of 3.3.2.1 on event 5 Change critical tasks 3 to 2 one of the critical tasks is not EOP based D-1 page 3, add Tech spec 3.7.2 condition B to MCC 17B01 Trip D-2 page 5, list the alarm noun names D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform D-2 page 7, give the noun name for the alarm that is required to be addressed D-2 page 10, add bounding criteria to Critical task D-2 page 12, add the conditions for TRM 3.3.2.1 D-2 page 13, list that alarm that will cue applicant with noun name Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG
* Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2
-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
* D-1 page 2, put TR in front of 3.3.2.1 on event 5
* Change critical tasks 3 to 2 one of the critical tasks is not EOP based
* D-1 page 3, add Tech spec 3.7.2 condition B to MCC 17B01 Trip
* D-2 page 5, list the alarm noun names
* D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform
* D-2 page 7, give the noun name for the alarm that is required to be addressed
* D-2 page 10, add bounding criteria to Critical task
* D-2 page 12, add the conditions for TRM 3.3.2.1
* D-2 page 13, list that alarm that will cue applicant with noun name S
* Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
: 1. ES: ES-301 checklists 4, 5, & 6 satisfied.
: 1. ES: ES-301 checklists 4, 5, & 6 satisfied.
: 2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
: 2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
: 3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES
: 3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
-D-2. 4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
: 4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
: 5. Pred: Scenario sequence and other factors avoid predictability issues.
: 5. Pred: Scenario sequence and other factors avoid predictability issues.
: 6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
: 6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
: 7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
: 7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
: 8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions. 9. Based on the reviewer
: 8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
: 9. Based on the reviewer=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
: 10. Provide a brief description of problem in the explanation column.
: 10. Provide a brief description of problem in the explanation column.
: 11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form.}}
: 11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0                                                                  Page 12 of 12                                                        OBDI 202 - IOLE Process}}

Latest revision as of 01:25, 31 October 2019

2015-12 Draft Op Test Comments
ML16028A021
Person / Time
Site: Grand Gulf  Entergy icon.png
Issue date: 12/14/2015
From: Vincent Gaddy
Operations Branch IV
To:
Entergy Operations
References
50-416/15-012, 50-417/15-012
Download: ML16028A021 (12)


Text

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link General comments:

1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
4. Noun and numerical names are required for pumps, valves, and controllers.
5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion.
6. JPM standards should include a statement that all critical steps are completed satisfactorily.
7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM.
8. Your numbering system for the JPMs is not very easy to follow. It should be S-1, S-2, S-3, and in-plants should be P-1, P-2, etc.

RO (AR1) S 2 X U

  • Label on title page 2015 AR1
  • No KA assigned to this JPM you are required to assign a K/A to each JPM
  • Task standard is incorrect (Need to use something like applicant determines that the time to 200 F in the reactoris 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> plus or minus 10 minutes or something like that.
  • No key was submitted.
  • The one critical step is about four hours which is not bounded as required by the NUREG
  • Need to bound the acceptable range in the calculation.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since S

this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.

  • Sat with Rev 2 as submitted in folder RO (AR2) S 3 X U
  • Label on title page 2015 AR2
  • No key was submitted.
  • No safety function in JPM.
  • No KA assigned to the JPM.
  • The task standard does not match critical steps.
  • The brief standard overview has more info in it than the actual task standard for this JPM.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since S

this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA. This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO-only to know fire door operability requirements. 0 Page 1 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Sat with Rev 2 as submitted in folder RO (AR3) S 3 X E
  • Label on title page 2015 AR3
  • No safety function in JPM.
  • The task overview states it is a detailed description of the task and yet it has less than the actual task standard for this JPM.
  • Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?

E

  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Recommended replacing the JPM S
  • Sat with Rev 2 as submitted in folder RO (AR4) S 3 X U
  • Label on title page 2015 AR4
  • No key was submitted.
  • No Safety Function in JPM
  • The task standard does not match critical steps. Is the battery charger needed or not?

If needed then the key would show the correct answer for this JPM.

  • Need error range on the calculation for the JPM S
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Sat with Rev 2 as submitted in folder SRO S 3 X E
  • Label on title page 2015 AS1 (AS1)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Do not provide them every reference that they could possibly need to complete the JPM.

They are expected to find some things on their own.

  • What are the handouts that are to be given? The file only has the section of the UFSAR E while the JPM has a list of several. Not sure what has changed.
  • Sat with Rev 2 as submitted in folder S

SRO S 3 X E

  • Label on title page 2015 AS2 (AS2)
  • The task standard does not match critical step. 0 Page 2 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Task standard should stateThe risk is yellow because of the EDG and the SSW pump..

just like to critical step states.

  • Need the References attached to the JPM S
  • Sat with Rev 2 as submitted in folder SRO S 3 X U
  • Label on title page 2015 AS3 (AS3)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Need a clearly marked key
  • Need to clearly mark handout.
  • I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn E out tail spin. We should consider getting rid of a couple of the non-critical errors that they need to catch.
  • Change Task Standard to read The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN-OP-102 (noun Name) and Identify all of the critical components listed in the JPM
  • Do not have the references attached to the handout.

S

  • Sat with Rev 2 as submitted in folder SRO S 2 E
  • Label on title page 2015 AS4 (AS4)
  • This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
  • It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
  • At least you have the dilution flow issue, which is good. It is probably okay though.

E

  • Revise Task Standard to Read The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM
  • Key still looks like 4 errors S
  • Sat with Rev 2 as submitted in folder SRO S X E
  • Label on title page 2015 AS5 (AS5)
  • Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
  • Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesnt just give the applicant the minimum info to get to the correct EAL call.
  • Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time. 0 Page 3 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 4 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link General Notes:

1. All of the JPMs need to be titled in sequence given on the 301-2 (S1, S2, S3P1, P2)
2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
3. The Procedure that will be handed to the applicant needs to be included in the JPM package
4. Task Standards need to be more detailed for example Applicant will start the SSW A pump in accordance with SOI by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW A pump or closing valve. Within XXX minutes of receiving alarm XXX)

S1 D 2 X X B U

  • Label on title page 2015 S1
  • Need to provide the procedure that will be given to the applicant
  • There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33-F067B).
  • Need to have the alternate path marked on the JPM where it begins
  • The cue to tell the applicant to take action is not allowed it cues the alternate path
  • The cue to tell applicant that fast detent can be used should not be in the standard.
  • Use both noun names and valve numbers in the cue to student and throughout the JPM
  • Task needs to be bounded, how long is the power increase allowed to occur?

Specific power level they should not reach pump runout conditions etc U

  • Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1 S
  • Sat with Rev 2 as submitted in folder S2 (CR2) S 2 B E
  • Label Title page 2015 S2/CR2
  • Need to provide the handout that will be given to the applicant.
  • Need to ensure that the work week schedule will not interfere with the performance of the JPM
  • Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
  • In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there. Needs to match the critical 0 Page 5 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link steps S

  • Sat with Rev 2 as submitted in folder S3 D 2 X N E
  • Label Title page 2015 S3
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Steps 4 and 5 should also be marked as critical
  • Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
  • Step 4 should be recognize a valid group 1 isolation signal occurs
  • It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs S
  • Sat with Rev 2 as submitted in folder S4 D 2 B E
  • Label Title page 2015 S4
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Task Overview section change third bullet to experience a failure to initiate and a failure to inject
  • Step 7. What is the consequence of not achieving the flow rate in the task standard? If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
  • Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student
  • What is the procedure 04-1-E22-1? Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
  • Need to include in the standard what would cause the HPCS start signal written out.
  • Task Standard needs to provide the procedure transition to the alternate path S
  • Sat with Rev 2 as submitted in folder S5 S 2 X B E
  • Label Title page 2015 S5/CR5
  • Need to provide the procedure that will be given to the applicant
  • I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13-P870.
  • If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
  • Step 1 Notes says to give the applicant attachment 6 should be attachment 14 0 Page 6 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 2 should be marked as critical
  • Step 5 should be marked as critical
  • Task standard is not written well. Just venting containment is not good enough.

Either state that and and all critical steps in the JPM completed satisfactorily or State it as Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans.

E

  • Changed location of the JPM and decided to use the V-panel to simulate the fan banks S
  • Sat with Rev 2 as submitted in folder S6 D 2 B U
  • Label Title page 2015 S6
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Do two or nine rods scram initially?
  • It is not clear on what is going on in the task standard. When depress the manual scram switch (step 5.4) the JPM std states that the standard Depress MAN SCRAM Switch and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
  • Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesnt match this either. It states that when the first scram switch is depressed, the nine rods scram
  • There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam
  • Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
  • How long is acceptable to wait to scram?
  • If we keep it then you will need to fix all the issues with the JPM E
  • Number of Critical steps need changed to 3
  • Fix Task Standard to The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05-1-02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily.
  • Step 4 it needs to indicate that 9 rods scram not 4 0 Page 7 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
  • Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM S
  • Sat with Rev 2 as submitted in folder S7 D 2 X B E
  • Label Title page 2015 S7
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • JPM step 1 What happens if the switch is not taken to TEST? Would this be a critical step to be able to accomplish the task?
  • There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path. No credit is given to the applicant for items that are cued
  • How long is it acceptable to take to isolate the leak? Needs included in task standard.
  • Step after step 12 should be step 13 not step 8
  • Task standard needs to be more detailed form a legal perspective E
  • Revise Task Standard to Applicant is to perform a manual start of SSW pump A in accordance with 04 01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily.
  • On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)

S

  • Sat with Rev 2 as submitted in folder S8 D 2 B E
  • Label Title page 2015 S8
  • Need to provide the procedure that will be given to the applicant
  • The task standard should be something like completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM.
  • Step 20 of JPM What happens if they hold it long enough to get a high alarm? Is this a failure criteria?
  • Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed
  • Step after step 25 should be step 26 not step 8
  • Minor Edit to make the JPM flow with little down time on unverifiable actions is to E

have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed. 0 Page 8 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder P1 S B E
  • Label Title page 2015 P1
  • Need to provide the procedure that will be given to the applicant
  • What happens if the applicant tries to reset an overcurrent really? The standard says that they should not do this.
  • Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps
  • In-plant JPM cues need to be correct for in-plant JPMs. You cant use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
  • The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
  • I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week
  • Sat with Rev 2 as submitted in folder S

P2 S B E

  • Label Title page 2015 P2
  • Need to provide the procedure that will be given to the applicant
  • I do not understand how this is an EN JPM, Please explain
  • Task Standard is not up to legal standards as explained in other JPM comments
  • Sat with Rev 2 as submitted in folder S

P3 S N E

  • Label Title Page 2015 P3
  • Need to provide the procedure that will be given to the applicant
  • JPM was altered during validation to more accurately mirror actual method of completion.

S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in 0 Page 9 of 12 OBDI 202 - IOLE Process

reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:

$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

$ The JPM does not contain sufficient cues that are objective (not leading).

$ All critical steps (elements) have not been properly identified.

$ Scope of the task is either too narrow (N) or too broad (B).

$ Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 10 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS Scenario 1. 2. 3. 4. 5. 6. 7. 8. 9.

10. Explanation (See below for instructions)

Set ES TS Crit IC Pred TL L/C Eff U/E/S 1 S No comments prior to validation E Validation Week Comments to incorporate

  • Added applicable tech spec entry to event 3
  • Added applicable tech spec condition to event 4
  • On D-1 corrected the order of events
  • Added bounding criteria to Critical Tasks
  • Changed Critical task 2 due to a change in event 8
  • Fixed simulator Setup information on page 2 of the D-2 from 7.7 to 7.5
  • Fixed turnover sheet to reflect step 7.5 as well
  • Added condition A to Tech Spec call in event 3 on D-2
  • Removed potential tech spec (tracking LCO) from event 3
  • Added condition A to LCO call for event 4
  • Added Critical Task bounding criteria to event 6 on D-2
  • Add direction to booth to bring completed attachments to the control room as would be done in the plant
  • Added critical Task bounding criterial to event 7
  • Made event 7 a NRC trigger S
  • Sat with Rev 2 as submitted in folder 2 S No Comments prior to validation week E Validation week comments to incorporate
  • Tech Specs on D-1 for events 1,2,4and 5 were incomplete
  • In Scenario Activities sections added appropriate Tech Specs and conditions to match D-2
  • Event 2 and 3 needs NRC trigger symbol
  • Event 4 is TR 3.1.5 condition A applicable?
  • Missing TS on event 5
  • Fix bullets on page 6
  • Fix typo on page 12 of D-2 for B RFPT
  • Add noun names on event 7 of D-2
  • Provide details for examiners on event 7 verification of ECCS systems
  • Add noun names for SRVs S
  • Sat with Rev 2 as submitted in folder 3 S No Comments prior to validations week E Validation week comments to incorporate 0 Page 11 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS

  • Page 1 od D-1 add the SBLOCA to event 9 as it is in the D-2
  • Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2
  • D-1 page 2, put TR in front of 3.3.2.1 on event 5
  • Change critical tasks 3 to 2 one of the critical tasks is not EOP based
  • D-2 page 5, list the alarm noun names
  • D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform
  • D-2 page 7, give the noun name for the alarm that is required to be addressed
  • D-2 page 10, add bounding criteria to Critical task
  • D-2 page 12, add the conditions for TRM 3.3.2.1
  • D-2 page 13, list that alarm that will cue applicant with noun name S
  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. ES: ES-301 checklists 4, 5, & 6 satisfied.
2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5. Pred: Scenario sequence and other factors avoid predictability issues.
6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9. Based on the reviewer=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
10. Provide a brief description of problem in the explanation column.
11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 12 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link General comments:

1. All admin JPMs that require calculations, use of a graph, or partially implemented procedures (such as tagouts, etc) need to be included as a marked up key. Where calcs are involved there must be a value and error bars. Approximate and around are not allowed.
2. Critical steps in a JPM need to be incorporated into the task standard for the standard to be complete (at least the steps most important to success of the JPM).
3. Alternate path JPMs are required to be marked when the alt path starts with the proper procedure transition documented in the JPM.
4. Noun and numerical names are required for pumps, valves, and controllers.
5. JPMs need to take into consideration a bounded limit if the applicant does not take appropriate action in a timely fashion.
6. JPM standards should include a statement that all critical steps are completed satisfactorily.
7. Sometimes the JPM step, standard, and notes sections are confusing. The step is always the actual step from the procedure, the std is what you expect them to do, what they see happen, and any transition piece (if required for alt path). Communications between applicant and CRS, field operators, etc, should be in the notes or in separate lines between the steps of the JPM.
8. Your numbering system for the JPMs is not very easy to follow. It should be S-1, S-2, S-3, and in-plants should be P-1, P-2, etc.

RO (AR1) S 2 X U

  • Label on title page 2015 AR1
  • No KA assigned to this JPM you are required to assign a K/A to each JPM
  • Task standard is incorrect (Need to use something like applicant determines that the time to 200 F in the reactoris 4 hours4.62963e-5 days <br />0.00111 hours <br />6.613757e-6 weeks <br />1.522e-6 months <br /> plus or minus 10 minutes or something like that.
  • No key was submitted.
  • The one critical step is about four hours which is not bounded as required by the NUREG
  • Need to bound the acceptable range in the calculation.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 205000 K3.03 (3.8/3.9) since S

this is specific to this concept. The 2.4.11 KA you included for this JPM is knowledge of AOPs and the point values are wrong for this KA.

  • Sat with Rev 2 as submitted in folder RO (AR2) S 3 X U
  • Label on title page 2015 AR2
  • No key was submitted.
  • No safety function in JPM.
  • No KA assigned to the JPM.
  • The task standard does not match critical steps.
  • The brief standard overview has more info in it than the actual task standard for this JPM.
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 286000 A1.03 (2.8/3.1) since S

this is specific to this concept. The 2.1.20 KA you included for this JPM is not as good as the system specific KA. This appears to be an SRO type admin JPM. During validation week we decided to change this out to a radiation control JPM since it could be argued that is is SRO-only to know fire door operability requirements. 0 Page 1 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Sat with Rev 2 as submitted in folder RO (AR3) S 3 X E
  • Label on title page 2015 AR3
  • No safety function in JPM.
  • The task overview states it is a detailed description of the task and yet it has less than the actual task standard for this JPM.
  • Also, sometimes the task standard is at the back of the JPM, sometimes the front, is there a formatting change to JPMs at GG from old bank JPMs?

E

  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 212000 K1.04 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Recommended replacing the JPM S
  • Sat with Rev 2 as submitted in folder RO (AR4) S 3 X U
  • Label on title page 2015 AR4
  • No key was submitted.
  • No Safety Function in JPM
  • The task standard does not match critical steps. Is the battery charger needed or not?

If needed then the key would show the correct answer for this JPM.

  • Need error range on the calculation for the JPM S
  • With rev 1 of the JPM it is now Sat. A better KA for this JPM is 209002 A1.07 since this is specific to this concept. The generic KA you included for this JPM is not as good as the system specific KA.
  • Sat with Rev 2 as submitted in folder SRO S 3 X E
  • Label on title page 2015 AS1 (AS1)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Do not provide them every reference that they could possibly need to complete the JPM.

They are expected to find some things on their own.

  • What are the handouts that are to be given? The file only has the section of the UFSAR E while the JPM has a list of several. Not sure what has changed.
  • Sat with Rev 2 as submitted in folder S

SRO S 3 X E

  • Label on title page 2015 AS2 (AS2)
  • The task standard does not match critical step. 0 Page 2 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link

  • Task standard should stateThe risk is yellow because of the EDG and the SSW pump..

just like to critical step states.

  • Need the References attached to the JPM S
  • Sat with Rev 2 as submitted in folder SRO S 3 X U
  • Label on title page 2015 AS3 (AS3)
  • This task is too long for the NRC exam and will probably cause the entire set of admin JPMs for the SRO applicants to exceed the time allowed by the NUREG to administer the admin JPMs (1.5 to 2 hrs is the target for the admin JPMs).
  • Need a clearly marked key
  • Need to clearly mark handout.
  • I believe that 6 errors is excessive for this JPM it will send the applicants into a long drawn E out tail spin. We should consider getting rid of a couple of the non-critical errors that they need to catch.
  • Change Task Standard to read The applicant is the review a tagout approval and determine that the proposed tagout lacks the required isolations of EN-OP-102 (noun Name) and Identify all of the critical components listed in the JPM
  • Do not have the references attached to the handout.

S

  • Sat with Rev 2 as submitted in folder SRO S 2 E
  • Label on title page 2015 AS4 (AS4)
  • This looks to like there are four errors with this JPM (from the key), not three as the JPM standard shows.
  • It would be better to have more SRO relevant information and more safety significant items to correct than just signatures.
  • At least you have the dilution flow issue, which is good. It is probably okay though.

E

  • Revise Task Standard to Read The applicant is required to review the data provided in parts 1,2,3,4,and 5 of LWD discharge Permit provided and identify all of the critical errors listed in the JPM
  • Key still looks like 4 errors S
  • Sat with Rev 2 as submitted in folder SRO S X E
  • Label on title page 2015 AS5 (AS5)
  • Cue is too easy and not operationally valid (not how this would happen in real life in the CR).
  • Too improve this JPM setup the stem with other important facts so that it is operationally relevant but doesnt just give the applicant the minimum info to get to the correct EAL call.
  • Some examples include provide the applicant with a narrative of the plant conditions so that the applicant needs to determine form that narrative what the important information is and then determine the correct EAL call, like give the release rate and a wind direction that is below the threshold of a EAL call and that the busses are being powered by the diesels with expected off site power to be back within a specific period of time. 0 Page 3 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS ADMIN JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation U/E/S (D/S) (1-5) IC Cues Critical Scope Over- Job- Minutia (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:
  • The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.
  • The JPM does not contain sufficient cues that are objective (not leading).
  • All critical steps (elements) have not been properly identified.
  • Scope of the task is either too narrow (N) or too broad (B).
  • Excessive overlap with other part of operating test or written examination.
4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 4 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link General Notes:

1. All of the JPMs need to be titled in sequence given on the 301-2 (S1, S2, S3P1, P2)
2. The Alternate Path must be clearly marked as to where it begins in the JPM either by inserting a box between the last normal step or at the start of the first alternate path step.
3. The Procedure that will be handed to the applicant needs to be included in the JPM package
4. Task Standards need to be more detailed for example Applicant will start the SSW A pump in accordance with SOI by completing all of the critical steps and recognize a leak has developed on the SSW system and Isolate the leak by either securing the SSW A pump or closing valve. Within XXX minutes of receiving alarm XXX)

S1 D 2 X X B U

  • Label on title page 2015 S1
  • Need to provide the procedure that will be given to the applicant
  • There should be two paths for the alternated path, one for tripping the hydraulic power unit as written and one for closing the valve that is listed in the task overview (B33-F067B).
  • Need to have the alternate path marked on the JPM where it begins
  • The cue to tell the applicant to take action is not allowed it cues the alternate path
  • The cue to tell applicant that fast detent can be used should not be in the standard.
  • Use both noun names and valve numbers in the cue to student and throughout the JPM
  • Task needs to be bounded, how long is the power increase allowed to occur?

Specific power level they should not reach pump runout conditions etc U

  • Have question if the second alternate path was OK in the original how is it not acceptable in the revision? Once that is explained the JPM is sat with the revision 1 S
  • Sat with Rev 2 as submitted in folder S2 (CR2) S 2 B E
  • Label Title page 2015 S2/CR2
  • Need to provide the handout that will be given to the applicant.
  • Need to ensure that the work week schedule will not interfere with the performance of the JPM
  • Need to provide a marked up picture of the inside of the cabinet that shows each relay and during validation ensure that the examiners understand exactly how to unclip and remove the relay.
  • In the task over view section (detailed description) list the relays and cabinets that will be pulled in addition to what is already there. Needs to match the critical 0 Page 5 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link steps S

  • Sat with Rev 2 as submitted in folder S3 D 2 X N E
  • Label Title page 2015 S3
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Steps 4 and 5 should also be marked as critical
  • Need the other 7 MSIV valve numbers included in the task standard for examiner place keeping.
  • Step 4 should be recognize a valid group 1 isolation signal occurs
  • It appears that 1 MSIV closes and the other seven remain open. The task standard and associated steps need to be clear that 1 MSIV went shut but seven remained open and the applicant is required to perform the manual isolation, which will be successful in closing the remaining seven MSIVs S
  • Sat with Rev 2 as submitted in folder S4 D 2 B E
  • Label Title page 2015 S4
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Task Overview section change third bullet to experience a failure to initiate and a failure to inject
  • Step 7. What is the consequence of not achieving the flow rate in the task standard? If the operator stays below or goes above and throttles back is it still a failure to meet the critical step?
  • Need to put a step after Step 7 that states that the operator is required to recognize that there is a failure to inject and give the report to the SRO, include the cues that the examiner is to give the student
  • What is the procedure 04-1-E22-1? Where is the applicant expected to get it, and is there a requirement to use the procedure or is skill of the craft acceptable?
  • Need to include in the standard what would cause the HPCS start signal written out.
  • Task Standard needs to provide the procedure transition to the alternate path S
  • Sat with Rev 2 as submitted in folder S5 S 2 X B E
  • Label Title page 2015 S5/CR5
  • Need to provide the procedure that will be given to the applicant
  • I do not understand why this is being done in the control room. If all of the actions are done on panel 1H13-P870.
  • If this needs to be done in the control room the work week schedule needs to be verified that we will not be held up due to other control room activities.
  • Step 1 Notes says to give the applicant attachment 6 should be attachment 14 0 Page 6 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 2 should be marked as critical
  • Step 5 should be marked as critical
  • Task standard is not written well. Just venting containment is not good enough.

Either state that and and all critical steps in the JPM completed satisfactorily or State it as Applicant vented containment IAW EOP Att 14 by opening valves M41F037, M41F036, and E61F057 and start one or both charcoal filter train fans.

E

  • Changed location of the JPM and decided to use the V-panel to simulate the fan banks S
  • Sat with Rev 2 as submitted in folder S6 D 2 B U
  • Label Title page 2015 S6
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • Do two or nine rods scram initially?
  • It is not clear on what is going on in the task standard. When depress the manual scram switch (step 5.4) the JPM std states that the standard Depress MAN SCRAM Switch and the std should include that two (?) control rods insert as expected (I assume the others are setup to not scram for some reason although this is not clear in the JPM).Next you verify lights then depress the SCRAM VLV (6C) pushbutton and when this occurs I believe this is where the alt path should begin and the std again is that two rods have scrammed but the notes state that nine rods have scrammed. Which is it?
  • Also would be good to have a reason for why before only two scram and now 9 more scram. Also, the task std doesnt match this either. It states that when the first scram switch is depressed, the nine rods scram
  • There are four total actions in this JPM and only one step in the alt path so not very good for the NRC exam
  • Where it states the applicant needs to go to the Control Rod/Drive Malfunction ONEP doe the applicant need to pull the procedure to take action or is it an immediate response to scram the reactor?
  • How long is acceptable to wait to scram?
  • If we keep it then you will need to fix all the issues with the JPM E
  • Number of Critical steps need changed to 3
  • Fix Task Standard to The applicant is to perform the Reactor Manual Scram Switch Test in accordance with 06-OP-1C71-W-0001, Reactor Manual Scram Switch Test Surveillance, determine that when the manual scram switch is depressed that 9 rods scram and manually scram the reactor per 05-1-02-IV-1, Control Rod/Drive Malfunctions ONEP by completing all of the critical steps satisfactorily.
  • Step 4 it needs to indicate that 9 rods scram not 4 0 Page 7 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link

  • Step 5 needs a note added to let the examiner know that this step may not be performed by the applicant.
  • Step 6 is not a critical step if the applicant realizes that more than 8 rods have scrammed they may elect to take the mode switch to shut down and end the JPM S
  • Sat with Rev 2 as submitted in folder S7 D 2 X B E
  • Label Title page 2015 S7
  • Need to provide the procedure that will be given to the applicant
  • Need to mark where the alternate path begins in the JPM
  • JPM step 1 What happens if the switch is not taken to TEST? Would this be a critical step to be able to accomplish the task?
  • There is cueing in a building operator will tell them where the leak is. You cannot cue the correct path to go down for the alternate path. No credit is given to the applicant for items that are cued
  • How long is it acceptable to take to isolate the leak? Needs included in task standard.
  • Step after step 12 should be step 13 not step 8
  • Task standard needs to be more detailed form a legal perspective E
  • Revise Task Standard to Applicant is to perform a manual start of SSW pump A in accordance with 04 01-P41-1/4.2, INSERT PROCEDURE NOUN NAME, Recognize that an isolable SSW leak has occurred during the performance of the evolution and isolate the leak by either closing F064A, INSERT NOUN NAME, or securing SSW pump A prior to basin level reaching 7.25 ft. by completing all critical steps satisfactorily.
  • On Reference Step 4.2.1 needs to be place marked circle/slash or whatever you do for place keeping at grand gulf. This will keep the JPM moving (I hope)

S

  • Sat with Rev 2 as submitted in folder S8 D 2 B E
  • Label Title page 2015 S8
  • Need to provide the procedure that will be given to the applicant
  • The task standard should be something like completes the ARM D21K601 functional test by performing all critical steps correctly in the JPM.
  • Step 20 of JPM What happens if they hold it long enough to get a high alarm? Is this a failure criteria?
  • Step 21 of JPM either the switch returns to auto or must be placed in auto make the task standard match what is actually needed
  • Step after step 25 should be step 26 not step 8
  • Minor Edit to make the JPM flow with little down time on unverifiable actions is to E

have in the cue that the prerequisites have been completed and on the handout have step 7.2.1a of attachment I already signed. 0 Page 8 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS CONTROL ROOM/IN-PLANT SYSTEMS JPMS

4. Job Content
1. 2. 3. Attributes 6.

Errors 5.

JPM# Dyn LOD Explanation IC Cues Critical Scope Over- Job- Minutia U/E/S (D/S) (1-5) (See below for instructions)

Focus Steps (N/B) lap Link S

  • Sat with Rev 2 as submitted in folder P1 S B E
  • Label Title page 2015 P1
  • Need to provide the procedure that will be given to the applicant
  • What happens if the applicant tries to reset an overcurrent really? The standard says that they should not do this.
  • Task Standard is not detailed enough Should say something like applicant will reset Bus 13AD and 12HE Bus Under voltage relays in accordance with ONEP 05-1-02-I-4 section 3.3.12e by completing all of the critical steps
  • In-plant JPM cues need to be correct for in-plant JPMs. You cant use some of the cues you have because they are not as required in the NUREG. You have to give them a cue as they would see it in the plant, not the actual position of the equipment as the cue For example, instead of saying the lockout relay is in the tripped position, you would have the cue state the the lockout relay flag is up or down or whatever they would actually see (examiner would need to use a pen or something like that to point to position changes, values on an analog meter, etc).
  • The JPM steps needs to include the procedure number and procedure step number that the applicant will be performing.
  • I am not sure that 15 minutes is a good validation time for this JPM we will see during validation week
  • Sat with Rev 2 as submitted in folder S

P2 S B E

  • Label Title page 2015 P2
  • Need to provide the procedure that will be given to the applicant
  • I do not understand how this is an EN JPM, Please explain
  • Task Standard is not up to legal standards as explained in other JPM comments
  • Sat with Rev 2 as submitted in folder S

P3 S N E

  • Label Title Page 2015 P3
  • Need to provide the procedure that will be given to the applicant
  • JPM was altered during validation to more accurately mirror actual method of completion.

S

  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in 0 Page 9 of 12 OBDI 202 - IOLE Process

reviewing operating tests. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.

1. Determine whether the task is dynamic (D) or static (S). A dynamic task is one that involves continuous monitoring and response to varying parameters. A static task is basically a system reconfiguration or realignment.
2. Determine level of difficulty (LOD) using established 1-5 rating scale. Levels 1 and 5 represent inappropriate (low or high) discriminatory level for the license being tested.
3. Check the appropriate box when an attribute weakness is identified:

$ The initiating cue is not sufficiently clear to ensure the operator understands the task and how to begin.

$ The JPM does not contain sufficient cues that are objective (not leading).

$ All critical steps (elements) have not been properly identified.

$ Scope of the task is either too narrow (N) or too broad (B).

$ Excessive overlap with other part of operating test or written examination.

4. Check the appropriate box when a job content error is identified:
  • Topics not linked to job content (e.g., disguised task, not required in real job).
  • Task is trivial and without safety significance.
5. Based on the reviewer=s judgment, is the JPM as written (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory?
6. Provide a brief description of any U or E rating in the explanation column.
7. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 10 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS Scenario 1. 2. 3. 4. 5. 6. 7. 8. 9.

10. Explanation (See below for instructions)

Set ES TS Crit IC Pred TL L/C Eff U/E/S 1 S No comments prior to validation E Validation Week Comments to incorporate

  • Added applicable tech spec entry to event 3
  • Added applicable tech spec condition to event 4
  • On D-1 corrected the order of events
  • Added bounding criteria to Critical Tasks
  • Changed Critical task 2 due to a change in event 8
  • Fixed simulator Setup information on page 2 of the D-2 from 7.7 to 7.5
  • Fixed turnover sheet to reflect step 7.5 as well
  • Added condition A to Tech Spec call in event 3 on D-2
  • Removed potential tech spec (tracking LCO) from event 3
  • Added condition A to LCO call for event 4
  • Added Critical Task bounding criteria to event 6 on D-2
  • Add direction to booth to bring completed attachments to the control room as would be done in the plant
  • Added critical Task bounding criterial to event 7
  • Made event 7 a NRC trigger S
  • Sat with Rev 2 as submitted in folder 2 S No Comments prior to validation week E Validation week comments to incorporate
  • Tech Specs on D-1 for events 1,2,4and 5 were incomplete
  • In Scenario Activities sections added appropriate Tech Specs and conditions to match D-2
  • Event 2 and 3 needs NRC trigger symbol
  • Event 4 is TR 3.1.5 condition A applicable?
  • Missing TS on event 5
  • Fix bullets on page 6
  • Fix typo on page 12 of D-2 for B RFPT
  • Add noun names on event 7 of D-2
  • Provide details for examiners on event 7 verification of ECCS systems
  • Add noun names for SRVs S
  • Sat with Rev 2 as submitted in folder 3 S No Comments prior to validations week E Validation week comments to incorporate 0 Page 11 of 12 OBDI 202 - IOLE Process

GG - 2015 - 12 DRAFT OPERATING TEST COMMENTS SCENARIOS

  • Page 1 od D-1 add the SBLOCA to event 9 as it is in the D-2
  • Page 2 D-1, add the other tech specs that are applicable 3.8.7 and 3.7.2
  • D-1 page 2, put TR in front of 3.3.2.1 on event 5
  • Change critical tasks 3 to 2 one of the critical tasks is not EOP based
  • D-2 page 5, list the alarm noun names
  • D-2 page 6, remove information to restore Fuel Pool Cleaning as the applicants will not have opportunity to perform
  • D-2 page 7, give the noun name for the alarm that is required to be addressed
  • D-2 page 10, add bounding criteria to Critical task
  • D-2 page 12, add the conditions for TRM 3.3.2.1
  • D-2 page 13, list that alarm that will cue applicant with noun name S
  • Sat with Rev 2 as submitted in folder Instructions for Completing Matrix This form is not contained in or required by NUREG-1021. Utilities are not required or encouraged to use it. The purpose of this form is to enhance regional consistency in reviewing operating test scenario sets. Additional information on these areas may be found in Examination Good Practices Appendix D. Check or mark any item(s) requiring comment and explain the issue in the space provided.
1. ES: ES-301 checklists 4, 5, & 6 satisfied.
2. TS: Set includes SRO TS actions for each SRO, with required actions explicitly detailed.
3. Crit: Each manipulation or evolution has explicit success criteria documented in Form ES-D-2.
4. IC: Out of service equipment and other initial conditions reasonably consistent between scenarios and not predictive of scenario events and actions.
5. Pred: Scenario sequence and other factors avoid predictability issues.
6. TL: Time line constructed, including event and process triggered conditions, such that scenario can run without routine examiner cuing.
7. L/C: Length and complexity for each scenario in the set is reasonable for the crew mix being examined, such that all applicants have reasonably similar exposure and events are needed for evaluation purposes.
8. Eff: Sequence of events is reasonably efficient for examination purposes, especially with respect to long delays or interactions.
9. Based on the reviewer=s judgment, rate the scenario set as (U)nacceptable (requiring repair or replacement), in need of (E)ditorial enhancement, or (S)atisfactory.
10. Provide a brief description of problem in the explanation column.
11. Save initial review comments as normal black text; indicate how comments were resolved using blue text so that each JPM used on the exam is reflected by a (S)atisfactory resolution on this form. 0 Page 12 of 12 OBDI 202 - IOLE Process