ML20210T633

From kanterella
Jump to navigation Jump to search
Submits Program Mgt Info Re Draft OSP Proceduce SA-102, Reviewing Common Performance Indicator 2,Technical Quality of Insps, (SP-99-021)
ML20210T633
Person / Time
Issue date: 03/30/1999
From: Lohaus P
NRC OFFICE OF STATE PROGRAMS (OSP)
To:
GENERAL, MINNESOTA, STATE OF, OHIO, STATE OF, OKLAHOMA, STATE OF, PENNSYLVANIA, COMMONWEALTH OF, WISCONSIN, STATE OF
References
SP-99-021, NUDOCS 9908190125
Download: ML20210T633 (32)
v • d • e


Text

)

1

.-AIAll 3 0lett

-e. +~

1W- O 4 L ALL AGREEMENT STATES MINNESOTA, OHlo, OKLAHOMA, PENNSYLVANIA, AND WISCONSIN

~ PROGRAM MANAGEMENT INFORMATION: DRAFT OSP PROCEDURE SA-102,

" REVIEWING COMMON PERFORMANCE INDICATOR #2, TECHNICAL QUALITY OF INSPECTIONS" (SP-99-021)

Enclosed for your review and comment is the draft OSP Procedure SA-102, " Reviewing Common Performance Indicator #2, Technical Quality of Inspections." This document has been drafted to incorporate procedures and guidance for the review conducted under the Integrated Materials Performance Evaluation Program. We would appreciate receiving your comments within one month of receipt of this letter. I This information request had been approved by OMB 3150-0029, expiration 04/30/01. The estimated burden per response to comply with this voluntary collection request is 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> (s).

Forward any comments regarding the burden estimate to the information and Records Management Branch (T-6F33), U.S. Nuclear Regulatory Commission, Washington DC 20555-

0001, and to the Paperwork Reduction Project (3150-0029), Office of Management and Budget, Washington, DC 20503. If a document does not display a curren.tly valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information.

If you have any questions regarding this correspondence, please contact me or the individual

. named below:

POINT OF CONTACT: Lance J. Rakovan INTERNET: LJR2ONRC. GOV

' TELEPHONE: (301) 415-2589 FAX: (301) 415-3502 ,

OriginalSigned By:

PAULH.LOHAUS Paul H. Lohaus, Director Office of State Programs

.d '

Enclosure:

1 As stated j Distribution:

DIR RF DCD (SP03) k SDroggitis PDR (NO_f)

- KSchneider Agreement State File IMPEP File W0003 DOCUMENT NAME: G:\lMPEP\SP99021.wp,d T* receive a cop r of this document, indicate in the box: "C" =/Odr$ ~ ettschment/ enclosure "E* = Copy with attachment / enclosure *N" a No copy OFFICE OSP /A IE OS#iA /I I I I 4 NAME- LJRakovan 7 PHLohdu@ l DATE 03/ vt /99~ 03/ Vl) /99 9908190125 990330 y

OSP FILE CODE: SP-A-4; SP-l-2 PDR STPRO E800EN @ ;;

PDR _

f

4 p. reg g $ UNITED STATES

- g j

t NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20555-0001

% # March 30, 1999 ALL AGREEMENT STATES MINNESOTA, OHIO, OKLAHOMA, PENNSYLVANIA, AND WISCONSIN PROGRAM MANAGEMENT INFORMATION: DRAFT OSP PROCEDURE SA-102,

" REVIEWING COMMON PERFORMANCE INDICATOR #2, TECHNICAL QUALITY OF INSPECTIONS" (SP-99-021)

Enclosed for your review and comment is the draft OSP Procedure SA-102, " Reviewing Common Performance Indicator #2, Technical Quality of inspections." This document has been drafted to incorporate procedures and guidance for the review conducted under the Integrated Materials Performance Evaluation Program. We would appreciate receiving your comments within one month of receipt of this letter.

This information request had been approved by OMB 3150-0029, expiration 04/30/01. The <

estimated burden per response to comply with this voluntary collection request is 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> (s).

Forward any comments regarding the burden estimate to the information and Records Management Branch (T-6F33), U.S. Nuclear Regulatory Commission, Washington DC 20555-0001, and to the Paperwork Reduction Project (3150-0029), Office of Management and Budget, Washington, DC 20503. If a document does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information.

I If you have any questions regarding this correspondence, please contact me or the individual named below:

POINT OF CONTACT: Lance J. Rakovan INTERNET: LJR2@NRC. GOV TELEPHONE: (301)415-2589 FAX: (301) 415-3502 6V N P ul . Lohaus, Director Office of State Programs

Enclosure:

As stated 1

c.

3/30m/3:53p /%g E

W??05Y OSP Procedure Approval Reviewing Common Perfonnance Indicator #2 Technical Quality of Inspections - SA-102 Issue Date:

Expiration Date:

Paul H. Lohaus Director, OSP Date:

Deputy Director, OSP Date:

Kathleen N. Schneider Procedure Contact, OSP Date:

l l

l NOTE The OSP Director's Secretary is responsiblefor the maintenance of this master copy document  ;

as part ofthe OSP Procedure Manual Any changes to theprocedure will be the responsibility of \

the OSP Procedure Contact. Copies of OSPprocedures will be distributedfor information.

  1. % Procedure

Title:

Page: 1 of 6

) Reviewing Common Performance IndicatorIssue #2, Date:

..,,. Technical Quality ofInspections Procedure Number: SA-102 I. INTRODUCTION This document describes the procedure for conducting reviews of NRC Regional Offices and Agreement States using Common Performance Indicator #2, Technical Quality of Inspections (NRC Management Directive (MD) 5.6, Integrated Materials Performance Evaluation Program (IMPEP)).

II. OBJECTIVES A. To ensure that inspection findings are well-founded and well-documented in reports describing the scope of each inspection, all violations and health and safety matters, the scope of each licensee's program, discussions with licensee management and each licensee's response.

B. To verify that inspections are complete and reviewed promptly by supervisors or management.

C. To determine that procedures are in place and used to help identify root causes and poor licensee performance.

D. To confirm that follow-up inspections address previously identified open items and/or past violations.

E. To verify that inspection findings lead to appropriate and prompt regulatory action.

F. To confirm that supervisors conduct annual accompaniments of each inspector to assess performance and assure application of appropriate and consistent policies and guides.

G. For Regions or States with separate licensing and inspection staffs, to verify that procedures are established and followed to provide feedback information to license reviewers.

H. For States, to determine that inspection guides are consistent with NRC guidance, and that they are being used consistently by inspectors to assure uniform and complete inspection practices.

Sku152iiReviewing Com'monPerfonnanceIndichtor#2/ 'Page:l2 of 6)% N TibhnicssQuality ofInissetlossW. , % SL4 i' : Issub Dateib , n IIL BACKGROUND This performance indicator provides a qualitative balance to Common Performance Indicator

  1. 1, Status of Materials Inspection Program, which looks at the status of an inspection program on a quantitative basis. Review team members will accompany a sample of inspectors at different types of licensed facilities to evaluate the knowledge and capabilities ofinspectors firsthand. Review team members will also conduct in-depth, onsite reviews of a cross section of completed inspection reports. These reviews will focus on the scope, completeness, and technical accuracy of completed inspections and related documentation.

IV. ROLES AND RESPONSIBILITIES A. Team Leader.

The team leader for the Regional or State review will determine which team member (s) is assigned lead review responsibility for this performance indicator. The principal reviewer should meet the appropriate requirements specified in MD 5.10, Formal QualificationsforIMPEP Team Members.

B. Principal Reviewer.

The principal reviewer is responsible for selecting license files / inspection reports to be reviewed, reviewing relevant documentation, conducting staff discussions, and maintaining a reference summary of all those reviewed.

V. GUIDANCE A. Scope.

1. This procedure applies only to the review (for adequacy, accuracy, completeness,  !

clarity, specificity, and consistency) of the technical quality of completed materials inspection actions taken by the Region or Agreement State in the period since the l last review. The principal reviewer for this indicator may, nonetheless, find it necessary to review earlier inspections to assure outstanding items have been addressed.

2. This procedure specifically excludes inspections of non-Atomic Energy Act materials or licensees, and inspections conducted by NRC Headquarters personnel.

S'Ah02i Rebliwing Cominon Perfonnance Indicator'#2, ;Page:Not6W TechNicalQuAlity ofInsOestibNM + l : ~ s-  ; Issue'Datei:  %

B. Evaluation Procedures.

1. The principal reviewer should refer to Pan III (Evaluation Criteria) of MD 5.6 for specific evaluation criteria. The Directive's Glossary defines the terms " Materials Inspection" and " Overdue Inspection."
2. All materials inspections conducted by Regions or Agreement States since the last I performance review are potential candidates for review. Inspections of license terminations, bankruptcies, and complex decommissioning will be treated as a subset of this common performance indicator.
3. Depending upon the size of the Regional or State program under review, the principal reviewer should select 10-25 inspection casework examples for review, concentrating on core licenses (i.e., initial inspection or Priority 1-3 as described in NRC Manual Chapter 2800, Materials Inspection Program). The selected casework should represent a cross-section of the Region or State's workload, including as many different inspectors, license categories, and geographic locations as practical. A mix of medical and academic use (universities, community hospitals, brachytherapy licenses, teletherapy licenses, physicians, broad scope facilities, etc.) and industrial use (research and development, radiography, irradiators, well logging, etc.) licenses should be sought. Inspections of complex decommissioning activities should also be sought.
4. If the initial review indicates a systematic weakness on the part of one inspector, or problems with respect to one or more inspection procedures, additional similar inspection files should be obtained and reviewed, in order to determine the magnitude of the programmatic weakness.
5. If the evaluation of the 10-25 casework examples does not reveal any programmatic weaknesses, no additional casework needs to be reviewed.
6. For the Regions, no attempt should be made to evaluate performance on a state-by-state basis for this indicator.

C. Review Guidelines.

1. The response generated by the Region or State to relevant questions in the IMPEP questionnaire should be used to focus the review.

l

2. For the Regions, tallies of completed inspections can be obtained from the Licensing Management System (LMS). This information can be obtained prior to the Regional visit from the Office of Nuclear Material Safety and Safeguards'

1 1

SA-102i Reviewing Commbn Performanie Indicator"#2,- Page:L4'of 6l w ,

TechnicalQuality ofInspectionsa > * +

Issue Datei Division ofIndustrial and Medical Nuclear Safety, Operations Branch. LMS has limited ability to sort these records, depending on the needs of the principal reviewer. Once the appropriate files are selected, a call to the Regional Office can

)

be made to have the inspection files pulled, and ready for review at the time of the visit.

3. For the States, inspection reports are not normally submitted to the Office of State  !

Programs. The principal reviewer should work with the IMPEP team leader in l selecting the appropriate inspection files for review.

i D. Review Details.

I For the technical quality ofinspections, the principal reviewer should evaluate the I following:

1. For each compliance action selected, that the inspection report adequately documents (as appropriate):
a. the scope of the inspection and the licensed program;
b. the licensee organization and the persons contacted;
c. the licensee's administrative controls and procedures; facilities and equipment; radiation safety procedures for procurement, use, transfer and disposal; posting and labeling; personnel monitoring, gaseous and liquid effluents, surveys and bioassay, incidents and overexposures, and radioactive waste packaging and shipping;
d. operations observed;
e. interviews of workers;
f. independent measurements;
g. status of previous violations;
h. new violations noted;
i. the exit interview with management;
j. the substance of discussions with licensee management;
k. licensee's response to any violations.
2. Any information missing from the file, e.g., documents, letters, file notes, and telephone conversations.
3. Inspection reports are sufficiently detailed to show that each inspection was i complete.

i

^

SA 102i Reyiewin[ Common Pe$$rman~ce Indicator?#2, Pagef5g,6?

Techn'i cal Quality ofInspections, '

Issue Date: &

4. All violations and safety recommendations are substantiated.
5. Appropriate regulatory action was taken for violations.
6. The documentation of violations is written in appropriate regulatory language and dispat:hed in a timely manner.
7. Any unresolved items or misunderstandings by the licensee were pursued to a satisfactory conclusion.
8. The inspection repon was reviewed by management.
9. Management notes report deficiencies (such as unsupported conclusions and opinions in the repon, violations not properly substantiated, apparent violations not cited, etc.) and brings these deficiencies to the attention of the inspector.
10. The licensee's response was reviewed for adequacy and any subsequent action taken by management.

I1. Instmmentation is adequate and functioning properly for surveying license operations (e.g., survey meters, air samples, lab counting equipment for smears, identification ofisotopes, etc.).

12. The effectiveness of the Region's or State's intemal program to evaluate its inspectors in the field. Regional or State supervisors should evaluate all inspectors on at least one inspection in the field per year.
13. Appendix A,"IMPEP Compliance File Review Guidance," was developed to assist in reviewing certain completed inspection reports. However, the principal reviewer should not feel compelled to address every item in the guidance.

E. Review Information Summary.

At a minimum, the summary maintained by the principal reviewer will include:

1. The licensee name, city, and state;
2. The license number;
3. The inspection priority;

~

S$102{ Reviewing Coninon s PerfoAn$sceindicator #2, 'PEge:f 6 of 6l wi Technical' Quality ofInspEctions l = W%S  % IssueDated ' N

4. The type of license operation (e.g., program code or license category); )

I

5. The inspector's initials, j I
6. The type of inspection (e.g., routine, reactive, closeout, announced, unannounced, team, other, etc.);
7. The date ofinspection;

{

8. The date inspection findings were issued.

F. Inspector Accompaniments / Field Evaluations.

1. In addition to performing a file review of the selected inspections, the principal reviewer for this indicator (or another qualified B1 PEP team member, as appropriate), should complete an appropriate number of accompaniments of the Region's or State's inspectors to observe, on a first-hand basis, the inspectors' demonstration of proper inspection techniques, and areas of emphasis.

Accompaniments should be performed prior to the BiPEP review at the Region or .

State. In accordance with the Region's or State's work schedules, the reviewer l should observe a representative sample ofinspectors and licensee types, concentrating on inspections of licensed facilities which have greater health and safety potential. One-day inspections are preferable for accompaniment so that the 1 reviewer may observe the entire inspection process from entrance to exit. l

2. In most cases, the goal for a State review is to accompany one-half of the program's inspectors. For larger States and NRC Regions, the goal is to accompany four or five inspectors. Priority should be given to newly qualified inspectors and those that have not been accompanied during previous BIPEP reviews.
3. BIPEP accompaniments are performance-based evaluations ofinspector effectiveness. It is important that these accompaniments focus on health and safety type issues. It is not the role of the reviewer to help with the inspection effon, but rather to observe the inspector's work.
4. Prior to the inspection, the reviewer and inspector should discuss:
a. the extent of the reviewer's participation in the inspection (observation not active panicipation);
b. the way the reviewer's presence will be explained to the licensee; and
c. the method that will be used in evaluating the inspector's performance. l

SA-102VReviewing Common Perfonnance Indicator #2, . ,Pkgen7 of 6L a Techssical Quality bfInspeitionsJ * '

i- '

~ Issue'Date:i ~ % a.

5. The evaluation should be discussed with the inspector's supervisor within one week of the accompaniment.
6. Appendix B,"IMPEP Inspector Fieldwork Evaluation Reviewer Guidance," was developed to assist the reviewer in completing the inspection accompaniments.

The reviewer should not feel compelled to address every item on the evaluation form. Accompaniment information should be sununarized as discussed in Section E,above.

VI. APPENDICES A. IMPEP Compliance File Reviewer Guidance.

B. IMPEP Inspector Fieldwork Evaluation Reviewer Guidance.

VII. REFERENCES

1. NRC Management Directive 5.6, Integrated Materials Performance Evaluation Program.
2. NRC Management Directive 5.10, Formal Qualificationsfor IMPEP Team Members.
3. NRC Inspection Manual Chapter 2800, Materials Inspection Program.

APPENDIX A iMPEP COMPLIANCE FILE REVIEWER CUIDANCE

. A/S OR REGION:

FILE #

LICENSEE: LICENSE #

LOCATION: LICENSE TYPE:

INSPECTION DATE: PRIORITY :

ANNOUNCED D UNANNOUNCED D COMPLETE D PARTIAL 0 ROUTINE O INITIAL 0 IR: OFFICE O FIELD D FOLLOW-UP O SPECIAL 0 RECIPROCITY O INSPECTION CONDUCTED WITHiN 25% OF SCHEDULED FREQUENCY 7 Y N N/A NO. COMMENTS FOR REPORT i

INSPECTOR: OFFICE:

SUPERVISORY REVIEW BY: DATE:

IMPEP REVIEW BY: DATE:

FINDINGS DISCUSSED WITH ON:

IMPEP COMPLIANCE FILE REVIEWER CUIDANCE PAGE 2

[ ITEM O.K. COMMENTS OR QUESTIONS

' ACTION DATES:

PREVIOUS INSPECTION:

INSPECTION DATE:

ENFORCEMENT LETTER: SHORT FORM O LICENSEE RESPONSE:

FOLLOW-UP:

ACKNOWLEDGMENT LETTER:

CLOSE-OUT:

DOCUMENTED EVIDENCE OF:

CLOSEOUT OF PREVIOUS VIOLATIONS REVIEW & CLOSEOUT OF PREVIOUS INCIDENTS EXIT MEETING ATTENDEES & TITLES SUBSTANCE OF OISCUSSIONS OBSERVED OPERATIONS WORKER / USER INTERVIEWS ANCILLARY WORKER INTERVIEWS INDEPENDENT MEASUREMENTS REPORT DOCUMENTS REVIEW OF:

LICENSE EXPIRATION DATE OR RENEWAL STATUS CONDITION. LOCATION OF FACILITIES & EQUIPMENT ALARA PROGRAM, ACTION LEVELS. INTERNAL AUDITS OPERATING PROCEDURES MANAGEMENT, ORGANIZATION, RSO, RSC, USERS EMERGENCY PLAN OR PROCEDURES INCIDENT FILE TRAINING PROGRAM - USERS & ANCILLARY WORKERS INSTRUMENTS, CAllBRATION POSTING, LABELING, REGULATIONS SECURITY PROCUREMENT, RECElPT, INVENTORY USE, TRANSFER, SHIPPING MONITORING & SURVEY PROGRAM RSC MINUTES, COMMITTEE COMPOSITION DOSIMETRY & BIOASSAY RECORDS LEAK TESTS, MAINTENANCE OA, OC GAS & LIQUID EFFLUENT RECORDS WASTE DISPOSAL

..e IMPEP COMPLIANCE FILE REVIEWER GUIDANCE PAGE 3

. USE OF FIELD OR TEMP JOB SITES AS APPROVED e

ffEM O.K. COMMENTS OR QUESTIONS INSPECTION FINDINGS '

CONDUCTED IN SUFFICIENT DEPTH & SCOPE REPORT COMPLETE AND IN STANDARD FORMAT REPORT CLEARLY IDENTIFIED VIOLATIONS VS RECS EXIT MEETING AT APPROPRIATE MANAGEMENT LEVEL FINDINGS INDICATIVE OF NEED FOR LICENSE CHANGES RELAYED TO LICENSING STAFF (VERIFY IN FILE)

ENFORCEMENT VIOLATIONS PROPERLY CITED REPEATED VIOLATIONS TAKEN INTO ACCOUNT LETTER CLEARLY IDENTIFIED VIOLATIONS VS RECS PROPER REGULATORY LANGUAGE IN LETTERS SUITABLE FOLLOW-UP TO LICENSEE *S RESPONSE ENFORCEMENT ACTION APPROPRIATE COMPLIANCE FILE FILE ORDERLY AND COMPLETE INCIDENT & COMPLIANCE FILES CROSS-REFERENCED ADEQUATE SUPERVISORY REVIEW OF REPORTS, LETTERS AND LICENSEE RESPONSES SUPERVISORY REVIEW ALL DEFICIENCIES IDENTIFIED BY SUPERVISOR I

COMMENTS FOR DISCUSSION WITH STAFF i

)

i

APPENDIX B IMPEP INSPECTOR FIELDWORK EVALUATl!N REVIEWER CUIDANCE e

o A/S OR REGION: DATE:

INSPECTOR: NRC REVIEWER:

LICENSEE: LICENSE NO:

LOCATION: INSPECTION TYPE:

LICENSE TYPE: ANNOUNCED D UNANNOUNCED D i

PRELIMINARY DISCUSSION WITH INSPECTOR DONE l A. EXPLAIN THE EXTENT OF THE REVIEWER'S PARTICIPATION IN INSPECTION. O B. DISCUSS PROCEDURE FOR INTRODUCING REVIEWER TO LICENSEE AND EXPLAINING HIS/HER PART IN iNCIE0 TION. O C. EXPLAIN METHOD TO BE USED IN EVALUATING INSPECTOR'S PERFORMANCE. O

SUMMARY

OF EVALUATION A. INSPECTOR'S PERFORMANCE RATING: MEETS OR EXCEEDS GUIDELINES O NEEDS IMPROVEMENT D B. COMMENTS: )

1 C. THE INSPECTOR WOULD BENEFIT FROM ADDITIONAL TRAINING IN j 1

D. EVALUATION DISCUSSED WITH ON (SUPERVISOR) (OAT E.)

ITEM O.K. COMMENTS OR OUESTIONS INSPECTOR'S PREPARATION ADEQUATE REVIEW OF LICENSE AND COMPLIANCE HISTORY INSPECTION PLAN OR FIELD FORM APPROPRIATE SURVEY INSTRUMENTS CAllBRATED D INSTRUMENT RESPONSE CHECK O SUPPLEMENTAL MATERIALS: REGS O FORMS O ID D nneturtnv n entincre n AurunurTrn e

, IMPEP FIELDWORK EVALUATION REVIEWER GUIDANCE PAGE 2 i

.. o ITEM O.K. COMMENTS OR QUESTIONS e-OPENING '

INTERVIEW CONDUCTED AT APPROPRIATE LEVEL EXPLANATION OF INSPECTION PURPOSE, SCOPE, METHOD INSPECTION ~

USE OF APPROPRIATE FORM OR CHECKLIST

" WALK THROUGH" AT BEGINNING OF INSPECTION OBSERVATION OF OPERATION AND HANDLING OF RAM FACILITIES CHECKED FOR PROPER POSTING, LABELING SECURITY VERIFIED WORKERS CHECKED FOR PERSONAL DOSIMETRY WORKER INTERVIEWS RAM USERS O ANCILLARY WORKERS O WIPES, SURVEYS, MEASUREMENTS TAKEN ADHFRENCE TO ALARA EVALUATED REVIEW OF INCIDENTS, OVEREXPOSURES, ETC.

RECORDS VERIFIED AGAINST ORAL STATEMENTS FOR:

PROCUREMENT & INVENTORY D RECEIPT & TRANSFER OF MATERIAL D INTERNAL AUDITS O SURVEYS & MONITORING D PERSONNEL DOSIMETRY, BIOASSAY D QUALIFICATION AND TRAINING OF PERSONNEL D EMERGENCY PLAN & PROCEDURES O COMMITTEE MEETINGS, MINUTES O AUTHORIZED USERS D INSTRUMENT CAllBRATION O DOSE CALIBRATOR TESTS; UTILIZATION LOG D LEAK TESTS O GENERATOR ASSAY, MOLY BREAKTHROUGH, LOGS O WASTE MANAGEMENT, DISPOSAL D RELEASE OF AIR & SEWER EFFLUENTS D OA & OC: MAINTENANCE D INSPECTION CONDUCTED IN SUFFICIENT SCOPE & DEPTH VERIFICATION OF CORRECTIONS TO PREVIOUS VIOLATIONS INSPECTOR'S PROFESSIONALISM USE OF PROPER HEALTH PHYSICS TECHNIQUES (SELF MONITORING, ETC.)

ACCURATE EVALUATION OF RADIATION SAFETY KNOWLEDGE OF HEALTH PHYSICS & REGS APPROPRIATE APPEARANCE FOR LICENSE TYPE SKILL IN WORDING OUESTIONS SUITABLE RAPPORT WITH MANAGEMENT AND WORKERS

  1. IMPEP FIELDWORK EVALUATION REVIEWER GUIDANCE PAGE 3
  • ITEM O.K.

,_ COMMENTS OR QUESTIONS CLOmNG PREPARATION FOR EXIT INTERVIEW: ASSEMBLY OF SUPPORTING MATERIAL EXIT CONDUCTED AT APPROPRIATE MANAGEMENT LEVEL VIO FULLY EXPLAINED: LICENSE CONDITION OR REG CITED RECOMMENDATIONS CLEARLY DISTINGUISHED FROM VIO IMPENDING ENFORCEMENT ACTIONS EXPLAINED LICENSEE ADVISED OF EXPECTED RESPONSE AND REQUIREMENTS FOR CHANGE VIOLATIONS 0.K.

ADDITIONAL COMMENTS

AIAR 3 01989

. V: 7_,,

w 3. a ALL AGREEMENT STATES MINNESOTA, OHIO, OKLAHOMA, PENNSYLVANIA, AND WISCONSIN PROGRAM MANAGEMENT INFORMATION: DRAFT OSP PROCEDURE SA-102,

" REVIEWING COMMON PERFORMANCE INDICATOR #2, TECHNICAL QUALITY OF INSPECTIONS" (SP-99-921)

Enclosed for your review and comment is the draft OSP Procedure SA-102, " Reviewing Common Performance Indicator #2, Technical Quality of Inspections." This document has been drafted to incorporate procedures and guidance for the review conducted under the Integrated Materials Performance Evaluation Program. We would appreciate receiving your comments within one month of receipt of this letter.

This information request had been approved by OMB 3150-0029, expiration 04/30/01. The estimated burden per response to comply with this voluntary co!!ection request is 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> (s).

Forward any comments regarding the burden estimate to the information and Records Management Branch (T-6F33), U.S. Nuclear Regulatory Commission, Washington DC 20555-0001, and to the Paperwork Reduction Project (3150-0029), Office of Management and Budget, Washington, DC 20503. If a document does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information.

If you have any questions regarding this correspondence, please contact me or the individual named below:

POINT OF CONTACT: Lance J. Rakovan INTERNET: LJR2@NRC. GOV TELEPHONE: (301) 415-2589 FAX: (301) 415-3502 OriginalSigned By:

PAUL H.LCHAUS Paul H. Lohaus, Director Office of State Programs

Enclosure:

As stated Distribution:

DIR RF DCD (SP03)

SDroggitis PDR (NOf_)

KSchneider Agreement State File IMPEP File DOCUMENT NAME: G:\lMPEP\SP99021.wp,d /

w iv. . ..n . e,i. eoeuen.nt, inee.i. in th. i.or c -corme,out atteenrnene.nciosur. r- copy witn an enrnonvencio ur. r - No copy OFFICE OSP /J. lC OER;pU I NAME LJRakovan ~f PHLohaude DATE 03/:i /99 03/ W /99 OSP FILE CODE: SP-A-4; SP-l-2

ys *,

g UNITED STATES

. s j 2

NUCLEAR REGULATORY COMMISSION WASHINGTON, D.C. 20666-0001

% # March 30, 1999 ALL AGREEMENT STATES MINNESOTA, OHIO, OKLAHOMA, PENNSYLVANIA, AND WISCONSIN PROGRAM MANAGEMENT INFORMATION: DRAFT OSP PROCEDURE SA-102,

" REVIEWING COMMON PERFORMANCE INDICATOR #2, TECHNICAL QUALITY OF INSPECTIONS"(SP-99 021)

Enclosed for your review and comment is the draft OSP Procedure SA-102, " Reviewing Common Performance Indicator #2, Technical Quality of Inspections." This document has been drafted to incorporate procedures and guidance for the review conducted under the Integrated Materials Performance Evaluation Program. We would appreciate receiving your comments within one month of receipt of this letter.

This information request had been approved by OMB 3150-0029, expiration 04/30/01. The estimated burden per response to comply with this voluntary collection request is 6 hour6.944444e-5 days <br />0.00167 hours <br />9.920635e-6 weeks <br />2.283e-6 months <br /> (s).

Forward any comments regarding the burden estimate to the information and Records Management Branch (T-6F33), U.S. Nuclear Regulatory Commission, Washington DC 20555-0001, and to the Paperwork Reduction Project (3150-0029), Office of Management and Budget, Washington, DC 20503. If a document does not display a currently valid OMB control number, the NRC may not conduct or sponsor, and a person is not required to respond to, a collection of information.

If you have any questions regarding this correspondence, please contact me or the individual named below:

POINT OF CONTACT: Lance J. Rakovan INTERNET: LJR2@NRC. GOV TELEPHONE: (301) 415-2589 FAX: (301) 415-3502 f

()AV (\

Paul'H. Lohaus, Director hN Office of State Programs

Enclosure:

As stated

3/30/99/3:53p [

(

D3 APT OSP Procedure Approval Reviewing Common Perfonnance Indicator #2 Technical Quality Of Inspections - SA-102 Issue Date:

Expiration Date:

Paul H. Lohaus Director, OSP Date:

Deputy Director, OSP Date:

Kathleen N. Schneider Procedure Contact, OSP Date:

NOTE The OSP Director's Secretary is responsiblefor the maintenance ofthis master copy document aspart ofthe OSP Procedure Manual Any changes to th eprocedure willbe the responsibility of the OSP Procedure Contact. Copies of OSPprocedures will be distributedfor information.

l

  1. h Procedure

Title:

Page: 1 of 6

\ ,, ..) Reviewing TechnicalQuality Common ofInspections Performance Issue Indicator Date: #2, Procedure Number: SA-102

1. INTRODUCTION This document describes the procedure for conducting reviews of NRC Regional Offices and Agreement States using Common Performance Indicator #2, Technical Quality of Inspections (NRC Management Directive (MD) 5.6, Integrated Materials Perfonnance Evaluation Program (IMPEP)).

II. OBJECTIVES l

A. To ensure that inspection findings are well-founded and well-documented in reports l describing the scope of each inspection, all violations and health and safety matters, the scope of each licensee's program, discussions with licensee management and each licensee's response.

B. To verify that inspections are complete and reviewed promptly by supervisors or management.

C. To determine that procedures are in piece and used to help identify root causes and poor licensee performance. l l

D. To confirm that follow-up inspections address previously identified open items and/or past violations.

1 E. To verify that inspection findings lead to appropriate and prompt regulatory action.

F. To confirm that supervisors conduct annual accompaniments of each inspector to assess performance and assure application of appropriate and consistent policies and guides.

G. For Regions or States with separate licensing and inspection staffs, to verify that procedures are established and followed to provide feedback information to license reviewers.

H. For States, to determine that inspection guides are consistent with NRC guidance, and that they are being used consistently by inspectors to assure uniform and complete inspection practices.

SA-102: Reviewing Common Performanceindicator #2, Page::2 of 6 ;

TechnicalQualityofInspections% N n '

Issue Date:~T

~

III. B A C K G R O U N D This performance indicator provides a qualitative balance to Common Performance Indicator

  1. 1, Status of Materials Inspection Program, which looks at the status of an inspection program on a quantitative basis. Review team members will accompany a sample of inspectors at different types oflicensed facilities to evaluate the knowledge and capabilities ofinspectors firsthand. Review team members will also conduct in-depth, onsite reviews of a cross section of completed inspection repons. These reviews will focus on the scope, completeness, and technical accuracy of completed inspections and related documentation.

1 IV. ROLES AND RESPONSIBILITIES j A. Team Leader.

The team leader for the Regional or State review will determine which team member (s) is assigned lead review responsibility for this performance indicator. The principal reviewer should meet the appropriate requirements specified in MD 5.10, Formal QualificationsforIMPEP Team Members.

B. Principal Reviewer.

The principal reviewer is responsible for selecting license files / inspection reports to be reviewed, reviewing relevant documentation, conducting staff discussions, and I maintaining a reference summary of all those reviewed.

V. GUIDANCE A. Scope.

1. This procedure applies only to the review (for adequacy, accuracy, completeness, clarity, specificity, and consistency) of the technical quality of completed materials  !

inspection actions taken by the Region or Agreement State in the period since the last review. The principal reviewer for this indicator may, nonetheless, find it necessary to review earlierinspections to assure outstanding items have been addressed.

2. This procedure specifically excludes inspections of non-Atomic Energy Act materials or licensees, and inspections conducted by NRC Headquarters personnel.

l

^

SA-102: Reviewing Common Performance Indicator #2; Page: 3 of 6l, Technical Quality ofInspection.& ' T - S ' ^= - Issue ~Datei i '

B. Evaluation Procedures.

1. The principal reviewer should refer to Pan III (Evaluation Criteria) of MD 5.6 for specific evaluation criteria. The Directive's Glossary defines the terms " Materials Inspection" and " Overdue Inspection."
2. All materials inspections conducted by Regions or Agreement States since the last performance review are potential candidates for review. Inspections of license terminations, bankruptcies, and complex decommissioning will be treated as a subset of this common perfonnance indicator.
3. Depending upon the size of the Regional or State program under review, the principal reviewer should select 10-25 inspection casework examples for review, concentrating on core licenses (i.e., initial inspection or Priority 1-3 as described in NRC Manual Chapter 2800, Materials Inspection Program). The selected casework should represent a cross-section of the Region or State's workload, including as many different inspectors, license categories, and geographic locations as practical. A mix of medical and academic use (universities, community hospitals, brachytherapy licenses, teletherapy licenses, physicians, broad scope facilities, etc.) and industrial use (research and development, radiography, irradiators, well logging, etc.) licenses should be sought. Inspections of complex decommissioning activities should also be sought.
4. If the initial review indicates a systematic weakness on the part of one inspector, or problems with respect to one or more inspection procedures, additional similar inspection files should be obtained and reviewed, in order to determine the magnitude of the programmatic weakness.
5. If the evaluation of the 10-25 casework examples does not reveal any programmatic weaknesses, no additional casework needs to be reviewed.
6. For the Regions, no attempt should be made to evaluate performance on a state-by-state basis for this indicator.

C. Review Guidelines.

1. The response generated by the Region or State to relevant questions in the IMPEP questionnaire should be used to focus the review.
2. For the Regions, tallies of completed inspections can be obtained from the Licensing Management System (LMS). This information can be obtained prior to the Regional visit from the Office of Nuclear Material Safety and Safeguards'

^

SA-102: Reviewing Common Performance Indicator "

  1. 2,~' Page:L4 of 6 : -

TecNnical Quality ofInspbetionsi W " '

IssueDate? . ~

Division ofIndustrial and Medical Nuclear Safety, Operations Branch. LMS has limited ability to sort these records, depending on the needs of the principal reviewer. Once the appropriate files are selected, a call to the Regional Office can be made to have the inspection files pulled, and ready for review at the time of the visit.

3. For the States, inspection repons are not normally submitted to the Office of State Programs. The principal reviewer should work with the IMPEP team leader in 1 selecting the appropriate inspection files for review.

D. Review Details.

For the technical quality of inspections, the principal reviewer should evaluate the following:

1. For each compliance action selected, that the inspection report adequately documents (as appropriate):
a. the scope of the inspection and the licensed program;
b. the licensee organization and the persons contacted;
c. the licensee's administrative controls and procedures; facilities and equipment; radiation safety procedures for procurement, use, transfer and disposal; posting and labeling; personnel monitoring, gaseous and liquid effluents, surveys and bioassay, incidents and overexposures, and radioactive waste packaging and shipping;
d. operations observed;
e. interviews of workers;
f. independent measurements;
g. status of previous violations; I
h. new violations noted;
i. the exit interview with management;

)

I

j. the substance of discussions with licensee management;
k. licensee's response to any violations.

l

2. Any information missing from the file, e.g., documents, letters, file notes, and telephone conversations.
3. Inspection repons are sufficiently detailed to show that each inspection was complete. I l

l

SA-102: Reviewing Common Performance Indicator #2, Page::5 of 6 -

W

Technical Quality ofInspections0 '

Isstie Dkte: - ' ~

4. All violations and safety recommendations are substantiated.
5. Appropriate regulatory action was taken for violations.
6. The documentation of violations is written in appropriate regulatory language and dispatched in a timely manner.
7. Any unresolved items or misunderstandings by the licensee were pursued to a satisfactory conclusion.
8. The inspection report was reviewed by management.

{

l

9. Management notes report deficiencies (such as unsupported conclusions and opinions in the repon, violations not properly substantiated, apparent violations not cited, etc.) and brings these deficiencies to the attention of the inspector.
10. The licensee's response was reviewed for adequacy and any subsequent action taken by management.

I1. Instrumentation is adequate and functioning properly for surveying license operations (e.g., survey meters, air samples, lab counting equipment for smears, identification ofisotopes, etc.).

12. The effectiveness of the Region's or State's internal program to evaluate its  ;

inspectors in the field. Regional or State supervisors should evaluate all inspectors l on at least one inspection in the field per year. I l

i

13. Appendix A,"IMPEP Compliance File Review Guidance," was developed to assist in reviewing cenain completed inspection repons. However, the principal reviewer should not feel compelled to address every item in the guidance.

l E. Review Information Summary. l l

At a minimum, the summary maintained by the principal reviewer will include:

1. The licensee name, city, and state;
2. The license number;
3. The inspection priority;

SA-102: Reviewing Common Performance Indicator #2, Page: 6 oi6/ 6 Technical Quality ofInspections -

.: -@ Issue Date':T

4. The type oflicense operation (e.g., program code or license category);
5. The inspector's initials;
6. The type of inspection (e.g., routine, reactive, closcout, announced, unannounced, team, other, etc.); '
7. The date ofinspection;
8. The date inspection findings were issued.

F. Inspector Accompaniments / Field Evaluations.

1. In addition to performing a file review of the selected inspections, the principal reviewer for this indicator (or another qualified IMPEP team member, as appropriate), should complete an appropriate number of accompaniments of the Region's or State's inspectors to observe, on a first-hand basis, the inspectors' demonstration of proper inspection techniques, and areas of emphasis.

Accompaniments should be performed prior to the IMPEP review at the Region or State. In accordance with the Region's or State's work schedules, the reviewer should observe a representative sample ofinspectors and licensee types, concentrating on inspections oflicensed facilities which have greater health and safety potential. One-day inspections are preferable for accompaniment so that the reviewer may observe the entire inspection process from entrance to exit.

2. In most cases, the goal for a State review is to accompany one-half of the program's inspectors. For larger States and NRC Regions, the goal is to accompany four or five inspectors. Priority should be given to newly qualified inspectors and those that have not been accompanied during previous IMPEP reviews.
3. IMPEP accompaniments are performance-based evaluations ofinspector l effectiveness. It is important that these accompaniments focus on health and safety type issues. It is not the role of the reviewer to help with the inspection effort, but rather to observe the inspector's work.
4. Prior to the inspection, the reviewer and inspector should discuss:
a. the extent of the reviewer's participation in the inspection (observation not active participation);

b, the way the reviewer's presence will be explained to the licensee; and

c. the method that will be used in evaluating the inspector's performance.

SA-102: Reviewing Common Perfonnance Indicator #2,

.PageL7 of 6:. "

TechnicalQu'ality ofInspeitiohiC ^ % . 6 W Issue Date? ~? O ~

5. The evaluation should be discussed with the inspector's supervisor within one week of the accompaniment.
6. Appendix B,"IMPEP Inspector Fieldwork Evaluation Reviewer Guidance," was developed to assist the reviewer in completing the inspection accompaniments.

The reviewer should not feel compelled to address every item on the evaluation form. Accompaniment information should be summarized as discussed in Section E,above.

VI. APPENDICES A. IMPEP Compliance File Reviewer Guidance.

B. IMPEP Inspector Fieldwork Evaluation Reviewer Guidance.

VII. REFERENCES

1. NRC Management Directive 5.6, Integrated Materials Performance Evaluation Program.
2. NRC Management Directive 5.10, Fonnal Quahficationsfor IMPEP Team Members.
3. NRC Inspection Manual Chapter 2800, Materials inspection Program.

t

~

APPENDIX A iMPEP C*MPLIANCE FILE REVIEWER CulDANCE

,' A/S OR REGION:

FILE #

LICENSEE: LICENSE #

LOCATION: LICENSE TYPE:

INSPECTION DATE: PRIORITY :

ANNOUNCED D UNANNOUNCED D COMPLETE D PARTIAL O ROUTINE D INITIAL 0 IR: OFFICE D FIELD D FOLLOW UP O SPECIAL 0 RECIPROCITY O INSPECTION CONDUCTED WITHIN 25% OF SCHEDULED FREQUENCY? Y N N/A i

l NO. COMMENTS FOR REPORT l

l i

l l

l l

1 INSPECTOR: OFFICE:

SUPERVISORY REVIEW BY: DATE: l lMPEP REVIEW BY: DATE:

FINDINGS DISCUSSED WITH ON:

IMPEP COMPLtANCE FILE REVIEWER CUIDANCE PAGE 2

,' ITEM O.K. COMMENTS OR QUESTIONS ACTION DATES:

PREVIOUS INSPECTION:

INSPECTION DATE:

ENFORCEMENT LETTER: SHORT FORM D LICENSEE RESPONSE:

FOLLOW-UP:

ACKNOWLEDGMENT LETTER:

CLOSE-OUT:

DOCUMENTED EVIDENCE OF:

CLOSEOUT OF PREVIOUS VIOLATIONS REVIEW & CLOSEOUT OF PREVIOUS INCIDENTS EXIT MEETING ATTENDEES & TITLES SUBSTANCE OF DISCUSSIONS OBSERVED OPERATIONS WORKER / USER INTERVIEWS ANCILLARY WORKER INTERVIEWS  !

INDEPENDENT MEASUREMENTS REPORT DOCUMENTS REVIEW OF:

LICENSE EXPIRATION DATE OR RENEWAL STATUS CONDITION, LOCATION OF FACILITIES & EQUIPMENT ALARA PROGRAM, ACTION LEVELS. INTERNAL AUDITS OPERATING PROCEDURES MANAGEMENT, ORGANIZATION, RSO, RSC, USERS EMERGENCY PLAN OR PROCEDURES INCIDENT FILE TRAIN!NG PROGRAM . USERS & ANCILLARY WORKERS l INSTRUMENTS, CAllBRATION POSTING, LABELING, REGULATIONS SECURITY PROCUREMENT, RECEIPT, INVENTORY USE, TRANSFER SHIPPING MONITORING & SURVEY PROGRAM RSC MINUTES. COMMITTEE COMPOSITION DOSIMETRY & BIOASSAY RECORDS LEAK TESTS. MAINTENANCE. QA, QC GAS & LIOUID EFFLUENT RECORDS -

WASTE DISPOSAL

IMPEP COMPLIANCE FILE REVIEWER CUIDANCE PAGE 3 USE OF FIELD OR TEMP JOB SITES AS APPROVED --

ITEM O.K. .Cg']* 8 OR QUESTIONS INSPECTION FINDINGS CONDUCTED IN SUFFICIENT DEPTH & SCOPE REPORT COMPLETE AND IN STANDARD FORMAT REPORT CLEARLY IDENTIFIED VIOLATIONS VS RECS EXIT MEETING AT APPROPRIATE MANAGEMENT LEVEL FINDINGS INDICATIVE OF NEED FOR LICENSE CHANGES RELAYED TO LICENSING STAFF (VERIFY IN FILE)

ENFORCEMENT VIOLATIONS PROPERLY CITED REPEATED VIOLATIONS TAKEN INTO ACCOUNT LETTER CLEARLY IDENTIFIED VIOLATIONS VS RECS ,

PROPER REGULATORY LANGUAGE IN LETTERS SUITABLE FOLLOW-UP TO LICENSEE'S RESPONSE ENFORCEMENT ACTION APPROPRIATE COMPLIANCE FILE FILE ORDERLY AND COMPLETE INCIDENT & COMPLIANCE FILES CROSS-REFERENCED ADEQUATE SUPERVISORY REVIEW OF REPORTS, LETTERS AND LICENSEE RESPONSES SUPERVISORY REVIEW ALL DEFICIENCIES IDENTIFIED BY SUPERVISOR I

COMMENTS FOR DISCUSSION WITH STAFF 9

APPENDIX B IMPEP INSPECTOR FIELDWORK EVALUATION REVIEWER GUIDANCE 9

A/S OR REGION: DATE:

INSPECTOR: NRC REVIEWER:

LICENSEE: LICENSE NO:

LOCATION: INSPECTION TYPE:

LICENSE TYPE: ANNOUNCEDD UNANNOUNCED D PRELIMINARY DISCUSSION WITH INSPECTOR DONE A. EXPLAIN THE EXTENT OF THE REVIEWER'S PARTICIPATION IN INSPECTION. O B. DISCUSS PROCEDURE FOR INTRODUCING REVIEWER TO LICENSEE AND EXPLAINING HIS/HER PART IN INSPECTION. O C, EXPLAIN METHOD TO BE USED IN EVALUATING INSPECTOR'S PERFORMANCE. O

SUMMARY

OF EVALUATION A. INSPECTOR'S PERFORMANCE RATING: MEETS OR EXCEEDS GUIDELINES D NEEDS IMPROVEMENT D B. COMMENTS:

C. THE INSPECTOR WOULD BENEFIT FROM ADDITIONAL TRAINING IN D. EVALUATION DISCUSSED WITH ON (SUPERvlSOR) (DATEI ITEM O.K. COMMENTS OR OUESTIONS INSPECTOR'S PREPARATION ADEQUATE REVIEW OF LICENSE AND COMPLIANCE HISTORY INSPECTION PLAN OR FIELD FORM APPROPRIATE SURVEY INSTRUMENTS CAllBRATED D INSTRUMENT RESPONSE CHECK D SUPPLEMENTAL MATERIALS: REGS O FORMS D ID D nneiurTav n entinere m AurunurTrn m l

l

)

,I IMPEP FIELDWORK EVALUATION REVIEWER GUIDANCE PAGE 2

, ITEM O.K. COMMENTS OR QUESTIONS OPENING INTERVIEW CONDUCTED AT APPROPRIATE LEVEL EXPLANATION OF INSPECTION PURPOSE, SCOPE, METHOD INSPECTION USE OF APPROPRIATE FORM OR CHECKLtST

" WALK THROUGH* AT BEGINNING OF INSPECTION OBSERVATION OF OPERATION AND HANDLING OF RAM FACILITIES CHECKED FOR PROPER POSTING, LABELING SECURITY VERIFIED WORKERS CHECKED FOR PERSONAL DOSIMETRY WORKER INTERVIEWS RAM USERS D ANCILLARY WORKERS D WIPES, SURVEYS, MEASUREMENTS TAKEN ADHERENCE TO ALARA EVALUATED REVIEW OF INCIDENTS, OVEREXPOSURES. ETC.

RECORDS VERIFIED AGAINST ORAL STATEMENTS FOR:

PROCUREMENT & INVENTORY D RECEIPT & TRANSFER OF MATERIAL D INTERNAL AUDITS O SURVEYS & MONITORING D PERSONNEL DOSIMETRY, BIDASSAY D QUALIFICATION AND TRAINING OF PERSONNEL D EMERGENCY PLAN & PROCEDURES D COMMITTEE MEETINGS, MINUTES O AUTHORIZED USERS O INSTRUMENT CALIBRATION D DOSE CALIBRATOR TESTS: UTILIZATION LOG D LEAK TESTS D GENERATOR. ASSAY, MOLY BREAKTHROUGH LOGS D WASTE MANAGEMENT, DISPOSAL D RELEASE OF AIR & SEWER EFFLUENTS D OA & OC; MAINTENANCE D INSPECTION CONDUCTED IN SUFFICIENT SCOPE & DEPTH VERIFICATION OF CORRECTIONS TO PREVIOUS VIOLATIONS INSPECTOR'S PROFESSIONALISM USE OF PROPER HEALTH PHYSICS TECHNIQUES (SELF MONITORING, ETC.)

ACCURATE EVALUATION OF RADIATION SAFETY KNOWiEDGE OF HEALTH PHYSICS & REGS APPROPRIATE APPEARANCE FOR LICENSE TYPE SKILL IN WORDING OUESTIONS I SUITABLE RAPPORT WITH MANAGEMENT AND WORKERS

6MPEP FIELDWORK EVALUATION REVIEWER GUIDANCE PAGE 3

  • ITEM

, O.K. COMMENTS OR QUESTIONS CLOSING PREPARATION FOR EXIT INTERVIEW; ASSEMBLY OF SUPPORTING MATERIAL EXIT CONDUCTED AT APPROPRIATE MANAGEMENT LEVEL VIO FULLY EXPLAINED: LICENSE CONDITION OR REG CITED RECOMMENDATIONS CLEARLY DISTINGUtSHED FROM V10 IMPENDING ENFORCEMENT ACTIONS EXPLAINED LICENSEE ADVISED OF EXPECTED RESPONSE AND REQUIREMENTS FOR CHANGE VIOLATIONS 0.K.

l ADDITIONAL COMMENTS I

$l i

'. -Mi '

l 1

i

Retrieved from "https://kanterella.com/w/index.php?title=ML20210T633&oldid=3443749"