ML23262B175
| ML23262B175 | |
| Person / Time | |
|---|---|
| Issue date: | 04/27/2023 |
| From: | Chang Y, Friedman C, Mishkin A, Polra S, Pringle S, Tanya Smith, Vasquez G NRC/RES/DRA/HFRB, Sphere of Influence |
| To: | |
| References | |
| Download: ML23262B175 (1) | |
Text
Machine Learning Demo Wednesday Prioritizing Inspections using ML Alec Mishkin, Guillermo Vasquez, Stuti Polra, Casey Friedman, Scott Pringle, Theresa Smith Thursday, April 27, 2023
Agenda Topic Modeling Metrics Update Topic Modeling with Machine Generated Summaries Safety Cluster Alignment Progress and Next Steps 2
Metrics Update 3
Metrics Update Last week we described the following three topic modeling metrics
- Inverted Rank-Biased Overlap
- Word Embedding-based Pairwise Distance
- Topic Coherence We are currently testing all three of these metrics The Topic Coherence metric needs a reference corpus. We have tried three different corpuses
- A subset of Wikipedia files (This is part of the gensim library)
- A collection of ~340 NRC Inspection Manual Chapters
- The Standard Technical Specifications of the following plants
> Westinghouse Plants
> Westinghouse Advanced passive 1000 Plants
> Combustion Engineering Plants
> General Electric bwr6 plants
> General Electric bwr4 plants
> Babcock and Wilcox plants Currently, the Standard Technical Specifications reference corpus is yielding the most consistent results We are still performing experiments with the coherence metric and are open to other reference corpuses the NRC recommends 4
Topic Modeling with Machine Generated Summaries 5
Topic Modeling with Machine Generated Summaries Min:13, Max: 258, Avg.: 84.97 Min: 42, Max: 11,670, Avg.: 1648.68 6
BART Summaries Inspection Findings Item Introduction Summaries Generated with BART
- 4hr 45 mins on GPU for 14,938 texts BERTopic configuration:
MMR; MMR + POS; POS +MMR; MMR + Text Generation custom stopwords, n-grams min cluster size 20 15 neighbors, 5 components all-MiniLM-L6-v2 7
BART Summaries (MMR Topic Representation)
Representative Documents A self-revealing non-cited violation (NCV) of 10 CFR 50, Appendix B, Criterion III, Design Topic 3 Control was identified for the licensee's failure to correctly update their design drawing for (558 documents) the Unit 2 Turbine Driven Auxiliary Feedwater (TDAFW) pump electrical controls. This condition revealed itself 24 days later when the licensee performed a surveillance test to confirm operability of the TDAFW pump from the Hot Shutdown Panel and the pump tripped driven auxiliary - turbine driven - auxiliary feedwater pump - A Green self-revealing non-cited violation (NCV) of 10 CFR 50, Appendix B, Criterion V, bearing - tdafw - "Instructions, Procedures, and Drawings," was identified. The licensee inadvertently feedwater afw - sx - operated the Unit 2 turbine driven auxiliary feedwater (TDAFW) pump with the suction essential service water - source isolated. The pump was rendered inoperable for a period of approximately 22 charging - pump hours.
inoperable The licensee failed to promptly identify and correct a condition adverse to quality (CAQ) associated with Unit 1 Turbine Driven Auxiliary Feedwater (TDAFW) pump turbine bearing oil. A finding of very low safety significance, with an associated non-cited violation of 10 CFR Part 50, Appendix B, Criterion 16, "Corrective Actions," was identified.
8
BART Summaries (MMR Topic Representation)
Topic Count Top_n_words
-1 5382 green - cooling reactor - appendix criterion - safety significance - 10 cfr - valves - finding low - design safe shutdown - facility operating - license condition protection - cited violation license - brigade - approved protection program - suppression - renewed -
0 927 combustibles - unit operating license 1 925 high radiation area - area - workers - 10 cfr 20 - safety cornerstone - dose rates - occupational radiation safety - radioactive material - 1501 - rem 2 790 emergency diesel - emergency diesel generator - diesel generator edg - oil - generators - start - division emergency - dg - generator fuel oil - inoperable 3 558 driven auxiliary - turbine driven - auxiliary feedwater pump - bearing - tdafw - feedwater afw - sx - essential service water - charging - pump inoperable entergy failed - entergy corrective - safety significance entergy - entergy personnel did - cr - action program entergy - staff - did adequately - entered issue 4 387
- pnp ncv technical specification - ncv ts procedures - green ncv ts - specification ts procedures - cited violation ncv - procedures identified licensee - ts identified 5 306
- technical - self revealing ncv - procedures programs 6 301 trip - reactor scram - control rod - recirculation pump - revealing finding identified - main turbine - feedwater - level - occurred - operators failed ncv 10 cfr - safety significance associated - significance associated - inspectors identified ncv - identified finding low - inspectors identified finding -
7 272 procedures drawings - 50 appendix criterion - criterion instructions - associated noncited violation 10 - procedures drawings licensee - xvi corrective - appendix criterion xvi - corrective action - inspectors identified noncited -
8 229 instructions procedures - criterion instructions - cfr 50 appendix - adverse quality exelon failed - ir - issue report - finding identified exelon - did - time estimate ete - evacuation time estimate - ete responsible offsite - required date -
9 224 response organizations green noncited violation - identified green noncited - green non - green non cited - inspectors identified green - violation 10 - instructions procedures 10 217 drawings - cfr 50 appendix - criterion - team identified 9
BART Summaries (Custom Stopwords Removed, MMR Topic Representation)
Representative Documents Topic 15 A noncited violation of 10 CFR Part 50, Appendix B, Criterion III, was identified for failure to (154 documents) perform adequate calculations for ac control circuit voltage drop under degraded voltage conditions. The team identified that calculations to determine voltage drop in motor control center and 120 Vac distribution panel control circuits were nonconservative.
degraded voltage -
circuit breakers - 4160 The team identified a Green non-cited violation of 10 CFR 50, Appendix B, Criterion XI, vac - motor center mcc - Test Control, involving the licensees failure to establish a test program which inverter - vacuum demonstrates that components will perform satisfactorily in service. Specifically, the breaker - team non cited licensee failed to record as-found test values prior to performing maintenance for safety
- 480 volt - related - related 4160 Vac magne blast circuit breakers.
appendix criterion iii licensee failed to assure that applicable regulatory requirements for undervoltage (degraded) voltage protection, including those prescribed in TS 3.3.5-1, item 2, were correctly translated into design calculation. The potential availability, reliability, and operability of the 6900 VAC safety buses during a potential degraded voltage condition was impacted as the licensee used a non-conservative degraded voltage input.
10
BART Summaries (Custom Stopwords Removed, MMR Topic Representation)
Topic Count Top_n_words
-1 4375 nrc - maintenance - cooling - appendix criterion - cfr 50 appendix - non cited - violation 10 cfr - cited violation ncv - ncv 10 - low 0 1166 water - driven - service - auxiliary feedwater pump - residual - heat removal - core cooling - turbine driven auxiliary - fan - afw safe shutdown - facility operating license - cited violation license - brigade - approved protection - suppression - renewed - combustibles - transient - unit 1 931 operating 2 921 high radiation area - area - workers - 10 cfr 20 - high - occupational radiation - dose rates - 1501 - rem - work permit 3 845 isolation - operated - main - steam generator - steam isolation valve - msiv - power operated relief - containment - line - solenoid 4 790 emergency diesel generator - emergency - generators - fuel - division - edgs - start - diesel fuel oil - generator fuel oil - oil storage entergy corrective action - entergy personnel - entergy implement - 50 65 entergy - entergy cap - staff - mitigating systems entergy - entergy provide 5 388 adequate - cap cr - cr jaf ncv technical specification - technical specification ts - revealing ncv ts - cited violation ncv - limiting operation lco - technical - ts properly - revealing -
6 370 programs - implement left uncorrected - concern - initiating events - uncorrected significant - lead significant concern - greater - attribute mitigating systems - imc 0612 -
7 312 uncorrected potential - aspect 8 300 turbine - events - operability functionality - self revealing fin - digital - phase - screened low result - cross cutting aspect - feedwater - low self 9 284 low ncv 10 - ncv 10 cfr - level iv ncv - team ncv - criterion appendix criterion instructions - drawings - cfr - having low exelon corrective action - ir - exelon maintain - ete - evacuation time estimate - specification exelon - responsible offsite response - response organizations 10 226
- exelon provide station - oros 11
BART Summaries (Custom Stopwords Removed, MMR Topic Representation)
Topic Count Top_n_words drawings corrective action - xvi corrective - instructions drawings - criterion instructions - appendix criterion xvi - adverse quality - cfr 50 appendix - cited 11 212 violation 10 - corrective - noncited violation technical specification - technical - self revealing noncited - noncited violation - technical specification operations - operations personnel -
12 203 regulatory guide 33 - reviewed self - guide - specification follow cited violation 10 - violation 10 cfr - instructions drawings - criterion instructions - team non cited - cfr 50 appendix - appendix criterion noncited 13 203 violation - cr ggn 14 182 pseg corrective actions - cited violation pseg - sw - strainer - revealing pseg - nuclear - positioner - chiller pseg - tripped - caq 15 174 degraded voltage - circuit breakers - 4160 vac - motor center mcc - inverter - vacuum breaker - team non cited - 480 volt - related - appendix criterion iii 16 160 nrc ncv 10 - ppl - 55a codes standards - codes systems ncv - standards - general visual examinations - xvi corrective action - cfr 50 17 142 charger - 125 vdc battery - related batteries - cells - 250 - direct current - sizing - battery sizing - vdc battery chargers - electrolyte vessel flange - appendix criterion instructions - scram - criterion instructions drawings - coolant - violation 10 - core verification - low non cited - cfr 50 18 131 appendix - low ncv 10 19 126 switchyard - transformers - reserve auxiliary transformer - loose materials - startup - offsite power - fault - non segregated - 345kv - bus duct 20 119 coolant pump rcp - inventory - coolant pressure - operational leakage - room staff - cooldown - pressure boundary - stage seal - rcs level - degrees experience - conditions follow - operability - ans 1978 - assurance manager met - qualification experience ansi - assigned position - position quality 21 113 assurance - quarantine - revision drawings greater - significant concern noncited - criterion instructions drawings - violation 10 cfr - uncorrected - examples noncited violation -
22 112 nonconforming conditions - concern - left - evaluate degraded 12
BART Summaries (Custom Stopwords Removed, MMR + POS Topic Representation)
Representative Documents Tennessee Valley Authority's actions in response to previous silt blockage problems in Topic 14 essential raw cooling water (ERCW) cooling lines did not constitute adequate corrective (162 documents) action. The frequency of monitoring the ERCW line to the 1A-A CCP was unchanged at once every 18 months. The root causes of the event were attributable to less than adequate sensitivity to silt, failure to use a systematic process for non-design engineering output products.
heat - exchangers - The inspectors identified a finding of very low safety significance (Green) regarding the failure component - coolers - to implement corrective actions in a timely manner to identify why the component cooling residual heat - water - water heat exchanger service water outlet valves failed to open in February 2003 and March 2003. A non-cited violation of 10 CFR 50 Appendix B, Criterion XVI, "Corrective Action," was removal - pump - flow -
identified. The reliability and capability of CV-0826 to automatically open on a recirculation fouling actuation signal and provide the required One Non-Cited Violation of 10 CFR Part 50 Appendix B, Criterion XVI "Corrective Action" was identified. Failure to implement actions to prevent recurrence of; failed containment spray heat exchanger tubes caused by microbiologically induced corrosion, and cracking of the component cooling water heat exchangers divider plate welds.
13
BART Summaries (Custom Stopwords Removed, MMR + POS Topic Representation)
Topic Count Top_n_words
-1 5106 water - power - pump - service - maintenance - self - criterion - appendix - cited violation - unit 0 926 radiation - dose - high radiation - area - worker - workers - occupational radiation - rates - radioactive material - rem 1 914 license - safe shutdown - brigade - shutdown - operating - facility - combustibles - suppression - combustible materials - transient combustibles 2 840 valves - valve - steam - main steam - isolation - relief - containment - line - motor - pressurizer 3 764 diesel - generator - emergency - oil - generators - fuel - division - exhaust - surveillance - starting 4 383 staff - risk - corrective actions - actions - operability - quality - damage - basis - higher risk - core 5 331 technical specification - specification - self - operation - unit - maintenance - work - instrumentation - cited violation - vessel 6 327 auxiliary feedwater - pump - feedwater - turbine - bearing - pumps - service - motor - oil - water 7 245 drawings - criterion - instructions - attribute - quality - level - criteria - acceptance - concern - installation 8 231 corrective action - action - quality - repetition - corrective actions - operability - actions - violation - cited violation - deficiency 9 223 offsite response - organizations - estimate - date - evacuation - unit - station - response - time - maintenance 10 220 technical specification - specification - noncited violation - operations - personnel - self - violation - systems - cited violation - maintenance 14
BART Summaries (Custom Stopwords Removed, MMR + POS Topic Representation)
Topic Count Top_n_words 11 204 team - criterion - appendix - examples - cited violation - violation - operability - determination - corrective actions - actions 12 174 chiller - strainer - positioner - refrigerant - service - maintenance - trip - valve - troubleshooting - corrective actions 13 165 team - basis - calculations - cited violation - violation - corrective action - drawings - action - measures - instructions 14 162 heat - exchangers - component - coolers - residual heat - water - removal - pump - flow - fouling 15 155 breakers - voltage - breaker - circuit - degraded voltage - volt - vacuum - team - center - motor 16 150 visual examinations - chase - examinations - test - leak - elevated containment - acceptance - containment - moisture - closure 17 148 battery - charger - batteries - chargers - direct current - cells - sizing - terminal - current - test 18 134 vessel - scram - drawings - coolant - flange - instructions - level - self - power - noncited violation 19 130 transformer - switchyard - bus - auxiliary transformer - transformers - reserve - startup - loose materials - lockout - auxiliary transformers 20 116 relays - relay - degraded voltage - voltage - undervoltage - settings - delay - bus - second level - lockout 21 115 experience - operability - quarantine - determination - corrective action - assurance - action - implementation - actions - aspect 22 114 concern - violation - cited violation - operability - corrective actions - actions - components - foreign material - presence - issues 15
T5 Summaries Inspection Findings Item Introduction Summaries Generated with T5
- 5hr 23 mins on GPU for 14,938 texts BERTopic configuration:
MMR; MMR + POS; POS +MMR; MMR + Text Generation custom stopwords, n-grams min cluster size 20 15 neighbors, 5 components all-MiniLM-L6-v2 16
T5 Summaries (MMR Topic Representation)
Representative Documents Topic 21 inspectors determined that the finding was more than minor because it was associated with the configuration control attribute of the Barrier Integrity cornerstone. finding did not (111 documents) cause spent fuel pool to exceed a maximum temperature limit; b) cause mechanical fuel damage and detectable release of radio-nuclides; c) result in the loss of spent fuel inventory; or d) affect spent fuel shutdown margin.
spent - pool - fuel pool inspectors found a fuel handling team failed to place a spent fuel assembly in its correct sfp - damage fuel - location. the error was contrary to the requirement of step 4.3.13 of procedure RFP 301, outage - transfer - "Refueling Bridge Operations" a prompt evaluation of the assembly was performed and refueling operations - determined that the assembly could remain in the incorrect location with no reduction in handling - bridge - safety margin. inspectors determined the contributing cause was associated with human cooling performance.
inspectors determined that Procedure 40OP-9PC05, "Augmentation of Fuel Pool Cooling with Shutdown Cooling," was inadequate. transfer of 27,000 gallons of borated water from refueling water tank to spent fuel pool. inspectors also determined that control room operators did not perform procedures. failure to maintain and implement a Regulatory Guide 1.33, Appendix A was a non-cited violation.
17
T5 Summaries (MMR Topic Representation)
Topic Count Top_n_words
-1 5429 non cited violation - feedwater - reactor - valve - appendix - cfr failure - criterion - inspectors identified non safety significance licensee - inspectors identified finding - identified finding low - licensee failed - concern - significant - corrective action program - cross 0 895 cutting - lead - performance deficiency dose - high radiation area - occupational radiation safety - workers - rates - cfr 20 - program process - safety cornerstone attribute - exposure - work 1 812 permit 2 672 violation license condition - brigade - combustibles - protection external - suppression - sprinkler - transient - factors - doors - safe shutdown safety significance finding - inspectors identified finding - cross - identified finding - area human performance - associated mitigating systems - cutting area 3 484 human - performance inspectors - cornerstone attribute - aspect area personnel did - cornerstone entergy did - entergy staff - significance green entergy - ncv entergy failure - attribute entergy - safety concern entergy -
4 394 deficiency entergy - entergy did promptly - gt safety significance green - green finding licensee - inspectors identified green - significance green finding - finding licensee failure - self - associated non 5 379 cited - green finding did - finding did - finding identified 6 378 diesel - emergency diesel generator - diesel generator edg - emergency - generators - fuel oil - start - inoperable - edgs - jacket water violation 10 cfr - corrective action licensee - 10 cfr - cfr 50 appendix - appendix corrective action - noncited violation 10 - corrective - criterion licensee 7 236 failed - 50 65 licensee - 50 appendix criterion ncv exelon failure - safety significance exelon - exelon did maintain - did - green exelon - integrity cornerstone exelon - identified green ncv - cornerstone 8 234 exelon did - correct condition adverse - correct appendix design control - design control identified - criterion iii design - control finding - iii - design control licensee - appendix - violation 10 cfr - control 9 230 licensee failed - associated design violation technical specification - technical - cited violation technical - technical specification licensee - noncited violation - specification licensee failed -
10 206 technical specification identified - inspectors identified non - identified non cited - specification procedures finding 18
T5 Summaries (Custom Stopwords Removed , MMR Topic Representation)
Representative Documents a licensees failure to accomplish an activity affecting quality in accordance with Topic 37 documented procedures was identified on October 8, 2020. Specifically, the operators (64 documents) failed to follow the procedure and adequately check to ensure CS-1-37 was closed. this is contrary with OP2.ID1, Clearances in that documented procedures, affecting. quality, were not adhered to while establishing a valve lineup on the Unit 1, containment spray system high pressure core -
hpcs - core spray pump inspectors identified a Green finding for the licensee's failure to have procedures ensuring
- spray cs - 1b core - the satisfactory performance of the high pressure core spray (HPCS) system and the unit containment - cs suppression pool during a station blackout out (SBO) event. Procedures Clinton Power pump - subsystem - Station (CPS) 4402.01 and CPS 4407.01 did not ensure the HPCS system would remain spray pump minimum - available for decay heat removal for the duration of the SBO event.
12 core spray the licensee failed to pre-plan and perform maintenance on level transmitter B21-LTN081C. the failure led to a perturbation in the C reactor water level instrument reference leg that caused an invalid actuation of the high pressure core spray (HPCS) system at power.
19
T5 Summaries (Custom Stopwords Removed , MMR Topic Representation)
Representative Documents Topic 63 inspectors documented a failure to follow station procedure ER-AA-200, Preventative Maintenance Program, Revision 3. licensee utilized a condition-based maintenance (30 documents) approach on the main steam isolation valves (MSIVs) that failed to monitor and trend equipment performance. as corrective actions, the licensee repaired and tested the valves prior to returning the unit to the modes of applicability.
main steam isolation -
main steam - isolation valve msiv - isolation -
outboard steam susquehanna did not correctly validate a deficient condition associated with the unit 1 'B' isolation - steam inboard main steam isolation valve (MSIV) direct current (DC) solenoid valve. a half-isolation valves - isolation by de-energizing the alternating current (AC) solenoids was performed. the inboard main - msiv inadvertent closure of the 'b' MSIV resulted in a high pressure scram.
inoperable - valve pciv -
unit main inspectors identified a finding of very low safety significance involving the preconditioning of the Unit 1 Main Steam Isolation Valves (MSIVs) pre-stroking all the MSIVs during the limit switch calibration and replacing the ASCO test solenoid valve on the 'D' MSIV unacceptably preconditioned the valves. finding is related to the cross cutting area of Problem Identification and Resolution (PI&R) 20
T5 Summaries (Custom Stopwords Removed , MMR Topic Representation)
Topic Count Top_n_words
-1 5177 non cited violation - cfr 50 appendix - steam - criterion - violation 10 cfr - feedwater - ncv technical - valves 0 845 concern low - lead significant concern - uncorrected - significant - left - quality - deficiency - cross cutting aspect - mitigating systems - corrective action 1 819 high radiation area - occupational radiation - workers - rates - dose rate - exposure - work permit - millirem - survey - cfr 20 1501 2 672 brigade - suppression - materials - sprinkler - external factors - safe shutdown - transient combustibles - license protection - watches - rating 3 599 emergency diesel generator - emergency - generators - fuel - jacket water - division - edgs - start - inoperable - oil transfer 4 495 auxiliary - service - driven - afw - heat removal rhr - bearing - residual - feedwater afw - auxiliary feedwater pump - pump motor cross cutting aspect - low mitigating - human - cutting area human - aspect area - mitigating systems attribute - attribute cross - significant concern - result operability -
5 475 area human low low entergy - ncv entergy - mitigating systems entergy - attribute entergy - gt - entergy verify - entergy personnel adequately - qualification deficiency entergy -
6 389 personnel - adverse violation technical specification - technical - noncited - specification low - source range - non cited violation - reactivity - technical specification implement - greater 7 279 affects - example involved 8 243 violation 10 cfr - appendix corrective - cfr 50 appendix corrective action 50 appendix criterion - noncited - adverse quality - risk assessment exelon assess - exelon verify - exelon establish - exelon perform adequate - integrity exelon - barrier integrity exelon - deficiency exelon - 50 65 exelon - correct adverse 9 236
- ncv exelon implement 10 219 screened represent - ginna - result operability functionality - low identify - self revealing - functionality - dating - cross cutting aspect - team - risk 21
T5 Summaries (Custom Stopwords Removed , MMR Topic Representation)
Topic Count Top_n_words main steam isolation - main steam - isolation valve msiv - isolation - outboard steam isolation - steam isolation valves - inboard main - msiv inoperable - valve pciv - unit 63 30 main abv - ventilation damper - damper failures ncv - building ventilation charcoal - inspector ncv ineffective - inoperable approximately 19 - approximately 19 hours2.199074e-4 days <br />0.00528 hours <br />3.141534e-5 weeks <br />7.2295e-6 months <br /> - cref -
64 30 follow general administrative - low standby gas action report ano - xec - translate installed deficiency - drawings corrective action - appendix verify adequacy - translate regulatory - action 10 cfr - iii corrective -
65 29 appendix translate - maintain deficiency non limiting operation lco - hatch covers - recombiners - hydrogen igniters - enter ts - containment atmosphere - emergency ventilation - radiological barrier - heater 66 28 breaker revision - functional repeat occurred molded case circuit - case circuit breakers - case - mccbs - related molded - class 1e - circuit breaker mccb - breakers deficiency - 125 vdc molded - nonconforming 67 28 involving molded maintain related - cables environment - medium voltage - environment - degraded cable - voltage cables cables - maintain normally energized - manhole - water 68 27 submergence - flooded degradation radiological barrier - radiological - degradation - barrier provided - aegts - represents - low represent degradation - provided - secondary containment 69 27 inoperable - provided auxiliary building specification allowed outage - greater technical specification - represent single train - actual single - deficiency represent single - train violation 10 - time noncited -
70 26 unavailability train - systems actual - allowed operated valves movs - related motor - operated - motor operated valve - thermal overload relays - owners - prevent movs performing - able demonstrate tols - valve 71 25 mov lacked - circuits sized violation verify - account instrument uncertainty - palo verde unit - limits surveillance established - opening porv unexpectedly - quality accordance non - methodology 72 23 rods insert - maintain motor actuator - lowered inventory vessel - louvers notice 73 22 emergency lighting units - elus - post shutdown - battery packs - install emergency - manual operation - install - safe - appendix emergency - lighting affects ability concern team low - frum - nrc inspection nuclear - faulty level gage - factors nextera appropriately - occurred june 12 - non sw - facilities violation - fitting fixed core -
74 22 notification manner 22
T5 Summaries (Custom Stopwords Removed , POS + MMR Topic Representation)
Representative Documents Topic 66 a licensee's vendor examiner failed to perform an ultrasonic (UT) examination. if left (31 documents) uncorrected, it would have the potential to lead to a more significant safety concern.
inspectors answered no to all of the characterizations worksheet questions.
ultrasonic - weld -
examination - ultrasonic a self-revealed finding of very low safety significance (i.e., Green) was identified for the examinations - welding - licensees failure to have adequately implemented established welding requirements.
rejectable flaw - Specifically, plant personnel conducting welding on a steam line for the Unit 1 main turbine characterizations - welded over and blocked a sensing line for an instrument manifold. the failure to execute ultrasonic testing - required weld preparation activities as prescribed by CC-AA-501-1021, Ex qualified weld - cracks inspectors identify two examples of failure to properly evaluate inservice inspection related corrective actions. a licensee failed to evaluate an indication found during a 1999 ultrasonic examination of a core weld. the safety significance of these issues was considered very low. failure to promptly identify and correct issues found during the examination was considered a non-cited violation.
23
T5 Summaries (Custom Stopwords Removed , POS + MMR Topic Representation)
Representative Documents Topic 74 inspectors found thermal overload relays on safety-related motor-operated valve circuits (23 documents) were not sized properly. finding was of very low safety significance because it was a design deficiency. licensee was able to demonstrate that the TOLs would not prevent any MOVs from performing safety function.
motor - thermal overload - overload -
valves - jogging - inspectors found thermal overload relays on safety-related motor-operated valve circuits systemrestoration were not sized properly. licensee was able to demonstrate that the TOLs would not prevent alignment - repetitive any MOVs from performing their safety function. the issue was of very low safety jogging - valve - significance because it was a design deficiency.
established acceptance
- bypass the team identified a finding having very-low significance and an associated Non-Cited Violation of Title 10 of the Code of Federal Regulations, Part 50. Specifically, Dresden had not verified that thermal overload relays (TOLs) on Unit 3 safety-related motor operated valves (MOVs) were properly sized. the team determined the performance deficiency was more than minor because it was associated with the design control attribute of the 24
T5 Summaries (Custom Stopwords Removed , POS + MMR Topic Representation)
Topic Count Top_n_words
-1 5230 pump - water - feedwater - valve - power - turbine - unit - service - trip - cited violation 0 908 significant concern - concern - cutting - cross - operability - aspect - quality - deficiency - cutting aspect - systems 1 817 radiation - dose - high radiation - worker - occupational radiation - area - workers - rates - exposure - millirem 2 682 protection - license - door - combustibles - watch - brigade - suppression - combustible materials - materials - sprinkler 3 488 cutting - cross - aspect - cutting aspect - cutting area - significant concern - systems - attribute - area - concern 4 405 diesel - generator - edg - emergency - generators - oil - start - jacket - leak - division 5 381 personnel - staff - implement - attribute - maintenance - wall - team - barrier - outage - operability 6 366 noncited violation - criterion - violation - cited violation - qualification - deficiency - personnel - engineering - evaluations - inspections 7 364 self - team - trip - cutting aspect - aspect - risk - functionality - likelihood - mitigation - operability 8 341 severity - corrective action - action - appendix - criterion - cited violation - violation - quality - level - team 9 214 technical specification - specification - noncited violation - cited violation - violation - examples - step - operators - guidance - operating 10 186 valves - valve - check - position - testing - stroke - torque - stem - motor - actuator 25
T5 Summaries (Custom Stopwords Removed , POS + MMR Topic Representation)
Topic Count Top_n_words 65 32 cooling - component - water - vital component - cooler - capillary line - refill - threads - primary component - temporary modification 66 31 ultrasonic - weld - examination - ultrasonic examinations - welding - rejectable flaw - characterizations - ultrasonic testing - qualified weld - cracks 67 30 effectiveness - classification - emergency - level - plan - change - thresholds - basis - unusual event - scheme 68 29 damper - dampers - ventilation - charcoal - previous damper - overhaul - effluents - radiological barrier - degraded damper - available operating 69 29 radiological barrier - degradation - barrier - necessary vacuum - open configuration - secondary containment - minutes - auxiliary building - structure - self 70 28 cables - cable - environment - submergence - medium voltage - physical environment - degraded cable - manhole - submersion - underground manholes 71 28 molded case - case - breakers - circuit - 1e - class - life - second example - significant safetyconcern - safetyconcern 72 25 single train - train - outage - time - nondestructive testing - noncited violation - technical basis - conservative bias - bias - qualification 73 24 lighting - lights - units - packs - emergency - manual operation - safe shutdown - shutdown - light - adequate emergency 74 23 motor - thermal overload - overload - valves - jogging - systemrestoration alignment - repetitive jogging - valve - established acceptance - bypass 75 23 devise - current protection - adequate root - flange - legacy - correction - techniques - manipulation - weakness - protection 76 20 welders - welding - weld - interpass - welds - preheat - search - examination - adequate search - volumetric inspection 26
Takeaways from Topic Modeling with Machine Generated Summaries Topic terms are more closely related to representative documents after using condensed summaries of long documents Increased appearance of stopwords when allowing n-grams in topic terms because there is less common text present between summaries of the documents Some summaries dont cover the technical issue but focus on regulations/violations or cornerstone/cross-cutting areas so the discovered topic terms are sometimes vague (cited or non cited violation, technical specification, personnel, staff, requirement, criterion, appendix, systems) 27
Safety Cluster Alignment 28
Safety Clusters sorted and color coded by number of findings (5 years) 29
Safety Cluster #2 - Limited number of inspection procedures 30
Drilling Down - Cornerstones, sites, and reports 31
Drilling Down - Safety Cluster information, summary, and entities
Title:
Failure to Verify the Adequacy of the Air Pressure Regulator Setpoint Value for Containment Isolation Valves 1(2)RF026 Cluster Key Terms: valves - valve - isolation - steam - containment - pathway - integrity - actuator - check Document Count: total = 741 5 years = 65 Machine Generated Summary: Inspectors identified a Green finding and an associated NCV of 10 CFR Part 50, Appendix B, Criterion III, Design Control, for the failure to verify the adequacy of the air pressure regulator setpoint value for air-operated containment isolation valves 1(2)RF026. The finding screened as of very low safety significance (Green) because it did not represent an actual open pathway.
32
Sites sorted and color coded (5 years)
- of Findings Safety Clusters with # of Findings Site -> Cluster -> Procedure 33
Progress 34
SOW Task Status Phase I: March 6, 2023 - April 9, 2023 Status Phase II: March 20, 2023 - May 7, 2023 Status Describe the Problem Complete Platform/system selection and installation In progress Search the Literature Complete Data acquisition and preparation In progress Select Candidates Complete Feature pipeline engineering In progress Select Evaluation Factors Complete Clustering method experimentation & selection In progress Develop evaluation factor weights Complete Cluster pipeline engineering In progress Define evaluation factor ranges Complete Anomaly detection (as needed) Not needed Perform assessment Complete Model Development, Training, Evaluation In progress Report Results Complete Test harness development In progress Deliver Trade study report Complete PoC integration and demonstration In progress Trial runs and evaluation In progress Demonstrate PoC capability Not started Phase III: April 19, 2023 - June 16, 2023 Status Live data ingestion In progress Model execution In progress Cluster evaluation In progress Critical Method documentation Not started Technical Report Document Not started Deliver final report with findings Not started 35
Next Steps 36
Next Steps Safety Cluster Formation
- Assign outlier documents to safety clusters - use metrics to guide quality Topic Modeling Summaries
- Summaries from other models (flan-t5, pgasus-cnn-daily) for all inspection findings item introductions
- Exploring other models that were designed for NLP tasks with long input sequences (Long T5, Longformer, Reformer, Big Bird, XLNet) (stretch)
- Use noun chunks (from SpaCy) as tokens instead of single words or bi-grams/tri-grams (stretch)
- Topic modeling with full inspection report text (stretch)
Metrics
- Evaluate Coherence and Diversity metrics to better evaluate topic models
- Add new corpi of text as needed Deliverables
- Consolidate experiments, inputs, and outputs to deliverable format 37