ML23262B175
| ML23262B175 | |
| Person / Time | |
|---|---|
| Issue date: | 04/27/2023 |
| From: | Chang Y, Friedman C, Mishkin A, Polra S, Pringle S, Tanya Smith, Vasquez G NRC/RES/DRA/HFRB, Sphere of Influence |
| To: | |
| References | |
| Download: ML23262B175 (1) | |
Text
Machine Learning Demo Wednesday
Prioritizing Inspections using ML
Alec Mishkin, Guillermo Vasquez, Stuti Polra, Casey Friedman, Scott Pringle, Theresa Smith
Thursday, April 27, 2023 Agenda
Topic Modeling Metrics Update Topic Modeling with Machine Generated Summaries Safety Cluster Alignment Progress and Next Steps
2 Metrics Update
3 Metrics Update
Last week we described the following three topic modeling metrics
- Inverted Rank-Biased Overlap
- Word Embedding-based Pairwise Distance
- Topic Coherence We are currently testing all three of these metrics
The Topic Coherence metric needs a reference corpus. We have tried three different corpuses
- A subset of Wikipedia files (This is part of the gensim library)
- A collection of ~340 NRC Inspection Manual Chapters
- The Standard Technical Specifications of the following plants
> Westinghouse Plants
> Westinghouse Advanced passive 1000 Plants
> Combustion Engineering Plants
> General Electric bwr6 plants
> General Electric bwr4 plants
> Babcock and Wilcox plants Currently, the Standard Technical Specifications reference corpus is yielding the most consistent results We are still performing experiments with the coherence metric and are open to other reference corpuses the NRC recommends
4 Topic Modeling with Machine Generated Summaries
5 Topic Modeling with Machine Generated Summaries
Min:13, Max: 258, Avg.: 84.97 Min: 42, Max: 11,670, Avg.: 1648.68
6 BART Summaries
Inspection Findings Item Introduction Summaries Generated with BART
- 4hr 45 mins on GPU for 14,938 texts BERTopic configuration:
MMR; MMR + POS; POS +MMR; MMR + Text Generation
custom stopwords, n-grams
min cluster size 20
15 neighbors, 5 components
all-MiniLM-L6-v2
7 BART Summaries (MMR Topic Representation)
Representative Documents
A s e l f-revealing non -cited violation (NCV) of 10 CFR 50, Appendix B, Criterion III, Design Topic 3 Control was identified for the licensee's failure to correctly update their design drawing for (558 documents) the Unit 2 Turbine Driven Auxiliary Feedwater (TDAFW) pump electrical controls. This condition revealed itself 24 days later when the licensee performed a surveillance test to confirm operability of the TDAFW pump from the Hot Shutdown Panel and the pump tripped driven auxiliary - turbine driven - auxiliary A Green self-revealing non -cited violation (NCV) of 10 CFR 50, Appendix B, Criterion V, feedwater pump - "Instructions, Procedures, and Drawings," was identified. The licensee inadvertently bearing - tdafw - operated the Unit 2 turbine driven auxiliary feedwater (TDAFW) pump with the suction feedwater afw - sx -source isolated. The pump was rendered inoperable for a period of approximately 22 essential service water - hours.
charging - pump inoperable
The licensee failed to promptly identify and correct a condition adverse to quality (CAQ) associated with Unit 1 Turbine Driven Auxiliary Feedwater (TDAFW) pump turbine bearing oil. A finding of very low safety significance, with an associated non-cited violation of 10 CFR Part 50, Appendix B, Criterion 16, "Corrective Actions," was identified.
8 BART Summaries (MMR Topic Representation)
To p i c Count Top_n_words
-1 5382 green - cooling - 50 - reactor - appendix criterion - safety significance - 10 cfr - valves - finding low - design
0 927 safe shutdown - facility operating - license condition protection - cited violation license - brigade - approved protection prog ram - suppression - renewed -combustibles - unit operating license
1 925 high radiation area - area - w o r ke rs - 10 cfr 20 - safety cornerstone - dose rates - occupational radiation safety - radioactive material - 1501 - rem
2 790 emergency diesel - emergency diesel generator - diesel generator edg - oil - ge n e rato rs - start - division emergency - dg - generator fuel oil - inoperable
3 558 driven auxiliary - turbine driven - auxiliary feedwater pump - bearing - tdafw - feedwater afw - sx - essential service water - charging - pump inoperable
4 387 entergy failed - entergy corrective - safety significance entergy - entergy personnel did - cr - action program entergy - staff - did adequately - entered issue
- pnp 5 306 ncv technical specification - ncv ts procedures - green ncv ts - specification ts procedures - cited violation ncv - procedures identified licensee - ts identified
- technical - self revealing ncv - procedures programs
6 301 trip - reactor scram - control rod - recirculation pump - revealing finding identified - main turbine - feedwater - level - occurred - operators failed
7 272 ncv 10 cfr - safety significance associated - significance associated - inspectors identified ncv - identified finding low - inspectors identified finding -procedures drawings - 50 appendix criterion - criterion instructions - associated
8 229 noncited violation 10 - procedures drawings licensee - xvi corrective - appendix criterion xvi - corrective action - inspectors identified noncited -
instructions procedures - criterion instructions - cfr 50 appendix - adverse quality 9 224 e xe l o n failed - ir - issue report - finding identified e xe l o n - did - time estimate ete - evacuation time estimate - ete responsible offsite - required date -
response organizations 10 217green noncited violation - identified green noncited - green non - green non cited - inspectors identified green - violation 10 - instructions procedures drawings - cfr 50 appendix - criterion - team identified
9 BART Summaries (Custom Stopwords Removed, MMR Topic Representation)
Representative Documents
Topic 15 A noncited violation of 10 CFR Part 50, Appendix B, Criterion III, was identified for failure to (154 documents) perform adequate calculations for ac control circuit voltage drop under degraded voltage conditions. The team identified that calculations to determine voltage drop in motor control center and 120 Vac distribution panel control circuits were nonconservative.
degraded voltage -
circuit breakers - 4160 The team identified a Green non-cited violation of 10 CFR 50, Appendix B, Criterion XI, vac - motor center mcc - Test Control, involving the licensees failure to establish a test program which inverter - vacuum demonstrates that components will perform satisfactorily in service. Specifically, the breaker - team non cited licensee failed to record as -found test values prior to performing maintenance for safety
- 480 volt - related - related 4160 Vac magne blast circuit breakers.
appendix criterion iii
licensee failed to assure that applicable regulatory requirements for undervoltage (degraded) voltage protection, including those prescribed in TS 3.3.5-1, item 2, were correctly translated into design calculation. The potential availability, reliability, and operability of the 6900 VA C safety buses during a potential degraded voltage condition was impacted as the licensee used a non-conservative degraded voltage input.
10 BART Summaries (Custom Stopwords Removed, MMR Topic Representation)
To p i c Count Top_n_words
-1 4375 nrc - maintenance - cooling - appendix criterion - cfr 50 appendix - non cited - violation 10 cfr - cited violation ncv - ncv 10 - low
0 1166 water - driven - service - auxiliary feedwater pump - residual - heat removal - core cooling - turbine driven auxiliary - fan - afw
1 931 safe shutdown - facility operating license - cited violation license - brigade - approved protection - suppression - renewed - combustibles - transient - unit operating
2 921 high radiation area - area - w o r ke rs - 10 cfr 20 - high - occupational radiation - dose rates - 1501 - rem - work permit
3 845 isolation - operated - main - steam generator - steam isolation valve - msiv - power operated relief - containment - line - solenoid
4 790 emergency diesel generator - emergency - ge n e rato rs - fuel - division - edgs - start - diesel fuel oil - generator fuel oil - oil storage
5 388 entergy corrective action - entergy personnel - entergy implement - 50 65 entergy - entergy cap - staff - mitigating systems entergy - entergy provide adequate - cap cr - cr jaf 6 370 ncv technical specification - technical specification ts - revealing ncv ts - cited violation ncv - limiting operation lco - technical - ts properly - revealing -
p ro gra m s - implement 7 312 left uncorrected - concern - initiating events - uncorrected significant - lead significant concern - greater - attribute mitigating systems - imc 0612 -
uncorrected potential - aspect
8 300 turbine - events - operability functionality - self revealing fin - digital - phase - screened low result - cross cutting aspect - feedwater - low self
9 284 low ncv 10 - ncv 10 cfr - level iv ncv - team ncv - criterion - 50 - appendix criterion instructions - drawings - cfr - having low
10 226 e xe l o n corrective action - ir - e xe l o n maintain - ete - evacuation time estimate - specification e xe l o n - responsible offsite response - response organizations
- e xe l o n provide station - oros
11 BART Summaries (Custom Stopwords Removed, MMR Topic Representation)
To p i c Count Top_n_words 11 212 drawings corrective action - xvi corrective - instructions drawings - criterion instructions - appendix criterion xvi - adverse quality - cfr 50 appendix - cited violation 10 - corrective - noncited 12 203 violation technical specification - technical - self revealing noncited - noncited violation - technical specification operations - operations personnel -
regulatory guide 33 - reviewed self - guide - specification follow 13 203 cited violation 10 - violation 10 cfr - instructions drawings - criterion instructions - team non cited - cfr 50 appendix - appendix criterion - 50 - noncited violation - cr ggn 14 182 pseg corrective actions - cited violation pseg - sw - strainer - revealing pseg - nuclear - positioner - chiller pseg - tripped - caq
15 174 degraded voltage - circuit breakers - 4160 vac - motor center mcc - inverter - vacuum breaker - team non cited - 480 volt - related - appendix criterion iii
16 160 nrc ncv 10 - ppl - 55a codes standards - codes - 65 - syste m s ncv - standards - general visual examinations - xvi corrective action - cfr 50
17 142 charger - 125 vdc battery - related batteries - cells - 250 - direct current - sizing - battery sizing - vdc battery chargers - electrolyte
18 131 vessel flange - appendix criterion instructions - scram - criterion instructions drawings - coolant - violation 10 - core verification - low non cited - cfr 50 appendix - low ncv 10
19 126 switchyard - transformers - reserve auxiliary transformer - loose materials - startup - offsite power - fault - non segregated - 345kv - bus duct
20 119 coolant pump rcp - inventory - coolant pressure - operational leakage - room staff - cooldown - pressure boundary - stage seal - rcs level - degrees
21 113 experience - conditions follow - operability - ans 1978 - assurance manager met - qualification experience ansi - assigned position - position quality assurance - quarantine - revision
22 112 drawings greater - significant concern noncited - criterion instructions drawings - violation 10 cfr - uncorrected - examples noncited violation -nonconforming conditions - concern - left - evaluate degraded
12 BART Summaries (Custom Stopwords Removed, MMR + POS Topic Representation)
Representative Documents
Tennessee Valley Authority's actions in response to previous silt blockage problems in Topic 14 essential raw cooling water (ERCW) cooling lines did not constitute adequate corrective (162 documents) action. The frequency of monitoring the ERCW line to the 1A-A CCP was unchanged at once every 18 months. The root causes of the event were attributable to less than adequate sensitivity to silt, failure to use a systematic process for non-design engineering output products.
heat - exchangers - The inspectors identified a finding of very low safety significance (Green) regarding the failure component - coolers - to implement corrective actions in a timely manner to identify why the component cooling residual heat - water - water heat exchanger service water outlet valves failed to open in February 2003 and March removal - pump - flow - 2003. A non-cited violation of 10 CFR 50 Appendix B, Criterion XVI, "Corrective Action," was fouling identified. The reliability and capability of CV-0826 to automatically open on a recirculation actuation signal and provide the required
One Non-Cited Violation of 10 CFR Part 50 Appendix B, Criterion XVI "Corrective Action" was identified. Failure to implement actions to prevent recurrence of; failed containment spray heat exchanger tubes caused by microbiologically induced corrosion, and cracking of the component cooling water heat exchangers divider plate welds.
13 BART Summaries (Custom Stopwords Removed, MMR + POS Topic Representation)
To p i c Count Top_n_words
-1 5106 water - p ower - pump - service - maintenance - self - criterion - appendix - cited violation - unit
0 926 radiation - dose - high radiation - area - worker - w o r ke rs - occupational radiation - rate s - radioactive material - rem
1 914 license - safe shutdown - brigade - shutdown - operating - facility - combustibles - suppression - combustible materials - transient combustibles
2 840 valves - valve - steam - main steam - isolation - relief - containment - line - motor - pressurizer
3 764 diesel - generator - emergency - oil - ge n e rato rs - fuel - division - exhaust - surveillance - starting
4 383 staff - risk - corrective actions - actions - operability - quality - damage - basis - higher risk - core
5 331 technical specification - specification - self - operation - unit - maintenance - work - instrumentation - cited violation - vessel
6 327 auxiliary feedwater - pump - feedwater - turbine - bearing - pumps - service - motor - oil - water
7 245 drawings - criterion - instructions - attribute - quality - level - criteria - acceptance - concern - installation
8 231 corrective action - action - quality - repetition - corrective actions - operability - actions - violation - cited violation - deficiency
9 223 offsite response - organizations - estimate - date - evacuation - unit - station - response - time - maintenance
10 220 technical specification - specification - noncited violation - operations - personnel - self - violation - syste m s - cited violation - maintenance
14 BART Summaries (Custom Stopwords Removed, MMR + POS Topic Representation)
To p i c Count Top_n_words
11 204 team - criterion - appendix - examples - cited violation - violation - operability - determination - corrective actions - actions
12 174 chiller - strainer - positioner - refrigerant - service - maintenance - trip - valve - troubleshooting - corrective actions
13 165 team - basis - calculations - cited violation - violation - corrective action - drawings - action - measures - instructions
14 162 heat - exchangers - component - coolers - residual heat - water - removal - pump - flow - fouling
15 155 breakers - voltage - breaker - circuit - degraded voltage - volt - vacuum - team - center - motor
16 150 visual examinations - chase - examinations - test - leak - elevated containment - acceptance - containment - moisture - closure
17 148 battery - charger - batteries - ch a rge rs - direct current - cells - sizing - terminal - current - test
18 134 vessel - scram - drawings - coolant - flange - instructions - level - self - p ower - noncited violation
19 130 transformer - switchyard - bus - auxiliary transformer - transformers - reserve - startup - loose materials - lockout - auxiliary transformers
20 116 re l ays - re l ay - degraded voltage - voltage - undervoltage - settings - d e l ay - bus - second level - lockout
21 115 experience - operability - quarantine - determination - corrective action - assurance - action - implementation - actions - aspect
22 114 concern - violation - cited violation - operability - corrective actions - actions - components - foreign material - presence - issues
15 T5 Summaries
Inspection Findings Item Introduction Summaries Generated with T5
- 5hr 23 mins on GPU for 14,938 texts BERTopic configuration:
MMR; MMR + POS; POS +MMR; MMR + Text Generation
custom stopwords, n-grams
min cluster size 20
15 neighbors, 5 components
all-MiniLM-L6-v2
16 T5 Summaries (MMR Topic Representation)
Representative Documents
Topic 21 inspectors determined that the finding was more than minor because it was associated
( 111 documents) with the configuration control attribute of the Barrier Integrity cornerstone. finding did not cause spent fuel pool to exceed a maximum temperature limit; b) cause mechanical fuel damage and detectable release of radio-nuclides; c) result in the loss of spent fuel inventory; or d) affect spent fuel shutdown margin.
spent - pool - fuel pool inspectors found a fuel handling team failed to place a spent fuel assembly in its correct sfp - damage fuel - location. the error was contrary to the requirement of step 4.3.13 of procedure RFP 301, outage - transfer - "Refueling Bridge Operations" a prompt evaluation of the assembly was performed and refueling operations - determined that the assembly could remain in the incorrect location with no reduction in handling - bridge - safety margin. inspectors determined the contributing cause was associated with human cooling performance.
inspectors determined that Procedure 40OP-9PC05, "Augmentation of Fuel Pool Cooling with Shutdown Cooling," was inadequate. transfer of 27,000 gallons of borated water from refueling water tank to spent fuel pool. inspectors also determined that control room operators did not perform procedures. failure to maintain and implement a Regulatory Guide 1.33, Appendix A was a non-cited violation.
17 T5 Summaries (MMR Topic Representation)
To p i c Count Top_n_words
-1 5429 non cited violation - feedwater - reactor - valve - appendix - cfr - 10 - failure - criterion - inspectors identified non 0 895 safety significance licensee - inspectors identified finding - identified finding low - licensee failed - concern - significant - corrective action program - cross cutting - lead - performance deficiency
1 812 dose - high radiation area - occupational radiation safety - w o r ke rs - rate s - cfr 20 - p ro g ra m p ro ce s s - safety cornerstone attribute - exposure - work permit
2 672 violation license condition - brigade - combustibles - protection external - suppression - sprinkler - transient - fa cto rs - doors - safe shutdown 3 484 safety significance finding - inspectors identified finding - cross - identified finding - area human performance - associated mitigating systems - cutting area human - performance inspectors - cornerstone attribute - aspect area 4 394 personnel did - cornerstone entergy did - entergy staff - significance green entergy - ncv entergy failure - attribute entergy - safety concern entergy -
deficiency entergy - entergy did promptly - gt
5 379 safety significance green - green finding licensee - inspectors identified green - significance green finding - finding licensee failure - self - associated non cited - green finding did - finding did - finding identified
6 378 diesel - emergency diesel generator - diesel generator edg - emergency - ge n e rato rs - fuel oil - start - inoperable - edgs - jacke t w a t e r
7 236 violation 10 cfr - corrective action licensee - 10 cfr - cfr 50 appendix - appendix corrective action - noncited violation 10 - corrective - criterion licensee failed - 50 65 licensee - 50 appendix criterion
8 234 ncv exelon failure - safety significance exelon - exelon did maintain - did - green exelon - integrity cornerstone exelon - identified green ncv - cornerstone exelon did - correct condition adverse - correct
9 230 appendix design control - design control identified - criterion iii design - control finding - iii - design control licensee - appendix - violation 10 cfr - control licensee failed - associated design
10 206 violation technical specification - technical - cited violation technical - technical specification licensee - noncited violation - specification licensee failed -
technical specification identified - inspectors identified non - identified non cited - specification procedures finding
18 T5 Summaries (Custom Stopwords Removed, MMR Topic Representation)
Representative Documents
a licensees failure to accomplish an activity affecting quality in accordance with Topic 37 documented procedures was identified on October 8, 2020. Specifically, the operators (64 documents) failed to follow the procedure and adequately check to ensure CS-1-37 was closed. this is contrary with OP2.ID1, Clearances in that documented procedures, affecting. quality, were not adhered to while establishing a valve lineup on the Unit 1, containment spray system
high pressure core -
hpcs - core spray pump inspectors identified a Green finding for the licensee's failure to have procedures ensuring
- spray cs - 1b core -the satisfactory performance of the high pressure core spray (HPCS) system and the unit containment - cs suppression pool during a station blackout out (SBO) event. Procedures Clinton Power pump - subsystem - Station (CPS) 4402.01 and CPS 4407.01 did not ensure the HPCS system would remain spray pump minimum - available for decay heat removal for the duration of the SBO event.
12 core spray
the licensee failed to pre-plan and perform maintenance on level transmitter B21-LTN081C. the failure led to a perturbation in the C reactor water level instrument reference leg that caused an invalid actuation of the high pressure core spray (HPCS) system at power.
19 T5 Summaries (Custom Stopwords Removed, MMR Topic Representation)
Representative Documents
Topic 63 inspectors documented a failure to follow station procedure ER-AA-200, Preventative (30 documents) Maintenance Program, Revision 3. licensee utilized a condition -based maintenance approach on the main steam isolation valves (MSIV s) that failed to monitor and trend equipment performance. as corrective actions, the licensee repaired and tested the valves prior to returning the unit to the modes of applicability.
main steam isolation -
main steam - isolation valve msiv - isolation - susquehanna did not correctly validate a deficient condition associated with the unit 1 'B' outboard steam inboard main steam isolation valve (MSIV ) direct current (DC) solenoid valve. a half-isolation - steam isolation by de-energizing the alternating current (AC) solenoids was performed. the isolation valves - inadvertent closure of the 'b' MSIV resulted in a high pressure scram.
inboard main - msiv inoperable - valve pciv -
unit main inspectors identified a finding of very low safety significance involving the preconditioning of the Unit 1 Main Steam Isolation Valves (MSIV s) pre-stroking all the MSIVs during the limit switch calibration and replacing the ASCO test solenoid valve on the 'D' MSIV unacceptably preconditioned the valves. finding is related to the cross cutting area of Problem Identification and Resolution (PI&R)
20 T5 Summaries (Custom Stopwords Removed, MMR Topic Representation)
To p i c Count Top_n_words
-1 5177 non cited violation - cfr 50 appendix - steam - criterion - violation 10 cfr - feedwater - ncv - 10 - technical - valves
0 845 concern low - lead significant concern - uncorrected - significant - left - quality - deficiency - cross cutting aspect - mitigating systems - corrective action
1 819 high radiation area - occupational radiation - workers - rates - dose rate - exposure - work permit - millirem - survey - cfr 20 1501
2 672 brigade - suppression - materials - sprinkler - external factors - safe shutdown - transient combustibles - license protection - watches - rating
3 599 emergency diesel generator - emergency - generators - fuel - jacket water - division - edgs - start - inoperable - oil transfer
4 495 auxiliary - service - driven - afw - heat removal rhr - bearing - residual - feedwater afw - auxiliary feedwater pump - pump motor
5 475 cross cutting aspect - low mitigating - human - cutting area human - aspect area - mitigating systems attribute - attribute cross - significant concern - result operability -
area human low 6 389 low entergy - ncv entergy - mitigating systems entergy - attribute entergy - gt - entergy verify - entergy personnel adequately - qualification deficiency entergy -
personnel - adverse 7 279 violation technical specification - technical - noncited - specification low - source range - non cited violation - reactivity - technical specification implement - greater affects - example involved
8 243 violation 10 cfr - appendix corrective - cfr 50 appendix - 10 - corrective action - 65 - 50 appendix criterion - noncited - adverse quality - risk assessment
9 236 exelon assess - exelon verify - exelon establish - exelon perform adequate - integrity exelon - barrier integrity exelon - deficiency exelon - 50 65 exelon - correct adverse
- ncv exelon implement
10 219 screened represent - ginna - result operability functionality - low identify - self revealing - functionality - dating - cross cutting aspect - team - risk
21 T5 Summaries (Custom Stopwords Removed, MMR Topic Representation)
To p i c Count Top_n_words 63 30 main steam isolation - main steam - isolation valve msiv - isolation - outboard steam isolation - steam isolation valves - inboard main - msiv inoperable - valve pciv - unit main 64 30 abv - ventilation damper - damper failures ncv - building ventilation charcoal - inspector ncv ineffective - inoperable approximately 19 - approximately 19 hours2.199074e-4 days <br />0.00528 hours <br />3.141534e-5 weeks <br />7.2295e-6 months <br /> - cref -
follow general administrative - low standby gas 65 29 action report ano - xec - translate installed deficiency - drawings corrective action - appendix verify adequacy - translate regulatory - action 10 cfr - iii corrective -
appendix translate - maintain deficiency non 66 28 limiting operation lco - hatch covers - recombiners - hydrogen igniters - enter ts - containment atmosphere - emergency ventilation - radiological barrier - heater breaker revision - functional repeat occurred
67 28 molded case circuit - case circuit breakers - case - mccbs - related molded - class 1e - circuit breaker mccb - breakers deficiency - 125 vdc molded - nonconforming involving molded
68 27 maintain related - cables environment - medium voltage - environment - degraded cable - voltage cables cables - maintain normally energized - manhole - water submergence - flooded
69 27 degradation radiological barrier - radiological - degradation - barrier provided - aegts - represents - low represent degradation - provided - secondary containment inoperable - provided auxiliary building
70 26 specification allowed outage - greater technical specification - represent single train - actual single - deficiency represent single - train violation 10 - time noncited -
unavailability train - systems actual - allowed 71 25 operated valves movs - related motor - operated - motor operated valve - thermal overload relays - owners - prevent movs performing - able demonstrate tols - valve mov lacked - circuits sized 72 23 violation verify - account instrument uncertainty - palo verde unit - limits surveillance established - opening porv unexpectedly - quality accordance non - methodology rods insert - maintain motor actuator - lowered inventory vessel - louvers notice
73 22 emergency lighting units - elus - post shutdown - battery packs - install emergency - manual operation - install - safe - appendix emergency - lighting affects ability
74 22 concern team low - frum - nrc inspection nuclear - faulty level gage - factors nextera appropriately - occurred june 12 - non sw - facilities violation - fitting fixed core -
notification manner
22 T5 Summaries (Custom Stopwords Removed, POS + MMR Topic Representation)
Representative Documents
Topic 66 a licensee's vendor examiner failed to perform an ultrasonic (UT) examination. if left (31 documents) uncorrected, it would have the potential to lead to a more significant safety concern.
inspectors answered no to all of the characterizations worksheet questions.
ultrasonic - weld -
examination - ultrasonic a self-revealed finding of very low safety significance (i.e., Green) was identified for the examinations - welding - licensees failure to have adequately implemented established welding requirements.
rejectable flaw - Specifically, plant personnel conducting welding on a steam line for the Unit 1 main turbine characterizations - welded over and blocked a sensing line for an instrument manifold. the failure to execute ultrasonic testing - required weld preparation activities as prescribed by CC-AA-501-1021, Ex qualified weld - cracks
inspectors identify two examples of failure to properly evaluate inservice inspection related corrective actions. a licensee failed to evaluate an indication found during a 1999 ultrasonic examination of a core weld. the safety significance of these issues was considered very low. failure to promptly identify and correct issues found during the examination was considered a non-cited violation.
23 T5 Summaries (Custom Stopwords Removed, POS + MMR Topic Representation)
Representative Documents
Topic 74 inspectors found thermal overload relays on safety-related motor-operated valve circuits (23 documents) were not sized properly. finding was of very low safety significance because it was a design deficiency. licensee was able to demonstrate that the TOLs would not prevent any MOVs from performing safety function.
motor - thermal overload - overload -
valves - jogging - inspectors found thermal overload relays on safety-related motor-operated valve circuits systemrestoration were not sized properly. licensee was able to demonstrate that the TOLs would not prevent alignment - repetitive any MOVs from performing their safety function. the issue was of very low safety jogging - valve - significance because it was a design deficiency.
established acceptance
- bypass
the team identified a finding having very-low significance and an associated Non -Cited Violation of Title 10 of the Code of Federal Regulations, Part 50. Specifically, Dresden had not verified that thermal overload relays (TOLs) on Unit 3 safety-related motor operated valves (MOVs) were properly sized. the team determined the performance deficiency was more than minor because it was associated with the design control attribute of the
24 T5 Summaries (Custom Stopwords Removed, POS + MMR Topic Representation)
To p i c Count Top_n_words
-1 5230 pump - water - feedwater - valve - p ower - turbine - unit - service - trip - cited violation
0 908 significant concern - concern - cutting - cross - operability - aspect - quality - deficiency - cutting aspect - syste m s
1 817 radiation - dose - high radiation - worker - occupational radiation - area - w o r ke rs - rate s - exposure - millirem
2 682 protection - license - door - combustibles - watch - brigade - suppression - combustible materials - materials - sprinkler
3 488 cutting - cross - aspect - cutting aspect - cutting area - significant concern - syste m s - attribute - area - concern
4 405 diesel - generator - edg - emergency - ge n e rato rs - oil - start - jacket - leak - division
5 381 personnel - staff - implement - attribute - maintenance - wall - team - barrier - outage - operability
6 366 noncited violation - criterion - violation - cited violation - qualification - deficiency - personnel - engineering - evaluations - inspections
7 364 self - team - trip - cutting aspect - aspect - risk - functionality - likelihood - mitigation - operability
8 341 severity - corrective action - action - appendix - criterion - cited violation - violation - quality - level - team
9 214 technical specification - specification - noncited violation - cited violation - violation - examples - step - o p e rato rs - guidance - operating
10 186 valves - valve - check - position - testing - s t ro ke - torque - stem - motor - actuator
25 T5 Summaries (Custom Stopwords Removed, POS + MMR Topic Representation)
To p i c Count Top_n_words
65 32 cooling - component - water - vital component - cooler - capillary line - refill - threads - primary component - temporary modification
66 31 ultrasonic - weld - examination - ultrasonic examinations - welding - rejectable flaw - characterizations - ultrasonic testing - qualified weld - cracks
67 30 effectiveness - classification - emergency - level - plan - change - thresholds - basis - unusual event - scheme
68 29 damper - dampers - ventilation - charcoal - previous damper - overhaul - effluents - radiological barrier - degraded damper - available operating
69 29 radiological barrier - degradation - barrier - necessary vacuum - open configuration - secondary containment - minutes - auxiliary building - structure - self
70 28 cables - cable - environment - submergence - medium voltage - physical environment - degraded cable - manhole - submersion - underground manholes
71 28 molded case - case - breakers - circuit - 1e - class - life - second example - significant safetyconcern - safetyconcern
72 25 single train - train - outage - time - nondestructive testing - noncited violation - technical basis - conservative bias - bias - qualification
73 24 lighting - lights - units - p acks - emergency - manual operation - safe shutdown - shutdown - light - adequate emergency
74 23 motor - thermal overload - overload - valves - jogging - systemrestoration alignment - repetitive jogging - valve - established acceptance - bypass
75 23 devise - current protection - adequate root - flange - legacy - correction - techniques - manipulation - weakness - protection
76 20 welders - welding - weld - interpass - weld s - preheat - search - examination - adequate search - volumetric inspection
26 Takeaways from Topic Modeling with Machine Generated Summaries
Topic terms are more closely related to representative documents after using condensed summaries of long documents Increased appearance of stopwords when allowing n-grams in topic terms because there is less common text present between summaries of the documents Some summaries dont cover the technical issue but focus on regulations/violations or cornerstone/cross-cutting areas so the discovered topic terms are sometimes vague (cited or non cited violation, technical specification, personnel, staff, requirement, criterion, appendix, systems)
27 Safety Cluster Alignment
28 Safety Clusters sorted and color coded by number of findings (5 years)
29 Safety Cluster #2 - Limited number of inspection procedures
30 Drilling Down - Cornerstones, sites, and reports
31 Drilling Down - Safety Cluster information, summary, and entities
Title:
Failure to Verify the Adequacy of the Air Pressure Regulator Setpoint Value for Containment Isolation Valves 1(2)RF026
Cluster Key Terms: valves - valve - isolation - steam - containment - pathway - integrity - actuator - check Document Count: total = 741 5 years = 65
Machine Generated Summary: Inspectors identified a Green finding and an associated NCV of 10 CFR Part 50, Appendix B, Criteri on III, Design Control, for the failure to verify the adequacy of the air pressure regulator setpoint value for air -operated containment isolation valves 1(2)RF 026. The finding screened as of very low safety significance (Green) because it did not represent an actual open pathway.
32 Sites sorted and color coded (5 years)
- of Findings Safety Clusters with # of Findings Site -> Cluster -> Procedure
33 Progress
34 SOW Task Status
Phase I: March 6, 2023 - April 9, 2023 Status Phase II: March 20, 2023 - May 7, 2023 Status Describe the Problem Complete Platform/system selection and installation In progress Search the Literature Complete Data acquisition and preparation In progress Select Candidates CompleteFeature pipeline engineering In progress Select Evaluation Factors Complete Clustering method experimentation & selection In progress Develop evaluation factor weights Complete Cluster pipeline engineering In progress Define evaluation factor ranges Complete Anomaly detection (as needed) Not needed Perform assessment Complete Model Development, Training, Evaluation In progress Report Results CompleteTest harness development In progress Deliver Trade study report Complete PoC integration and demonstration In progress Trial runs and evaluation In progress Demonstrate PoC capability Not started
Phase III: April 19, 2023 - June 16, 2023 Status Live data ingestion In progress Model execution In progress Cluster evaluation In progress Critical Method documentation Not started Technical Report Document Not started Deliver final report with findings Not started
35 Next Steps
36 Next Steps
Safety Cluster Formation
- Assign outlier documents to safety clusters - use metrics to guide quality Topic Modeling Summaries
- Summaries from other models (flan-t5, pgasus-cnn-daily) for all inspection findings item introductions
- Exploring other models that were designed for NLP tasks with long input sequences (Long T5, Longformer, Reformer, Big Bird, XLNet ) (stretch)
- Use noun chunks (from SpaCy) as tokens instead of single words or bi-grams/tri -grams (stretch)
- Topic modeling with full inspection report text (stretch)
Metrics
- Evaluate Coherence and Diversity metrics to better evaluate topic models
- Add new corpi of text as needed Deliverables
- Consolidate experiments, inputs, and outputs to deliverable format
37