An Attention mechanism-based CNN-BiLSTM real-time transient electromagnetic method

Authors

GU Yao, College of Geology and Environment, Xi’an University of Science and Technology, Xi’an 710054, China; Key Laboratory of Coal Resources Exploration and Comprehensive Utilization, Ministry of Natural Resources, Xi’an 710048, ChinaFollow
XIE Haijun, College of Geology and Environment, Xi’an University of Science and Technology, Xi’an 710054, China
ZHOU Zipeng, Key Laboratory of Coal Resources Exploration and Comprehensive Utilization, Ministry of Natural Resources, Xi’an 710048, China
LI Lu, College of Geology and Environment, Xi’an University of Science and Technology, Xi’an 710054, China

Abstract

The one-dimensional transient electromagnetic (TEM) method is time-consuming and suffers other drawbacks such as difficult parameter adjustment and heavy dependence on the initial model. Therefore, this study proposed a real-time TEM inversion method—the Attention mechanism-based convolutional neural network (CNN) - bidirectional Long Short-Term Memory (BiLSTM) (AC-BiLSTM). By fully utilizing the time difference, the AC-BiLSTM performed model training in non-observation time and the real-time inversion of the collected data in observation time. With the measured data mixed with a certain proportion of data obtained from forward modeling as the dataset, the sampling time and apparent resistivity as the input features in the form of supervised learning, and the log-constrained Occam inversion results as the learning target, the whole process of the AC-BiLSTM method is as follows: (1) the encoder-decoder model is established based on CNN and LSTM; (2) based on the data characteristics, the Attention mechanism is added to the decoder to extract the output data from the hidden layer; (3) finally, the depth-resistivity data are obtained from the fully connected layer. The study results indicate that the AC-BiLSTM algorithm can fully dig out the spatio-temporal characteristics of data and quickly yield resistivity images that meet the electrical characteristics of strata. The predicted values of the AC-BiLSTM algorithm on the TEM dataset obtained from forward modeling showed a goodness of fit of 0.898 with the forward model, with root mean squared error of 18.44 and an average relative error of 0.065. Furthermore, compared to the single LSTM neural network and the Occam method, the AC-BiLSTM algorithm showed that the goodness of fit was improved by 0.086 and 0.176, respectively, the root mean squared error was reduced by 2.97 and 9.32, and the average relative error was reduced by 0.012 and 0.068, respectively. The AC-BiLSTM inversion of measured TEM data from the V8 Receiver enabled the quick and accurate stratification of strata in the study area and the delineation of the distribution range of coal mine goaf, with the obtained results consistent with the actual situation. Research results break through the limitations of traditional inversion methods and improve the accuracy and defficiency of transient electromagnetic data interpretion.

Keywords

transient electromagnetic method (TEM), Convolutional Neural Network (CNN), Long Short-Term Memory (LSTM) Neural Network, Attention mechanism, inversion

DOI

10.12363/issn.1001-1986.22.12.1000

Recommended Citation

GU Yao, XIE Haijun, ZHOU Zipeng, et al. (2023) "An Attention mechanism-based CNN-BiLSTM real-time transient electromagnetic method," Coal Geology & Exploration: Vol. 51: Iss. 10, Article 15.
DOI: 10.12363/issn.1001-1986.22.12.1000
Available at: https://cge.researchcommons.org/journal/vol51/iss10/15

Reference

[1] 刘明宏,蔡红柱,杨浩,等. 地面与半航空瞬变电磁法三维联合反演[J]. 地球物理学报,2022,65(10):3997−4011.

LIU Minghong,CAI Hongzhu,YANG Hao,et al. Three–dimensional joint inversion of ground and semi–airborne transient electromagnetic method[J]. Chinese Journal of Geophysics,2022,65(10):3997−4011.

[2] 张小娟. 基于BP神经网络的电法勘探反演成像[D]. 湘潭：湖南科技大学，2016.

ZHANG Xiaojuan. Inversion imaging of electrical prospecting based on BP neural network[D]. Xiangtan：Hunan University of Science and Technology，2016.

[3] LI Ruiyou,ZHANG Huaiqing,YU Nian,et al. A fast approximation for 1–D inversion of transient electromagnetic data by using a back propagation neural network and improved particle swarm optimization[J]. Nonlinear Processes in Geophysics,2019,26:445−456.

[4] XUE Guoqiang,LI Hai,HE Yiming,et al. Development of the inversion method for transient electromagnetic data[J]. IEEE Access,2020,8:146172−146181.

[5] 田宵,汪明军,张雄,等. 基于多输入卷积神经网络的天然地震和爆破事件识别[J]. 地球物理学报,2022,65(5):1802−1812.

TIAN Xiao,WANG Mingjun,ZHANG Xiong,et al. Discrimination of earthquake and quarry blast based on multi–input convolutional neural network[J]. Chinese Journal of Geophysics,2022,65(5):1802−1812.

[6] 王泽峰,许辉群,杨梦琼,等. 时域卷积神经网络地震波阻抗反演因素影响的研究[J]. 地球物理学进展,2022,37(5):2062−2071.

WANG Zefeng,XU Huiqun,YANG Mengqiong,et al. Study on the influence of preprocessing and hyper parameters on temporal convolutional network seismic impedance inversion[J]. Progress in Geophysics,2022,37(5):2062−2071.

[7] 王竟仪,王治国,陈宇民,等. 深度人工神经网络在地震反演中的应用进展[J]. 地球物理学进展,2023,38(1):298−320.

WANG Jingyi,WANG Zhiguo,CHEN Yumin,et al. Deep artificial neural network in seismic inversion[J]. Progress in Geophysics,2023,38(1):298−320.

[8] 李实,闫述,李创社,等. 人工神经网络专家系统在瞬变电磁法反演中的应用[J]. 煤田地质与勘探,2001,29(6):48−51.

LI Shi,YAN Shu,LI Chuangshe,et al. The application of artificial neural network expert system to the transient electromagnetic method inversion[J]. Coal Geology & Exploration,2001,29(6):48−51.

[9] 王秀臣. 基于人工神经网络的瞬变电磁反演解释及应用研究[D]. 西安：西北大学，2006.

WANG Xiuchen. Study on inversion and interpretation of the transient electromagnetic method based on artificial neural network and its applications[D]. Xi’an：Northwest University，2006.

[10] 秦善强. 瞬变电磁法的神经网络快速成像及其在接地网检测中的应用[D]. 重庆：重庆大学，2019.

QIN Shanqiang. Rapid apparent resistivity imaging of transient electromagnetic using ANN and application in grounding grid detection[D]. Chongqing：Chongqing University，2019.

[11] 闫国才. 基于神经网络的全空间瞬变电磁法岩层富水性预测研究[J]. 能源与环保,2020,42(12):81−84.

YAN Guocai. Research on prediction of full−space transient electromagnetic method of rock format water richness based on neural network[J]. China Energy and Environmental Protection,2020,42(12):81−84.

[12] 范涛,薛国强,李萍,等. 瞬变电磁长短时记忆网络深度学习实时反演方法[J]. 地球物理学报,2022,65(9):3650−3663.

FAN Tao,XUE Guoqiang,LI Ping,et al. TEM real–time inversion based on long−short term memory network[J]. Chinese Journal of Geophysics,2022,65(9):3650−3663.

[13] 任建吉,位慧慧,邹卓霖,等. 基于CNN–BiLSTM–Attention的超短期电力负荷预测[J]. 电力系统保护与控制,2022,50(8):108−116.

REN Jianji,WEI Huihui,ZOU Zhuolin,et al. Ultra–short–term power load forecasting based on CNN–BiLSTM–Attention[J]. Power System Protection and Control,2022,50(8):108−116.

[14] 袁英淏. 基于深度学习的地震储层图像解释方法研究[D]. 成都：电子科技大学，2020.

YUAN Yinghao. Research of seismic reservoir image interpretation method based on deep learning[D]. Chengdu：University of Electronic Science and Technology of China，2020.

[15] CAO Wei,GUO Xuebao,TIAN Feng,et al. Seismic velocity inversion based on CNN−LSTM fusion deep neural network[J]. Applied Geophysics,2021,18(4):499−514.

[16] 冼锦炽,蔡红柱,熊咏春,等. 基于深度学习的地面拖曳式瞬变电磁快速成像方法[J]. 工程地球物理学报,2022,19(4):536−545.

XIAN Jinchi,CAI Hongzhu,XIONG Yongchun,et al. Ground–based towed transient electromagnetic imaging method based on deep learning[J]. Chinese Journal of Engineering Geophysics,2022,19(4):536−545.

[17] 曹连雨. 基于深度卷积神经网络的遥感影像目标检测技术研究及应用[D]. 北京：北京科技大学，2021.

CAO Lianyu. Research and application of object detection of remote sensing images based on deep convolution neural network[D]. Beijing：University of Science and Technology Beijing，2021.

[18] 廖晓龙,张志厚,姚禹,等. 基于卷积神经网络的大地电磁反演[J]. 中南大学学报(自然科学版),2020,51(9):2546−2557.

LIAO Xiaolong,ZHANG Zhihou,YAO Yu,et al. Magnetotelluric inversion based on convolutional neural network[J]. Journal of Central South University (Science and Technology),2020,51(9):2546−2557.

[19] 李佳蔚. 基于优化的卷积神经网络储层参数反演方法研究[D]. 青岛：中国石油大学(华东)，2020.

LI Jiawei. Study on reservoir parameter inversion based on optimized convolutional neural network[D]. Qingdao：China University of Petroleum (East China)，2020.

[20] 吴易智,范宜仁,巫振观,等. 基于卷积神经网络和MPGA–LM算法的阵列侧向测井快速反演方法[J]. 地球物理学报,2021,64(9):3410−3425.

WU Yizhi,FAN Yiren,WU Zhenguan,et al. A fast inversion method for array laterolog based on convolutional neural network and hybrid MPGA–LM algorithm[J]. Chinese Journal of Geophysics,2021,64(9):3410−3425.

[21] 范振宇. 基于卷积神经网络的大地电磁深度学习反演研究[D]. 北京：中国地质大学(北京)，2020.

FAN Zhenyu. Magnetotelluric deep learning inversion based on convolutional neural network[D]. Beijing：China University of Geosciences (Beijing)，2020.

[22] 王琪凯,熊永康,陈瑛,等. 基于Attention机制优化CNN–seq2seq模型的非侵入式负荷监测[J]. 电力系统及其自动化学报,2022,34(12):27−34.

WANG Qikai,XIONG Yongkang,CHEN Ying,et al. Non–intrusive load monitoring based on CNN–seq2seq model optimized by Attention mechanism[J]. Proceedings of the CSU–EPSA,2022,34(12):27−34.

[23] 张海涛,杨小明,陈阵,等. 基于增强双向长短时记忆神经网络的测井数据重构[J]. 地球物理学进展,2022,37(3):1214−1222.

ZHANG Haitao,YANG Xiaoming,CHEN Zhen,et al. Log data reconstruction method based on enhanced bidirectional long short–term memory neural network[J]. Progress in Geophysics,2022,37(3):1214−1222.

[24] 梁耍,王世博,谢洋,等. 基于LSTM的煤层厚度动态预测方法研究[J]. 煤炭科学技术,2021,49(增刊1):150−157.

LIANG Shua,WANG Shibo,XIE Yang,et al. Dynamic prediction method of coal seam thickness based on LSTM[J]. Coal Science and Technology,2021,49(Sup.1):150−157.

[25] 查文舒,乔奇,刘子雄,等. 基于相关性分析的Bi–LSTM测井曲线预测方法[J]. 合肥工业大学学报(自然科学版),2022,45(5):700−706.

ZHA Wenshu,QIAO Qi,LIU Zixiong,et al. A method for well log prediction using Bi–LSTM based on correlation analysis[J]. Journal of Hefei University of Technology (Natural Science),2022,45(5):700−706.

[26] SUEBSOMBUT P,SEKHARI A,SUREEPHONG P,et al. Field data forecasting using LSTM and Bi–LSTM approaches[J]. Applied Sciences,2021,11(24):11820.

[27] 唐菲菲,唐天俊,朱洪洲,等. 结合注意力机制和Bi–LSTM的降雨型滑坡位移预测[J]. 测绘通报,2022(9):74−79.

TANG Feifei,TANG Tianjun,ZHU Hongzhou,et al. Rainfall landslide deformation prediction based on attention mechanism and Bi–LSTM[J]. Bulletin of Surveying and Mapping,2022(9):74−79.

[28] 吴汉瑜,严江,黄少滨,等. 用于文本分类的CNN–BiLSTM–Attention混合模型[J]. 计算机科学,2020,47(增刊2):23−27.

WU Hanyu,YAN Jiang,HUANG Shaobin,et al. CNN–BiLSTM–Attention hybrid model for text classification[J]. Computer Science,2020,47(Sup.2):23−27.

[29] 潘信亮,杨仁辉,江涛,等. 基于Bi–LSTM的近岸水体深度反演[J]. 光学学报,2021,41(10):1012003.

PAN Xinliang,YANG Renhui,JIANG Tao,et al. Depth inversion of coastal waters based on Bi–LSTM[J]. Acta Optica Sinica,2021,41(10):1012003.

[30] 曾庆田,吕珍珍,石永奎,等. 基于Prophet+LSTM模型的煤矿井下工作面矿压预测研究[J]. 煤炭科学技术,2021,49(7):16−23.

ZENG Qingtian,LYU Zhenzhen,SHI Yongkui,et al. Research on prediction of underground coal mining face pressure based on Prophet+LSTM model[J]. Coal Science and Technology,2021,49(7):16−23.

[31] 朱丽,杨青,吴涛,等. 基于CNN和Bi–LSTM的脑电波情感分析[J]. 应用科学学报,2022,40(1):1−12.

ZHU Li,YANG Qing,WU Tao,et al. Emotional analysis of brain waves based on CNN and Bi–LSTM[J]. Journal of Applied Sciences,2022,40(1):1−12.