Image sharpening method of automatic loading and unloading drill pipe target in underground coal mine

Authors

YAO Ningping, CCTEG Xi’an Research Institute (Group) Co., Ltd., Xi’an 710077, China; China Coal Research Institute, Beijing 100013, ChinaFollow
LIANG Chunmiao, CCTEG Xi’an Research Institute (Group) Co., Ltd., Xi’an 710077, ChinaFollow
YAO Yafeng, CCTEG Xi’an Research Institute (Group) Co., Ltd., Xi’an 710077, China; China Coal Research Institute, Beijing 100013, China
SONG Haitao, CCTEG Xi’an Research Institute (Group) Co., Ltd., Xi’an 710077, China

Abstract

Due to the space limitation of underground roadway in coal mine, the drilling robot is designed with two independent systems, i.e., the drilling system and the drilling pipe loading and unloading system, which are in flexible layout. Using the monocular visual identification based on cooperative target, the drilling system is associated with the drilling pipe loading and unloading system for the cooperative operation of the two systems to deliver the drill pipe. However, there are problems such as borehole cinder, cooperative target polluted by sewage, high dust concentration, low visibility, low luminance of artificial light source, uneven illumination resulting in low contrast of target image, and the reflection of cooperative target under the direct lighting of mining lamp in the complex scenarios of drilling construction in the underground coal mine, which will affect the resolution of cooperative target image and reduce the accuracy of target identification and the position to load and unload the drill pipe, thus influencing the success rate of automatic loading and unloading of drill pipe by drilling robot. Therefore, in view of the problem of cooperative target pollution by coal cinder and coal slime returned from the borehole in construction, the open operation of first corrosion and then expansion in morphological filtering was used to remove the discontinuous patches and prominent target pixels of coal cinder. Meanwhile, the closing operation of first expansion and then corrosion was also adopted to remove the small holes and intermittent small gaps, thereby eliminating the impact of solid pollutants on the cooperative targets. Besides, to eliminate the effect of dust, rock dust, water mist and sewage on image contrast, the initial transmittance was obtained by dark channel prior, the transmittance was refined by gradient domain guided filtering, the dark channel was improved by the guided filtering, and the atmospheric light value was solved. On this basis, the defogged image was obtained with the atmospheric scattering model, so as to realize the purpose of sharpening the image and enhancing the image contrast. Finally, sub-pixel corners and directions were refined, and the energy function was optimized. In addition, the seed checkerboard was used to grow checkerboard along the four edges, and the energy changes were used to confirm whether the largest checkerboard was grown, and thus the missing cooperative target image due to strong light reflection was completed. The experimental results show that the combination of morphological filtering, defogging algorithm and growth based checkerboard corner detection method could enhance the resolution of the cooperative target image in the complex scenarios of drilling construction in underground coal mine and improve the image accuracy based on visual identification, laying a foundation for the precise control of automatic loading and unloading of drill pipe by the drilling robot.

Keywords

automatic loading and unloading drill pipes, cooperation target, morphological filtering, defogging algorithm, growth based checkerboard corner detection method, image sharpening, underground coal mine

DOI

10.12363/issn.1001-1986.22.11.0885

Recommended Citation

YAO Ningping, LIANG Chunmiao, YAO Yafeng, et al. (2023) "Image sharpening method of automatic loading and unloading drill pipe target in underground coal mine," Coal Geology & Exploration: Vol. 51: Iss. 5, Article 19.
DOI: 10.12363/issn.1001-1986.22.11.0885
Available at: https://cge.researchcommons.org/journal/vol51/iss5/19

Reference

[1] 董洪波,姚宁平,马斌,等. 煤矿井下坑道钻机电控自动化技术研究[J]. 煤田地质与勘探,2020,48(3):219−224.

DONG Hongbo,YAO Ningping,MA Bin,et al. Research on electronically controlled automation technology of underground drilling rig for coal mine[J]. Coal Geology & Exploration,2020,48(3):219−224.

[2] 罗光强,李扬,周策,等. 深孔智能化钻井参数远程监控及工况识别系统研究[J]. 地质与勘探,2022,58(3):696−703.

LUO Guangqiang,LI Yang,ZHOU Ce,et al. Intelligent remote monitoring and the work-condition identification system for deep drilling[J]. Geology and Exploration,2022,58(3):696−703.

[3] 王清峰,陈航. 瓦斯抽采智能化钻探技术及装备的发展与展望[J]. 工矿自动化,2018,44(11):18−24.

WANG Qingfeng,CHEN Hang. Development and prospect on intelligent drilling technology and equipment for gas drainage[J]. Industry and Mine Automation,2018,44(11):18−24.

[4] AUGENSTEIN S，ROCK S M. Improved frame–to–frame pose tracking during vision–only SLAM/SFM with a tumbling target[C]//2011 IEEE International Conference on Robotics and Automation (ICRA). Shanghai，2011：3131–3138.

[5] DONG Gangqi,ZHU Z H. Position–based visual servo control of autonomous robotic manipulators[J]. Acta Astronautica,2015,115:291−302.

[6] WEN Zhuoman,WANG Yanjie,KUIJPER A,et al. On−orbit real−time robust cooperative target identification in complex background[J]. Chinese Journal of Aeronautics,2015,28(5):1451−1463.

[7] WEN Zhuoman,WANG Yanjie,LUO Jun,et al. Robust,fast and accurate vision–based localization of a cooperative target used for space robotic arm[J]. Acta Astronautica,2017,136:101−114.

[8] 王晓鸣,吴高升. 基于单目视觉的水下机器人相对位姿精确控制[J]. 水下无人系统学报,2021,29(3):299−307.

WANG Xiaoming,WU Gaosheng. Relative position and attitude precise control of underwater robot based on monocular vision[J]. Journal of Unmanned Undersea Systems,2021,29(3):299−307.

[9] 雷孟宇,张旭辉,杨文娟,等. 煤矿掘进装备视觉位姿检测与控制研究现状与趋势[J]. 煤炭学报,2021,46(增刊2):1135−1148.

LEI Mengyu,ZHANG Xuhui,YANG Wenjuan,et al. Current status and trend of research on visual pose detection and control of heading equipment in coal mines[J]. Journal of China Coal Society,2021,46(Sup.2):1135−1148.

[10] 吴佳奇，张文琪，陈伟，等. 基于改进CycleGAN的煤矿井下低照度图像增强方法[J/OL]. 华中科技大学学报(自然科学版)，2022：1–10 [2022-11-16]. DOI：10.13245/j.hust.229323.

WU Jiaqi，ZHANG Wenqi，CHEN Wei，et al. Image enhancement method of low illumination in coal mine based on improved CycleGAN[J/OL]. Journal of Huazhong University of Science and Technology (Natural Science Edition)，2022：1–10 [2022-11-16]. DOI：10.13245/j.hust.229323.

[11] 智宁,毛善君,李梅. 基于双伽马函数的煤矿井下低亮度图像增强算法[J]. 辽宁工程技术大学学报(自然科学版),2018,37(1):191−197.

ZHI Ning,MAO Shanjun,LI Mei. An enhancement algorithm for coal mine low illumination images based on bi–Gamma function[J]. Journal of Liaoning Technical University(Natural Science),2018,37(1):191−197.

[12] 张谢华,张申,方帅,等. 煤矿智能视频监控中雾尘图像的清晰化研究[J]. 煤炭学报,2014,39(1):198−204.

ZHANG Xiehua,ZHANG Shen,FANG Shuai,et al. Clearing research on fog and dust images in coalmine intelligent video surveillance[J]. Journal of China Coal Society,2014,39(1):198−204.

[13] 王启明,李季. 煤矿井下高清图像快速去雾算法研究[J]. 小型微型计算机系统,2018,39(11):2557−2560.

WANG Qiming,LI Ji. Study on fast haze removal algorithm for underground high definition image[J]. Journal of Chinese Computer Systems,2018,39(11):2557−2560.

[14] 曹陇鑫,马宗方,石晶. 基于Retinex的自适应非均匀低照度图像增强算法[J]. 计算机测量与控制,2020,28(10):155−159.

CAO Longxin,MA Zongfang,SHI Jing. Retinex–based adaptive non–uniform low light image enhancement algorithm[J]. Computer Measurement & Control,2020,28(10):155−159.

[15] 马丽娜. 基于机器视觉的磁体器件表面缺陷识别方法研究[D]. 西安：西安理工大学，2021.

MA Lina. Research on surface defect recognition method of magnet device based on machine vision[D]. Xi’an：Xi’an University of Technology，2021.

[16] 齐咏生,樊佶,李永亭,等. 基于增强型形态学滤波的风电机组轴承故障诊断方法[J]. 振动与冲击,2021,40(4):212−220.

QI Yongsheng,FAN Ji,LI Yongting,et al. A fault diagnosis method of wind turbine bearings based on an enhanced morphological filter[J]. Journal of Vibration and Shock,2021,40(4):212−220.

[17] HE Kaiming，SUN Jian，TANG Xiao’ou. Single image haze removal using dark channel prior[C]//2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2009). Miami，2009：1956–1963.

[18] HE Kaiming,SUN Jian,TANG Xiao’ou. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2013,35(6):1397−1409.

[19] KOU Fei,CHEN Weihai,WEN Changyun,et al. Gradient domain guided image filtering[J]. IEEE Transactions on Image Processing,2015,24(11):4528−4539.

[20] 火元莲,郑海亮,李明,等. 基于融合亮度模型和梯度域滤波的图像去雾[J]. 计算机工程与科学,2021,43(9):1623−1633.

HOU Yuanlian,ZHENG ailian,LI Ming,et al. Image dehazing based on fusiong luminance model and gradient domain filter[J]. Computer Eengineering & Science,2021,43(9):1623−1633.

[21] 王健,何自豪,刘洁,等. 基于梯度域导向滤波器和改进PCNN的图像融合算法[J]. 系统工程与电子技术,2022,44(8):2381−2392.

WANG Jian,HE Zihao,LIU Jie,et al. Image fusion algorithm based on gradient domain guided filtering and improved PCNN[J]. Systems Engineering and Electronics,2022,44(8):2381−2392.

[22] GEIGER A，MOOSMANN F，CAR O，et al. Automatic camera and range sensor calibration using a single shot[C]//2012 IEEE International Conference on Robotics & Automation. Saint Paul，2012：3936–3943.

[23] RUFLI M，SCARAMUZZA D，SIEGWART R. Automatic detection of checkerboards on blurred and distorted images[C]//2008 IEEE/RSJ International Conference on Intelligent Robots and Systems. Nice，2008.

[24] ORGHIDAN R,MOUADDIB E M,SALVI J,et al. Catadioptric single–shot rangefinder for textured map building in robot navigation[J]. IET Computer Vision,2007,1(2):43−53.