Research on digital twin-driven tracking, positioning and collision detection method of excavating and anchoring equipment

Authors

YANG Wenjuan, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi’an 710054, ChinaFollow
ZHAO Dian, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
ZHANG Xuhui, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi’an 710054, ChinaFollow
CHEN Xin, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
ZHENG Xili, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
LIU Yanhui, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
ZHANG Ran, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
WAN Jicheng, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China

Abstract

The automation of excavating and anchoring operation is critical for intelligent excavation of coal mine roadway. Aiming at the challenges of relative pose measurement and collision detection during the alternating operation of the current excavating and anchoring equipment, a digital twin-driven tracking, positioning and collision detection method was proposed. Firstly, in order to overcome the influence of low illumination, high dust and complex background interference in the underground excavating face, the multi-point infrared LED target is taken as the information source, the infrared LED feature point image is collected by industrial camera, the center of the spot is extracted using the Hough contour detection and the center of mass method, the ID of the target is identified by binary coding, and the improved sparse optical flow method is adopted to track the spot. Meanwhile, a PNP-based pose solution model of excavating and anchoring equipment is established, and the relative pose of equipment is obtained using a dual quaternion. Secondly, the digital twin technology is utilized to establish the digital twin model of the excavating and anchoring equipment and working face at the actual size based on the Unity3D platform. The real-time data transmission and exchange between the virtual space and the physical entity is realized by Socket communication, the 3D visualization of the real-time pose of the excavating and anchoring equipment in the virtual space is achieved, and the virtual collision detection of the excavating and anchoring equipment is implemented in combination with the oriented bounding box (OBB) collision detection algorithm. Finally, an experimental platform is set up to complete the pose measurement test of the excavating and anchoring equipment, and at the same time, the virtual and real movement trajectories and collision detection effect are verified. The experimental results show that the position and angle errors of the tracking and positioning experiment of the excavating and anchoring equipment are less than 20 mm and 0.30° respectively. In the comparison of virtual and real position coordinates, the maximum error in the X-axis direction does not exceed 1.14 mm, the maximum error in the Y-axis direction does not exceed 1.10 mm, which can ensure the virtual and real consistency and synchronization of the system, meeting the requirements of real-time tracking, positioning and collision detection of the excavating and anchoring equipment during the operation of the excavating face

Keywords

digital twin, excavating and anchoring equipment, pose measurement, collision early warning, sparse optical flow algorithm, coal mine

DOI

10.12363/issn.1001-1986.24.02.0097

Recommended Citation

YANG Wenjuan, ZHAO Dian, ZHANG Xuhui, et al. (2024) "Research on digital twin-driven tracking, positioning and collision detection method of excavating and anchoring equipment," Coal Geology & Exploration: Vol. 52: Iss. 5, Article 17.
DOI: 10.12363/issn.1001-1986.24.02.0097
Available at: https://cge.researchcommons.org/journal/vol52/iss5/17

Reference

[1] 王双明，耿济世，李鹏飞，等. 煤炭绿色开发地质保障体系的构建[J]. 煤田地质与勘探，2023，51(1)：33−43.

WANG Shuangming，GENG Jishi，LI Pengfei，et al. Construction of geological guarantee system for green coal mining[J]. Coal Geology & Exploration，2023，51(1)：33−43.

[2] 葛世荣，王世博，管增伦，等. 数字孪生：应对智能化综采工作面技术挑战[J]. 工矿自动化，2022，48(7)：1−12.

GE Shirong，WANG Shibo，GUAN Zenglun，et al. Digital twin：Meeting the technical challenges of intelligent fully mechanized working face[J]. Journal of Mine Automation，2022，48(7)：1−12.

[3] 王国法，张建中，薛国华，等. 煤矿回采工作面智能地质保障技术进展与思考[J]. 煤田地质与勘探，2023，51(2)：12−26.

WANG Guofa，ZHANG Jianzhong，XUE Guohua，et al. Progress and reflection of intelligent geological guarantee technology in coal mining face[J]. Coal Geology & Exploration，2023，51(2)：12−26.

[4] 王海军，曹云，王洪磊. 煤矿智能化关键技术研究与实践[J]. 煤田地质与勘探，2023，51(1)：44−54.

WANG Haijun，CAO Yun，WANG Honglei. Research and practice on key technologies for intelligentization of coal mine[J]. Coal Geology & Exploration，2023，51(1)：44−54.

[5] 马宏伟，王世斌，毛清华，等. 煤矿巷道智能掘进关键共性技术[J]. 煤炭学报，2021，46(1)：310−320.

MA Hongwei，WANG Shibin，MAO Qinghua，et al. Key common technology of intelligent heading in coal mine roadway[J]. Journal of China Coal Society，2021，46(1)：310−320.

[6] 张旭辉，杨文娟，薛旭升，等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报，2022，47(1)：579−597.

ZHANG Xuhui，YANG Wenjuan，XUE Xusheng，et al. Challenges and developing of the intelligent remote control on roadheaders in coal mine[J]. Journal of China Coal Society，2022，47(1)：579−597.

[7] 陶云飞，杨健健，李嘉赓，等. 基于惯性导航技术的掘进机位姿测量系统研究[J]. 煤炭技术，2017，36(1)：235−237.

TAO Yunfei，YANG Jianjian，LI Jiageng，et al. Research on position and orientation measurement system of heading machine based on inertial navigation technology[J]. Coal Technology，2017，36(1)：235−237.

[8] 田原. 基于零速修正的掘进机惯性导航定位方法[J]. 工矿自动化，2019，45(8)：70−73.

TIAN Yuan. Inertial navigation positioning method of roadheader based on zero–velocity update[J]. Industry and Mine Automation，2019，45(8)：70−73.

[9] 刘送永，崔玉明，孟德远，等. 巷道掘进机多传感融合定位系统及试验研究[J]. 振动、测试与诊断，2023，43(3)：476–484.

LIU Songyong，CUI Yuming，MENG Deyuan，et al. Multi–sensor fusion positioning system and experimental study of roadheader[J]. Journal of Vibration，Measurement & Diagnosis，2023，43(3)：476–484.

[10] 符世琛，李一鸣，宗凯，等. 面向掘进机的超宽带位姿检测系统精度分析[J]. 仪器仪表学报，2017，38(8)：1978−1987.

FU Shichen，LI Yiming，ZONG Kai，et al. Accuracy analysis of UWB pose detection system for roadheader[J]. Chinese Journal of Scientific Instrument，2017，38(8)：1978−1987.

[11] 贾文浩，陶云飞，张敏骏，等. 基于iGPS的煤巷狭长空间中掘进机绝对定位精度研究[J]. 仪器仪表学报，2016，37(8)：1920−1926.

JIA Wenhao，TAO Yunfei，ZHANG Minjun，et al. Research on absolute positioning accuracy of roadheader based on indoor global positioning system in narrow and long coal tunnel[J]. Chinese Journal of Scientific Instrument，2016，37(8)：1920−1926.

[12] 周玲玲，董海波，杜雨馨. 基于双激光标靶图像识别的掘进机位姿检测方法[J]. 激光与光电子学进展，2017，54(4)：041205.

ZHOU Lingling，DONG Haibo，DU Yuxin. Method of roadheader position detection based on image recognition of double laser targets[J]. Laser & Optoelectronics Progress，2017，54(4)：041205.

[13] 杨文娟，张旭辉，张超，等. 基于三激光束标靶的煤矿井下长距离视觉定位方法[J]. 煤炭学报，2022，47(2)：986−1001.

YANG Wenjuan，ZHANG Xuhui，ZHANG Chao，et al. Long distance vision localization method based on triple laser beams target in coal mine[J]. Journal of China Coal Society，2022，47(2)：986−1001.

[14] 陶飞，程颖，程江峰，等. 数字孪生车间信息物理融合理论与技术[J]. 计算机集成制造系统，2017，23(8)：1603−1611.

TAO Fei，CHENG Ying，CHENG Jiangfeng，et al. Theories and technologies for cyber–physical fusion in digital twin shop–floor[J]. Computer Integrated Manufacturing Systems，2017，23(8)：1603−1611.

[15] 袁亮，张平松. 煤炭精准开采透明地质条件的重构与思考[J]. 煤炭学报，2020，45(7)：2346−2356.

YUAN Liang，ZHANG Pingsong. Framework and thinking of transparent geological conditions for precise mining of coal[J]. Journal of China Coal Society，2020，45(7)：2346−2356.

[16] 葛世荣，张帆，王世博，等. 数字孪生智采工作面技术架构研究[J]. 煤炭学报，2020，45(6)：1925−1936.

GE Shirong，ZHANG Fan，WANG Shibo，et al. Digital twin for smart coal mining workface：Technological frame and construction[J]. Journal of China Coal Society，2020，45(6)：1925−1936.

[17] 吴淼，李瑞，王鹏江，等. 基于数字孪生的综掘巷道并行工艺技术初步研究[J]. 煤炭学报，2020，45(增刊1)：506−513.

WU Miao，LI Rui，WANG Pengjiang，et al. Preliminary study on the parallel technology of fully mechanized roadway based on digital twin[J]. Journal of China Coal Society，2020，45(Sup.1)：506−513.

[18] 王学文，谢嘉成，李素华，等. VR/AR技术在智能化综采工作面建设中的应用现状与展望[J]. 智能矿山，2020，1(1)：132−136.

WANG Xuewen，XIE Jiacheng，LI Suhua，et al. Research status and development direction of VR/AR technology in intelligent fully–mechanized mining face[J]. Journal of Intelligent Mine，2020，1(1)：132−136.

[19] 杨健健，葛世荣，王飞跃，等. 平行掘进：基于ACP理论的掘–支–锚智能控制理论与关键技术[J]. 煤炭学报，2021，46(7)：2100−2111.

YANG Jianjian，GE Shirong，WANG Feiyue，et al. Parallel tunneling：Intelligent control and key technologies for tunneling，supporting and anchoring based on ACP theory[J]. Journal of China Coal Society，2021，46(7)：2100−2111.

[20] HEO Y S，LEE K M，LEE S U. Robust stereo matching using adaptive normalized cross–correlation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2011，33(4)：807−822.

[21] 于海军，马纯永，张涛，等. 基于图像空间的快速碰撞检测算法[J]. 计算机应用，2013，33(2)：530−533.

YU Haijun，MA Chunyong，ZHANG Tao，et al. Fast collision detection algorithm based on image space[J]. Journal of Computer Applications，2013，33(2)：530−533.

[22] QU Huiyan，LI Wenhui，ZHAO Wei. Human–vehicle collision detection algorithm based on image processing[J]. International Journal of Pattern Recognition and Artificial Intelligence，2020，34(8)：2055015.

[23] 李建波，潘振宽，孙志军. 基于包围盒与空间分解的碰撞检测算法[J]. 计算机科学，2005，32(6)：155−157.

LI Jianbo，PAN Zhenkuan，SUN Zhijun. The collision detection algorithm based on combination of bounding volumes and space division[J]. Computer Science，2005，32(6)：155−157.

[24] 张建平，李丁，胡振中. 一种集成空间分解与占用的精确碰撞检测算法及其在建筑工程中的应用[J]. 工程力学，2014，31(5)：79−85.

ZHANG Jianping，LI Ding，HU Zhenzhong. An accurate collision detection algorithm integrating spatial decomposition and spatial occupancy and its applications in constructions[J]. Engineering Mechanics，2014，31(5)：79−85.

[25] 张宇，张得礼，张文奇，等. 基于混合层次包围盒的水下训练机械臂碰撞检测方法研究[J]. 载人航天，2022，28(5)：627−636.

ZHANG Yu，ZHANG Deli，ZHANG Wenqi，et al. Researchon collision detection method of underwater training manipulator based on hybrid hierarchical bounding box[J]. Manned Spaceflight，2022，28(5)：627−636.

[26] GAN Baiqiang，DONG Qiuping. An improved optimal algorithm for collision detection of hybrid hierarchical bounding box[J]. Evolutionary Intelligence，2022，15(4)：2515−2527.

[27] 耿宏，高璐璐. 面向飞机虚拟维修的改进混合层次包围盒碰撞检测算法[J]. 科学技术与工程，2018，18(21)：63−68.

GENG Hong，GAO Lulu. Improved hybrid hierarchical bounding box collision detection algorithm for aircraft virtual maintenance[J]. Science Technology and Engineering，2018，18(21)：63−68.

[28] CHANG J W，WANG Wenping，KIM M S. Efficient collision detection using a dual OBB–sphere bounding volume hierarchy[J]. Computer–Aided Design，2010，42(1)：50−57.

[29] 张旭辉，吕欣媛，王甜，等. 数字孪生驱动的掘进机器人决策控制系统研究[J]. 煤炭科学技术，2022，50(7)：36−49.

ZHANG Xuhui，LYU Xinyuan，WANG Tian，et al. Research on decision control system of tunneling robot driven by digital twin[J]. Coal Science and Technology，2022，50(7)：36−49.

[30] 王妙云，张旭辉，马宏伟，等. 远程控制综采设备碰撞检测与预警方法[J]. 煤炭科学技术，2021，49(9)：110−116.

WANG Miaoyun，ZHANG Xuhui，MA Hongwei，et al. Collision detection and pre–warning method for remotely controlled fully–mechanized mining equipment[J]. Coal Science and Technology，2021，49(9)：110−116.

[31] 张旭辉，王甜，张超，等. 数字孪生驱动的悬臂式掘进机虚拟示教记忆截割方法[J]. 煤炭学报，2023，48(11)：4247−4260.

ZHANG Xuhui，WANG Tian，ZHANG Chao，et al. DT–driven memory cutting control method using VR instruction of boom–type roadheader[J]. Journal of China Coal Society，2023，48(11)：4247−4260.

[32] 李成美，白宏阳，郭宏伟，等. 一种改进光流法的运动目标检测及跟踪算法[J]. 仪器仪表学报，2018，39(5)：249−256.

LI Chengmei，BAI Hongyang，GUO Hongwei，et al. Moving object detection and tracking based on improved optical flow method[J]. Chinese Journal of Scientific Instrument，2018，39(5)：249−256.