Research on path planning methods for underground roadheader robots

Authors

ZHANG Xuhui, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratoty of Mine Electromechanical Equipment Intelligent Monitoring, Xi’an 710054, ChinaFollow
ZHENG Xili, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, ChinaFollow
YANG Wenjuan, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratoty of Mine Electromechanical Equipment Intelligent Monitoring, Xi’an 710054, China
LI Yuyang, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
MA Bing, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
DONG Zheng, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
CHEN Xin, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China

Abstract

In order to solve the problems of difficulty and low efficiency in the movement of robotic roadheaders under conditions of non-full-section roadways in coal mines, the characteristics of unstructured environments in coal mines and the motion characteristics of robotic roadheaders were analyzed, and a path planning method for robotic roadheaders based on deep reinforcement learning was proposed. The tunnel environment was constructed in real time using depth cameras, a virtual model for detecting roadheader-tunnel collisions was established, collision detection was performed in a virtual environment using the hierarchical bounding box method, and an obstacle avoidance strategy under the restrictions of tunnel boundary was developed. Considering the size of the roadheader robot and the single goal in the path planning process, the HER-SAC algorithm was proposed based on the traditional SAC algorithm by introducing the retrospective experience playback technology. The algorithm expands the target subset through the trajectory obtained by the initial target in the environment to increase training samples and training speed. On this basis, an agent was established based on the reward and punishment mechanism, and its state space and action space were defined according to the motion characteristics of the roadheader robot. The agent was trained using three algorithms under the same scenario, and the performances of these algorithms were comparatively analyzed using four indicators, namely, the average reward value, the maximum reward value, the number of steps to reach the maximum reward value, and robustness. In order to further verify the reliability of the proposed method, a virtual-real combination method was adopted, roadheader path planning was performed in two experimental scenarios set by adjusting the target position, and the results produced by the traditional SAC algorithm and the HER-SAC algorithm were compared. The results show that the HER-SAC algorithm converges faster and generally performs better than the PPO and SAC algorithms; in the two experimental scenarios, the path planned by the HER-SAC algorithm is smoother and shorter than that planned by the traditional SAC algorithm, and the error between the end point of the path planned by the HER-SAC algorithm and the target position is less than 3.53 cm, indicating that the HER-SAC algorithm can effectively execute and complete path planning tasks. This study lays a theoretical foundation for autonomous transfer control of roadheader robots and provides a new approach to the automation of coal mining equipment.

Keywords

roadheader robot, path planning, deep reinforcement learning, agent, combination of virtual and real, improved SAC algorithm, coal mine

DOI

10.12363/issn.1001-1986.23.11.0748

Recommended Citation

ZHANG Xuhui, ZHENG Xili, YANG Wenjuan, et al. (2024) "Research on path planning methods for underground roadheader robots," Coal Geology & Exploration: Vol. 52: Iss. 4, Article 16.
DOI: 10.12363/issn.1001-1986.23.11.0748
Available at: https://cge.researchcommons.org/journal/vol52/iss4/16

Reference

[1] 王海军,曹云,王洪磊. 煤矿智能化关键技术研究与实践[J]. 煤田地质与勘探,2023,51(1):44−54

WANG Haijun,CAO Yun,WANG Honglei. Research and practice on key technologies for intelligentization of coal mine[J]. Coal Geology & Exploration,2023,51(1):44−54

[2] 张旭辉,杨文娟,薛旭升,等. 煤矿远程智能掘进面临的挑战与研究进展[J]. 煤炭学报,2022,47(1):579−597

ZHANG Xuhui,YANG Wenjuan,XUE Xusheng,et al. Challenges and developing of the intelligent remote control on roadheaders in coal mine[J]. Journal of China Coal Society,2022,47(1):579−597

[3] 王国法,张建中,薛国华,等. 煤矿回采工作面智能地质保障技术进展与思考[J]. 煤田地质与勘探,2023,51(2):12−26

WANG Guofa,ZHANG Jianzhong,XUE Guohua,et al. Progress and reflection of intelligent geological guarantee technology in coal mining face[J]. Coal Geology & Exploration,2023,51(2):12−26

[4] 王国法,张德生. 煤炭智能化综采技术创新实践与发展展望[J]. 中国矿业大学学报,2018,47(3):459−467

WANG Guofa,ZHANG Desheng. Innovation practice and development prospect of intelligent fully mechanized technology for coal mining[J]. Journal of China University of Mining and Technology,2018,47(3):459−467

[5] 庞义辉,王国法,任怀伟. 智慧煤矿主体架构设计与系统平台建设关键技术[J]. 煤炭科学技术,2019,47(3):35−42

PANG Yihui,WANG Guofa,REN Huaiwei. Main structure design of intelligent coal mine and key technology of system platform construction[J]. Coal Science and Technology,2019,47(3):35−42

[6] 葛世荣,张晞,薛光辉,等. 我国煤矿煤机智能技术与装备发展研究[J]. 中国工程科学,2023,25(5):146−156

GE Shirong,ZHANG Xi,XUE Guanghui,et al. Development of intelligent technologies and machinery for coal mining in China’s underground coal mines[J]. Strategic Study of Chinese Academy of Engineering,2023,25(5):146−156

[7] 谢进,王飞. 煤矿智能掘进机器人关键技术探讨[J]. 工矿自动化,2021,47(增刊2):39−42

XIE Jin,WANG Fei. Discussion on key technologies of intelligent tunneling robot in coal mine[J]. Industry and Mine Automation,2021,47(Sup.2):39−42

[8] HOU Shengzhe,LU Xinming,GAO Wenli,et al. Interactive physically based simulation of roadheader robot[J]. Arabian Journal for Science and Engineering,2023,48(2):2441−2454.

[9] 张捍东,郑睿,岑豫皖. 移动机器人路径规划技术的现状与展望[J]. 系统仿真学报,2005,17(2):439−443

ZHANG Handong,ZHENG Rui,CEN Yuwan. Present situation and future development of mobile robot path planning technology[J]. Journal of System Simulation,2005,17(2):439−443

[10] HEO Y S,LEE K M,LEE S U. Robust stereo matching using adaptive normalized cross–correlation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2011,33(4):807−822.

[11] QU Huiyan,LI Wenhui,ZHAO Wei. Human–vehicle collision detection algorithm based on image processing[J]. International Journal of Pattern Recognition and Artificial Intelligence,2020,34(8):2055015.

[12] 张国飚,张华,刘满禄,等. 基于空间剖分的碰撞检测算法研究[J]. 计算机工程与应用,2014,50(7):46−49

ZHANG Guobiao,ZHANG Hua,LIU Manlu,et al. Research of collision detection algorithm based on spatial subdivision[J]. Computer Engineering and Applications,2014,50(7):46−49

[13] 彭晏飞,卢真真. 基于空间剖分和包围盒的快速碰撞检测算法[J]. 计算机应用与软件,2015,32(8):150−153

PENG Yanfei,LU Zhenzhen. Fast collision detection algorithm based on space subdivision and bounding box[J]. Computer Applications and Software,2015,32(8):150−153

[14] 张宇,张得礼,张文奇,等. 基于混合层次包围盒的水下训练机械臂碰撞检测方法研究[J]. 载人航天,2022,28(5):627−636

ZHANG Yu,ZHANG Deli,ZHANG Wenqi,et al. Research on collision detection method of underwater training manipulator based on hybrid hierarchical bounding box[J]. Manned Spaceflight,2022,28(5):627−636

[15] GAN Baiqiang,DONG Qiuping. An improved optimal algorithm for collision detection of hybrid hierarchical bounding box[J]. Evolutionary Intelligence,2022,15(4):2515−2527.

[16] 魏立新,吴绍坤,孙浩,等. 基于多行为的移动机器人路径规划[J]. 控制与决策,2019,34(12):2721−2726

WEI Lixin,WU Shaokun,SUN Hao,et al. Mobile robot path planning based on multi–behaviours[J]. Control and Decision,2019,34(12):2721−2726

[17] 万俊,孙薇,葛敏,等. 基于含避障角人工势场法的机器人路径规划[J]. 农业机械学报,2024,55(1):409−418

WAN Jun,SUN Wei,GE Min,et al. Robot path planning based on artificial potential field method with obstacle avoidance angles[J]. Transactions of the Chinese Society of Agricultural Machinery,2024,55(1):409−418

[18] 韩志军,花传杰,王磊. 基于A*算法的CGF坦克实体路径规划研究[J]. 计算机工程与应用,2003(35):222−224

HAN Zhijun,HUA Chuanjie,WANG Lei. Path planning for CGF entities’ intelligent behavior using A* algorithm[J]. Computer Engineering and Applications,2003(35):222−224

[19] 张伟民,张月,张辉. 基于改进A*算法的煤矿救援机器人路径规划[J]. 煤田地质与勘探,2022,50(12):185−193

ZHANG Weimin,ZHANG Yue,ZHANG Hui. Path planning of coal mine rescue robot based on improved A* algorithm[J]. Coal Geology & Exploration,2022,50(12):185−193

[20] 刘海鸥，薛明轩，关海杰，等. 基于分层2.5D地图的无人履带车辆路径规划[J/OL]. 北京理工大学学报，2023：1–9 [2024-01-02]. https://doi.org/10.15918/j.tbit1001–0645.2023.119.

LIU Hai’ou，XUE Mingxuan，GUAN Haijie，et al. Path planning algorithm based on layered 2.5D map for unmanned tracked vehicle[J/OL]. Transactions of Beijing Institute of Technology，2023：1–9 [2024-01-02]. https://doi.org/10.15918/j.tbit1001–0645.2023.119.

[21] ZHANG Lin,ZHANG Yingjie,LI Yangfan. Mobile robot path planning based on improved localized particle swarm optimization[J]. IEEE Sensors Journal,2021,21(5):6962−6972.

[22] 谭玉新,杨维,徐子睿. 面向煤矿井下局部复杂空间的机器人三维路径规划方法[J]. 煤炭学报,2017,42(6):1634−1642

TAN Yuxin,YANG Wei,XU Zirui. Three–dimensional path planning method for robot in underground local complex space[J]. Journal of China Coal Society,2017,42(6):1634−1642

[23] 赵少林,程杰. 基于粒子群并行优化的煤矿井下机器人路径规划[J]. 计算机测量与控制,2014,22(5):1600−1602

ZHAO Shaolin,CHENG Jie. Coal mine underground robot path planning based on parallel particle swarm optimization[J]. Computer Measurement and Control,2014,22(5):1600−1602

[24] LI Siding，XU Xin，ZUO Lei. Dynamic path planning of a mobile robot with improved Q–learning algorithm[C]//2015 IEEE International Conference on Information and Automation. Lijiang：IEEE，2015：409–414.

[25] BAE H,KIM G,KIM J,et al. Multi–robot path planning method using reinforcement learning[J]. Applied Sciences,2019,9(15):3057.

[26] 张敏骏,蔡岫航,吕馥言,等. 受限巷道空间区域栅格化掘进机自主纠偏研究[J]. 仪器仪表学报,2018,39(3):62−70

ZHANG Minjun,CAI Xiuhang,LYU Fuyan,et al. Research on roadheader auto rectification in limited roadway space based on regional grid[J]. Chinese Journal of Scientific Instrument,2018,39(3):62−70

[27] ZHOU Shijie,LI Zelun,LYU Zhongliang,et al. Research on positioning accuracy of mobile robot in indoor environment based on improved RTABMAP algorithm[J]. Sensors,2023,23(23):9468.

[28] 杨鑫,王天明,许端清. 基于GPU的层次包围盒快速构造方法[J]. 浙江大学学报(工学版),2012,46(1):84−89

YANG Xin,WANG Tianming,XU Duanqing. Fast BVH construction on GPU[J]. Journal of Zhejiang University (Engineering Science),2012,46(1):84−89

[29] 李曾琳,李波,白双霞,等. 基于AM–SAC的无人机自主空战决策[J]. 兵工学报,2023,44(9):2849−2858

LI Zenglin,LI Bo,BAI Shuangxia,et al. UAV autonomous air combat decision–making based on AM–SAC[J]. Acta Armamentarii,2023,44(9):2849−2858

[30] 夏家伟,朱旭芳,罗亚松,等. 基于深度强化学习的无人艇轨迹跟踪算法研究[J]. 华中科技大学学报(自然科学版),2023,51(5):74−80

XIA Jiawei,ZHU Xufang,LUO Yasong,et al. Study on trajectory tracking algorithm of unmanned surface vehicle based on deep reinforcement learning[J]. Journal of Huazhong University of Science and Technology (Natural Science Edition),2023,51(5):74−80

[31] 李源潮，陶重犇，王琛. 基于最大熵深度强化学习的双足机器人步态控制方法[J/OL]. 计算机应用，2023：1–7 [2023-08-19]. http://kns.cnki.net/kcms/detail/51.1307.TP.20230524.1455.003.html.

LI Yuanchao，TAO Chongben，WANG Chen. Gait control method based on maximum entropy deep reinforcement learning for biped robot[J/OL]. Journal of Computer Applications，2023：1–7 [2023-08-19]. http://kns.cnki.net/kcms/detail/51.1307.TP.20230524.1455.003.html.

[32] 诸程瑛,王振雷. 基于改进深度强化学习的乙烯裂解炉操作优化[J]. 化工学报,2023,74(8):3429−3437

ZHU Chengying,WANG Zhenlei. Operation optimization of ethylene cracking furnace based on improved deep reinforcement learning algorithm[J]. CIESC Journal,2023,74(8):3429−3437