Cross-modal LiDAR-camera joint calibration for heading and mining face in coal mines

Authors

YANG Wenjuan, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi’an 710054, ChinaFollow
REN Zhiteng, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China
ZHANG Xuhui, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi’an 710054, China
DU Yuyang, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi’an 710054, China
LI Changpeng, State Key Laboratory of Digital Intelligent Technology for Unmanned Coal Mining, Huainan 232001, China
ZHANG Chao, College of Mechanical Engineering, Xi’an University of Science and Technology, Xi’an 710054, China; Shaanxi Key Laboratory of Mine Electromechanical Equipment Intelligent Detection and Control, Xi’an 710054, China
XU Jie, CCTEG Information Technology Co., Ltd., Xi’an 710000, China
WAN Jicheng, CCTEG Information Technology Co., Ltd., Xi’an 710000, China
LEI Mengyu, CCTEG Information Technology Co., Ltd., Xi’an 710000, China

Abstract

Objective In underground coal mines, it is challenging to extract the features of the heading and mining face stably under insufficient illumination, significant dust interference, and sparse point clouds. This severely undermines the accuracy and robustness of camera-LiDAR joint calibration based on underground unstructured scene characteristics.Methods Targeting complex operating conditions, including fully mechanized mining face and tunneling roadways, this study developed a LiDAR-camera joint calibration method based on cross-depth feature coupling. In this method, an improved random sample consensus (RANSAC) algorithm for multi-plane fitting was designed for feature extraction. By integrating normal vector-based pre-clustering and an adaptive iteration mechanism, the improved RANSAC algorithm can efficiently extract geometric structures such as hydraulic support roof beams and tunnel boring machine (TBM) casings. Meanwhile, a cross-depth edge fusion strategy was proposed, which utilized curvature discontinuities and planar intersection features collaboratively to enhance the integrity and robustness of edge structures. Regarding the calibration framework, a two-stage registration strategy consisting of coarse and fine registration was developed. Among these, the coarse registration allowed for the rapid estimation of the initial extrinsic parameters through axial cyclic perturbations. In contrast, the fine registration adopted joint point-line constraints within Lie group space and nonlinear optimization through iteration. This registration strategy was aimed at ensuring high-precision alignment even under coal dust interference and complex operating conditions. Results and Conclusions The proposed method was validated on the Gazebo simulation platform and in actual underground scenarios. The results indicate that under noise-free conditions, the proposed method yielded rotational errors of less than 0.2°, translational errors of below 0.02 m, and mean reprojection errors of 3.5 px or less. Furthermore, this method remained highly stable even in environments with high noise levels. Under the scenarios of heading face and fully mechanized mining face, the proposed method yielded mean reprojection errors of 2.89 px and 3.03 px, respectively, significantly outperforming traditional methods. Therefore, this method, requiring no artificial calibration targets, exhibits superior environmental adaptability and stability, meeting the requirements for high-precision calibration of multimodal perception systems in complex underground environments in coal mines.

Keywords

LiDAR, camera, joint calibration, point cloud registration, point cloud plane extraction, coal mine, heading and coal mining face

DOI

10.12363/issn.1001-1986.25.10.0797

Recommended Citation

YANG Wenjuan, REN Zhiteng, ZHANG Xuhui, et al. (2026) "Cross-modal LiDAR-camera joint calibration for heading and mining face in coal mines," Coal Geology & Exploration: Vol. 54: Iss. 2, Article 23.
DOI: 10.12363/issn.1001-1986.25.10.0797
Available at: https://cge.researchcommons.org/journal/vol54/iss2/23

Reference

[1] GU Shuo,ZHANG Yigong,TANG Jinhui,et al. Integrating dense LiDAR–camera road detection maps by a multi–modal CRF model[J]. IEEE Transactions on Vehicular Technology,2019,68(12):11635−11645.

[2] BASSO F,MENEGATTI E,PRETTO A. Robust intrinsic and extrinsic calibration of RGB–D cameras[J]. IEEE Transactions on Robotics,2018,34(5):1315−1332.

[3] DI Kaichang,WAN Wenhui,ZHAO Hongying,et al. Progress and applications of visual SLAM[J]. Journal of Geodesy and Geoinformation Science,2019,2(2):38−49.

[4] WALLACE A M,HALIMI A,BULLER G S. Full waveform LiDAR for adverse weather conditions[J]. IEEE Transactions on Vehicular Technology,2020,69(7):7064−7077.

[5] 魏超,吴西涛,朱耿霆,等. 基于视觉相机和激光雷达融合的无人车障碍物检测与跟踪研究[J]. 机械工程学报,2025,61(2):296−309

WEI Chao,WU Xitao,ZHU Gengting,et al. Research on obstacle detection and tracking of autonomous vehicles based on the fusion of vision camera and LiDAR[J]. Journal of Mechanical Engineering,2025,61(2):296−309

[6] 王旭,陆佳嘉,江敏,等. 改进DBSCAN的自适应LiDAR/相机联合标定研究[J]. 应用激光,2025,45(2):148−156

WANG Xu,LU Jiajia,JIANG Min,et al. Study on adaptive LiDAR/camera joint calibration with improved DBSCAN[J]. Applied Laser,2025,45(2):148−156

[7] 丁雅斌,梅江平,张文昌,等. 基于单目视觉的并联机器人末端位姿检测[J]. 机械工程学报,2014,50(21):174−179

DING Yabin,MEI Jiangping,ZHANG Wenchang,et al. Position and orientation measurement of parallel robot based on monocular vision[J]. Journal of Mechanical Engineering,2014,50(21):174−179

[8] 乔贵方,孙大林,宋光明,等. 串联机器人标定系统的坐标系快速转换方法[J]. 机械工程学报,2020,56(14):1−8

QIAO Guifang,SUN Dalin,SONG Guangming,et al. A rapid coordinate transformation method for serial robot calibration system[J]. Journal of Mechanical Engineering,2020,56(14):1−8

[9] 李兵,傅卫平,王雯,等. 一种基于EIH的装配机器人标定方法[J]. 机械工程学报,2018,54(7):38−44

LI Bing,FU Weiping,WANG Wen,et al. EIH based calibration method for assembly robot[J]. Journal of Mechanical Engineering,2018,54(7):38−44

[10] CHENG Lei，CAO Siyang. Online targetless radar–camera extrinsic calibration based on the common features of radar and camera[C]//NAECON 2023：IEEE National Aerospace and Electronics Conference. Dayton：IEEE，2023：294–299.

[11] ZHANG Guanyu,WU Kunyang,LIN Jun,et al. Automatic extrinsic parameter calibration for camera–LiDAR fusion using spherical target[J]. IEEE Robotics and Automation Letters,2024,9(6):5743−5750.

[12] ZHANG Qilong，PLESS R. Extrinsic calibration of a camera and laser range finder (improves camera calibration)[C]//2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Sendai：IEEE，2005：2301–2306.

[13] 覃兴胜,李晓欢,唐欣,等. 基于标定板关键点的激光雷达与相机外参标定方法[J]. 激光与光电子学进展,2022,59(4):0428001

QIN Xingsheng,LI Xiaohuan,TANG Xin,et al. Extrinsic calibration method of LiDAR and camera based on key points of calibration board[J]. Laser & Optoelectronics Progress,2022,59(4):0428001

[14] FAN Song,YU Ying,XU Maolin,et al. High–precision external parameter calibration method for camera and LiDAR based on a calibration device[J]. IEEE Access,2023,11:18750−18760.

[15] 徐孝彬,曹晨飞,张磊,等. 基于四面体特征的面阵激光雷达与相机标定方法[J]. 光子学报,2024,53(7):0712002

XU Xiaobin,CAO Chenfei,ZHANG Lei,et al. Planar array LiDAR and camera calibration method based on tetrahedral features[J]. Acta Photonica Sinica,2024,53(7):0712002

[16] 陈广永,陈芳,叶方舟,等. 自动匹配角点特征的激光雷达与光学相机标定方法[J]. 激光杂志,2025,46(10):20−26

CHEN Guangyong,CHEN Fang,YE Fangzhou,et al. LiDAR–camera joint calibration based on automatically matched corner features[J]. Laser Journal,2025,46(10):20−26

[17] ITAMI F,YAMAZAKI T. An improved method for the calibration of a 2–D LiDAR with respect to a camera by using a checkerboard target[J]. IEEE Sensors Journal,2020,20(14):7906−7917.

[18] ZHANG Yuan,XU Shaoyi,WANG Chengtao,et al. Feature–preserving point cloud simplification method based on cylindrical neighborhoods[J]. The Journal of Supercomputing,2025,81(16):1535.

[19] XU Shaoyi,SHI Boxuan,WANG Chengtao,et al. Novel high–performance automatic removal method of interference points for point cloud data in coal mine roadway environment[J]. International Journal of Remote Sensing,2023,44(5):1433−1459.

[20] XU Shaoyi,HUO Yingrui,WANG Chengtao,et al. Novel feature preserving method for simplifying the surface point cloud of underground space tunnel[J]. IEEE Transactions on Instrumentation and Measurement,2024,73:7004414.

[21] 刘万里,刘扬,张学亮. 融合互信息和多特征约束的激光雷达与相机外参标定方法[J]. 仪器仪表学报,2025,46(1):29−41

LIU Wanli,LIU Yang,ZHANG Xueliang. Extrinsic calibration of LiDAR and camera through mutual information integrated multi–feature constraints[J]. Chinese Journal of Scientific Instrument,2025,46(1):29−41

[22] 郑旺,于红绯,吕晋. 基于线特征的激光雷达与相机外参标定[J]. 激光与光电子学进展,2024,61(22):2215002

ZHENG Wang,YU Hongfei,LYU Jin. External parameter calibration of LiDAR and camera based on line feature[J]. Laser & Optoelectronics Progress,2024,61(22):2215002

[23] BROWN M,WINDRIDGE D,GUILLEMAUT J Y. A family of globally optimal branch–and–bound algorithms for 2D–3D correspondence–free registration[J]. Pattern Recognition,2019,93:36−54.

[24] OU Ni,CAI Hanyu,YANG Jiawen,et al. Targetless extrinsic calibration of camera and low–resolution 3–D LiDAR[J]. IEEE Sensors Journal,2023,23(10):10889−10899.

[25] 杨佳佳,张传伟,周李兵,等. 矿用激光雷达与相机的无目标自动标定方法研究[J]. 工矿自动化,2024,50(10):53−61

YANG Jiajia,ZHANG Chuanwei,ZHOU Libing,et al. Research on the targetless automatic calibration method for mining LiDAR and camera[J]. Journal of Mine Automation,2024,50(10):53−61

[26] 彭湃,耿可可,殷国栋,等. 基于传感器融合里程计的相机与激光雷达自动重标定方法[J]. 机械工程学报,2021,57(20):206−214

PENG Pai,GENG Keke,YIN Guodong,et al. Automatic recalibration of camera and LiDAR using sensor fusion odometry[J]. Journal of Mechanical Engineering,2021,57(20):206−214

[27] YUAN Chongjian,LIU Xiyuan,HONG Xiaoping,et al. Pixel–level extrinsic self calibration of high resolution LiDAR and camera in targetless environments[J]. IEEE Robotics and Automation Letters,2021,6(4):7517−7524.

[28] LIU Xiyuan,YUAN Chongjian,ZHANG Fu. Targetless extrinsic calibration of multiple small FoV LiDARs and cameras using adaptive voxelization[J]. IEEE Transactions on Instrumentation and Measurement,2022,71:8502612.

[29] ZHEN Weikun,HU Yaoyu,LIU Jingfeng,et al. A joint optimization approach of LiDAR–camera fusion for accurate dense 3–D reconstructions[J]. IEEE Robotics and Automation Letters,2019,4(4):3585−3592.

[30] 熊超,乌萌,刘宗毅,等. 激光雷达与相机联合标定进展研究[J]. 导航定位学报,2024,12(2):155−166

XIONG Chao,WU Meng,LIU Zongyi,et al. Review of joint calibration of LiDAR and camera[J]. Journal of Navigation and Positioning,2024,12(2):155−166

[31] AN Pei,MA Tao,YU Kun,et al. Geometric calibration for LiDAR–camera system fusing 3D–2D and 3D–3D point correspondences[J]. Optics Express,2020,28(2):2122−2141.

[32] 叶锦华,林旭敏,吴海彬. 基于DBSCAN的改进RANSAC点云平面拟合算法[J]. 湖南大学学报(自然科学版),2025,52(2):76−87

YE Jinhua,LIN Xumin,WU Haibin. Improved RANSAC point cloud plane fitting algorithm based on DBSCAN[J]. Journal of Hunan University (Natural Sciences),2025,52(2):76−87

[33] BODE L,WEINMANN M,KLEIN R. BoundED:Neural boundary and edge detection in 3D point clouds via local neighborhood statistics[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2023,205:334−351.

[34] GALLARDO–ALVARADO J,GARCIA–MURILLO M A,TABARES–MARTINEZ J M,et al. Symbolic computation of the lie algebra Se(3) of the Euclidean Group SE(3):An application to the infinitesimal kinematics of robot manipulators[J]. Mathematics,2024,12(16):2538.

[35] FU Bo,WANG Yue,DING Xiaqing,et al. LiDAR–camera calibration under arbitrary configurations:Observability and methods[J]. IEEE Transactions on Instrumentation and Measurement,2020,69(6):3089−3102.

[36] LYU Xudong，WANG Boya，DOU Ziwen，et al. LCCNet：LiDAR and camera self–calibration using cost volume network[C]//2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Nashville：IEEE，2021：2888–2895.