Research on drill pipe pose recognition technology based on monocular vision

Authors

MA Bin, Xi’an Research Institute (Group) Co. Ltd., China Coal Technology and Engineering Group Corp., Xi’an 710077, ChinaFollow
PENG Guangyu, Xi’an Research Institute (Group) Co. Ltd., China Coal Technology and Engineering Group Corp., Xi’an 710077, ChinaFollow

Abstract

As a key technology in the field of intelligent drilling equipment, automatic drill pipe loading and unloading technology restricts the automation and intelligent development of underground drilling equipment in coal mines. The existing drill pipe automatic loading and unloading system mainly relies on the mechanical structure and proximity switches for positioning, which has the problem of poor positioning accuracy and low automation. To solve this problem, a drill pipe pose recognition algorithm based on monocular vision technology was proposed. The camera was used to capture the image containing the cooperative target, and the relative distance and posture between the camera and the cooperative target were calculated; the position and posture of the drill pipe relative to the manipulator was deduced through fixed coordinate transformation, and the manipulator was guided to automatically load and unload the drill pipe. First, we determine the overall scheme of the system and then establish a mathematical model of camera imaging using the principle of pinhole imaging and Zhang Zhengyou’s calibration method, so as to solve the internal and external parameters of the camera. Secondly, using the checkerboard calibration plate as the cooperation target of the drill pipe to be tested, a monocular ranging model of any plane in space was established according to the small hole imaging model and the spatial imaging relationship, and the distance between the optical center of the camera and the cooperative target point was calculated. Finally, the attitude matrix of the cooperative target was obtained through the camera imaging model. Combined with the internal and external parameters of the camera, the coordinate transformation was used to obtain the attitude matrix of the cooperative target in the world coordinate system, and then the position and attitude recognition of the drill pipe was completed through fixed coordinate transformation. To verify the accuracy of the algorithm, the drill pipe pose recognition test was carried out indoors. In the test, the repetitive distance measurement and attitude estimation were carried out for each on-site picture. The results show that the drill pipe distance recognition deviation is within 0.12%, and the drill pipe attitude recognition deviation is within 1.08%, which meets the precision requirements of automatic loading and unloading of drill pipes. The results also show that the drill pipe pose recognition algorithm based on monocular vision technology is real and effective. The algorithm can realize the intelligent recognition of drill pipe positioning, improve the automatic loading and unloading accuracy of drill pipe and the intelligent level of drilling equipment.

Keywords

drill pipe, pose recognition, vision technology, monocular vision, ranging model, coal mine intellectualization

DOI

10.12363/issn.1001-1986.22.01.0036

Recommended Citation

M P. (2022) "Research on drill pipe pose recognition technology based on monocular vision," Coal Geology & Exploration: Vol. 50: Iss. 10, Article 20.
DOI: 10.12363/issn.1001-1986.22.01.0036
Available at: https://cge.researchcommons.org/journal/vol50/iss10/20

Reference

[1] 国家安全监管总局. 关于开展“机械化换人、自动化减人”科技强安专项行动的通知[Z]. 安监总科技，2015，63号.

[2] 国家发展改革委，国家能源局. 关于印发《能源技术革命创新行动计划(2016–2030年)》的通知[Z]. 发改能源，2016，513号.

[3] 国家发展改革委，国家能源局，应急部，等. 关于印发《关于加快煤矿智能化发展的指导意见》的通知[Z]. 发改能源，2020，283号.

[4] 马帅. 井下智能瓦斯钻机自动续接装置结构设计及控制[D]. 太原：太原理工大学，2020.

MA Shuai. Structure design and control of the automatic connection device of the underground intelligent gas drilling machine[D]. Taiyuan：Taiyuan University of Technology，2020.

[5] 吕晋军,辛德忠. ZYWL–4000SY智能遥控自动钻机的设计[J]. 煤矿机械,2019,40(3):4−6

LYU Jinjun,XIN Dezhong. Design of ZYWL−4000SY intelligent remote control automatic drilling machine[J]. Coal Mine Machinery,2019,40(3):4−6

[6] 陈林. 液压钻机自动上下杆机构的设计与分析[D]. 淮南：安徽理工大学，2018.

CHEN Lin. Design and analysis of automatic loading and unloading rod mechanism of hydraulic drill[D]. Huainan：Anhui University of Science and Technology，2018.

[7] 姚亚峰,李晓鹏,张刚,等. 煤矿坑道钻机自动加卸钻杆装置的研发[J]. 煤矿机械,2017,38(6):91−93

YAO Yafeng,LI Xiaopeng,ZHANG Gang,et al. Development of automatic loading drill rods device on coal mine tunnel drill rig[J]. Coal Mine Machinery,2017,38(6):91−93

[8] 王清峰,陈航. 基于路径规划的大容量钻杆自动输送系统研究[J]. 矿业安全与环保,2020,47(1):1−6

WANG Qingfeng,CHEN Hang. Research on automatic conveying system of large–capacity drill pipe based on path planning[J]. Mining Safety & Environmental Protection,2020,47(1):1−6

[9] 邢艳军,王浩,叶东,等. 基于单目视觉的非合作目标相对位姿测量[J]. 中国空间科学技术,10/25/2022,42(4):36−44

XING Yanjun,WANG Hao,YE Dong,et al. Relative pose measurement for non−cooperative target based on monocular vision[J]. Chinese Space Science and Technology,10/25/2022,42(4):36−44

[10] 李又文,张喜涛,张学锋. 基于单目相机的空间非合作目标姿态测量[J]. 红外技术,2014,36(2):110−114

LI Youwen,ZHANG Xitao,ZHANG Xuefeng. Research of attitude measuring system using single camera for non–cooperative spacecraft[J]. Infrared Technology,2014,36(2):110−114

[11] 戴云彤. 多相机测量中相机外部参数优化与高精度姿态识别[D]. 南京：东南大学，2018.

DAI Yuntong. External parameters optimization and high–precision pose recognition in multi–camera measurement[D]. Nanjing：Southeast University，2018.

[12] 郝仁杰,王中宇,李亚茹. 一种单目视觉位姿测量系统的误差分析方法[J]. 应用光学,2019,40(1):79−85

HAO Renjie,WANG Zhongyu,LI Yaru. Error analysis method for monocular vision pose measurement system[J]. Journal of Applied Optics,2019,40(1):79−85

[13] 徐伟高. 基于单目视觉的位姿测量关键技术研究[D]. 西安：中国科学院大学(中国科学院西安光学精密机械研究所)，2016.

XU Weigao. Key technology research on pose measurement based on monocular vision[D]. Xi’an：Xi’an Institute of Optics & Precision Mechanics，Chinese Academy of Science，2016.

[14] 屈也频,刘坚强,侯旺. 单目视觉高精度测量中的合作目标图形设计[J]. 光学学报,2020,40(13):1315001

QU Yepin,LIU Jianqiang,HOU Wang. Graphics design of cooperative targets on monocular vision high precision measurement[J]. Acta Optica Sinica,2020,40(13):1315001

[15] 刘韬，葛大伟. 机器视觉及其应用技术[M]. 北京：机械工业出版社，2020.

[16] 赵海平. 车载单目视觉测距模型与应用研究[D]. 呼和浩特：内蒙古农业大学，2019.

ZHAO Haiping. Research on vehicle–borne monocular vision ranging model and its application[D]. Hohhot：Inner Mongolia Agricultural University，2019.

[17] 彭妍,郭君斌,于传强,等. 基于平面变换的高精度相机标定方法[J]. 北京航空航天大学学报,10/25/2022,48(7):1297−1303

PENG Yan,GUO Junbin,YU Chuanqiang,et al. High precision camera calibration method based on plane transformation[J]. Journal of Beijing University of Aeronautics and Astronautics,10/25/2022,48(7):1297−1303