Open Access Open Access  Restricted Access Subscription Access

Fusion of Millimeter Wave Radar and Monocular Vision for Obstacle Detection Based on Rotor-Wing UAV

Juntong Qi, Chao Jiang, Chunsheng Hua, Tianyu Lin, Yong Xia, Jianda Han


This paper presents a framework for detecting the obstacles from a rotor-wing UAV by fusing the information from millimeter-wave radar and monocular camera. In order to implement the angle measurement function of ranging radar, we designed a dual radar network system to locate the target position, and through calibration we achieved correspondence between target orientation and image area. In order to deal with the complex movements of UAV (such as yaw and pitch), this paper proposes a robust algorithm, which can calculate the accurate height of target position points in the image as well as the bottom boundary of the target area in the image, thus avoiding over segmentation of the target. Finally, the target is segmented from the background of the partition after fusion through the threshold algorithm according to the mutational gray value of the bottom boundary of the target position. Through extensive experiments, the effectiveness and efficiency of proposed algorithm have been proved and the experimental results show that the proposed method is novel, simple and feasible.

Full Text:



Gai WenDong, Wang HongLun, Li DaWei, “ Flight dynamic modeling and analysis for the Canard Rotor/Wing UAV,” Kongqi Donglixue Xuebao/Acta Aerodynamica Sinica, volume 30, number 2, 2012, pp. 244-249.

Lin Na, Yang Wunian, Wang Bin, “ Improved neural network algorithm for classification of UAV imagery related to wenchuan earthquake,” Second International Conference on Earth Observation for Global Change, volume 7471, 2009.

Bednowitz N, Batta R, Nagi R, “ Dispatching and loitering policies for unmanned aerial vehicles under dynamically arriving multiple priority targets,” Journal of Simulation, volume 8, number 1, 2014, pp. 9-24.

Ma Lei, Cheng Liang and Han Wenquan, “ Cultivated land information extraction from high-resolution unmanned aerial vehicle imagery data,” Journal of Applied Remote Sensing, volume 8, 2014.

Lisein Jonathan, Pierrot Deseilligny Marc and Bonnet Stephanie, “ A Photogrammetric Workflow for the Creation of a Forest Canopy Height Model from Small Unmanned Aerial System Imagery,” Forests, volume 4, number 4, 2013, pp. 922-944.

Sinopoli B, Micheli M and Donato G, “ Vision based navigation for an unmanned aerial vehicle,” IEEE International Conference on Robotics and Automation, May. 2001, pp. 1757-1764.

Oyabu Takashi, “Sensing Technology on Human Science,” Sensor Letters, volume 9, number 1, 2011, pp. 379-383.

Cho Sungwook, Huh Sungsik, Shim David Hyunchul, “ Vision-Based Detection and Tracking of Airborne Obstacles in a Cluttered Environmen,” Journal of Intelligent & Robotic Systems, volume 69, number 1-4, 2013, pp. 475-488.

Chowdhary Girish, Johnson Eric N, Magree Daniel, “ GPS-denied Indoor and Outdoor Monocular Vision Aided Navigation and Control of Unmanned Aircraft,” Journal of Field Robotics, volume 30, number 3, 2013, pp. 415-438.

Mejias Luis, Saripalli Srikanth, Campoy Pascual, “ Visual servoing of an autonomous helicopter in urban areas using feature tracking

,” Journal of Field Robotics, volume 23, number 3-4, 2006, pp. 185-199.

Marlow Sean Quinn, Langelaan Jack W, “Poputation of a range brearing map for lacal obstacle avoidance using monocular vision,” Guigance and Control, volume 144, 2012, pp. 559-573.

Wu Jiang, Fei Wenkai, Li Qianru, “An integrated measure and location method based on airborne 2D laser scanning sensor for UAV's power line inspection,” International Conference on Measuring Technology and Mechatronics Automation, Jan.2013, pp. 213-217.

Goktogan Ali Haydar, Brooker Graham, Sukkarieh Salah, “A compact Millimeter Wave radar sensor for Unmanned Air Vehicles,” Field and Service Robotics: Recent Advances in Research and Applications, volume 24, Jul.2006, pp. 311-320.

Yamamoto K, Yamada K, “Obstacle detection for helicopter flights by infrared in ages,” Enhanced and Synthetic Vision, volume 4363, 2001, pp. 76-85.

Yonemoto N, Yamamoto K, Yamada K, “Obstacle detection and warning for helicopter flight using infrared and millimeter wave,” Enhanced and Synthetic Vision 2003, volume 5081, 2003, pp. 31-38.

Tsai I-Cheng, Chiu Ching-Te, “Depth-based posture recognition by radar and vision fusion for real-time applications,” International Conference on Acoustics Speech and Signal Processing ICASSP, 2013, pp. 2702-2706.

Lind Henrik, “How new Active Safety Systems and Always Connected Vehicles leads to Challenges on Antenna Design and Integration in the Automotive Domain,” Proceedings of the European Conference on Antennas and Propagation, 2013, pp. 3-4.

Isermann Rolf, Mannale Roman, Schmitt Ken, “ Collision-avoidance systems PRORETA: Situation analysis and intervention control,” Control Engineering Practice, volume 20, number 11, 2012, pp. 1236-1246.

Oyabu Takashi, “ A sensor fusion method for mobile robot navigation,” 2006 SICE-ICASE International Joint Conference, volume 1-13, number 1, 2006, pp. 3445-3451.

Jack Browne, “ Millimeter-Wave Bands Support Diverse Uses,” Microwaves & RF, volume 47, number 8, 2008.

Sugimoto,S.,Tateda.H, Takahashi.H,”Obstacle Detection Using Millimeter-Wave Radar and Its Visualization on Image Sequence,” In Proceedings of the 17 International Conference on Pattern Recognition, Cambridge, UK, 23-26 August 2004;Volume 3, pp. 342-345.

Bhanu Bir, Lee Sungkee, Ming John, “Adaptive Image Segmentation Using a Genetic Algorithm,” IEEE Transactions on Systems, Man and Cybernetics, volume 25, number 12, pp 1543-1567, Dec 1995.

Otsu Nobuyuki, “Discriminant and least square threshold selection,” Proc. of the 4th International Joint Conference on Pattern Recognition, 1978:592-596.

Kittler J, lllingworth J., “On threshold selection using clustering criteria,” IEEE Transactions on Systems Man and Cybernetics, 1985, pp.652-655.

Kang ChungChia, Wang WenJune, Kang ChungHao, “Image segmentation with complicated background by using seeded region growing,” International Journal of Electronics and Communications, volume 66, number 9, 2012.

Jin Lisheng, Wang Rongben, Gao Long,” Shadow path mark segmentation method based on region-growing for intelligent vehicle,” Journal of Jilin University(Enginerring and Technology Edition) , volume 36, number 12, 2006.

Chuanmin Xiao, Zelin Shi, Renbo XIA, Wei Wu, “Edge-Detection Algorithm Based on Visual Saliency,” Information and Control, 2014 volume 43 , number 1, 2014 , pp. 9 -13.

Wenhui Xiang, Yu Liu , Yang Cao, Zengfu Wang , “3D Ground Plane Estimation from a Monocular Vehicle-borne Image,” Robot, volume 36, number 1, Jan. 2014.



  • There are currently no refbacks.