A comprehensive approach to detection and tracking of unmanned aerial vehicles based on computer vision

Аuthors

Golubev A. S.^*, Ilamanova V. , Nguyen M. Z., Soshnikov D. M., Khilenko A. A.

State University named after Alexander and Nikolay Stoletovs, 87, Gorky str. Vladimir, 600000, Russia

*e-mail: andrey.golubev@vlsu.ru

Abstract

A comprehensive approach to countering unmanned aerial vehicles is proposed, covering all processing stages: detection, classification, tracking, and trajectory prediction. At the detection stage, a comparative study of modern neural network architectures (YOLOv8, YOLOX, SSD) was conducted, where the YOLOX model showed the highest accuracy with mAP50=0.964. Convolutional neural networks were used for UAV type classification, providing accuracy up to 91.35%. At the tracking stage, the CSRT algorithm was recognized as optimal, demonstrating robustness to occlusions and lighting changes. For trajectory prediction, polynomial regression showed the best results with an average error of 1.27%. Created test datasets and fine-tuning of models using data augmentation were performed. The results can be applied in airspace monitoring and protection systems. The work considers traditional and modern UAV detection methods, including radio frequency, acoustic, and optical approaches. An analysis of existing solutions and their limitations is carried out. A research methodology is developed, including the creation of specialized datasets, training, and testing of models. Experiments were performed in the Google Colab environment using NVIDIA Tesla T4 GPU. The software implementation was carried out in Python 3.9 with PyTorch and OpenCV 4.8.0 libraries. The obtained results confirm the effectiveness of the proposed approach in real conditions.

Keywords:

unmanned aerial vehicles; computer vision; object detection; tracking; trajectory prediction; YOLO; CSRT; OpenCV

References

1. Main Methods of Detection and Jamming of Drones (UAVs), AO "TRANZAS Consulting", June 2025 (in Russian). URL: https://trancons.ru/information/stati_i_publikacii/drone-supression. 
2. Khawaja W., Ezuma M., Semkin V., Erden F., Ozdemir O., Guvenc I. A Survey on Detection, Classification, and Tracking of UAVs Using Radar and Communications Systems // IEEE Communications Surveys & Tutorials. 2025. DOI: https://doi.org/10.1109/COMST.2025.3554613. 
3. A system for detecting UAVs over the city by the sounds they emit has been developed, CNews, August 6, 2025 (in Russian). URL: https://www.cnews.ru/news/top/2025-08-06_razrabotana_sistema_dlya.
4. Bakhalov I. M., Startsev S. V. Recognition of UAVs of various classes by means of computer vision, NITU "MISIS", Moscow, 2025 (in Russian). URL: https://www.researchgate.net/publication/387896752.
5. Kul'chitskiy A.I., Makarchuk N.S., Kalashnikov G.A. Using computer vision for equipment recognition, Nauchnyi Al'manakh, 2023. Vol. 18, no. 5, pp. 2245-2253. (in Russian). URL: https://na-journal.ru/5-2023-informacionnye-tekhnologii/5535-ispolzovanie-kompyuternogo-zreniya-dlya....
6. Liu, Z.; An, P.; Yang, Y.; Qiu, S.; Liu, Q.: Xu, X. Vision-Based Drone Detection in Complex Environments: A Survey. Drones 2024, 8, 643. DOI: https://doi.org/10.3390/drones8110643.
7. Alshaer, N., Abdelfatah, R., Ismail, T. and Mahmoud, H. (2025), Vision-Based UAV Detection and Tracking Using Deep Learning and Kalman Filter. Computational Intelligence, 41: e70026. DOI: https://doi.org/10.1111/coin.70026.
8. Ramachandran, Anitha Sangaiah, Arun. (2021). A Review on Object Detection in Unmanned Aerial Vehicle Surveillance. International Journal of Cognitive Computing in Engineering. 2. DOI: https://doi.org/10.1016/j.ijcce.2021.11.005. 
9. Unlu E., Zenou E., Riviere N. et al. Deep learning-based strategies for the detection and tracking of drones using several cameras // IPSJ Transactions on Computer Vision and Applications. 2019. Vol. 11. Art. 7. DOI: https://doi.org/10.1186/s41074-019-0059-x. 
10. Raed Abu Zitar, Mohammad Al-Betar, Mohamad Ryalat, Sofian Kassaymeh. (2023) A review of UAV Visual Detection and Tracking Methods. arXiv:2306.05089 [cs.CV]. URL: https://arxiv.org/pdf/2306.05089.
11. K. He, X. Zhang, S. Ren and J. Sun, Deep Residual Learning for Image Recognition (2016) IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2016, pp. 770-778. DOI: https://doi.org/10.1109/CVPR.2016.90.
12. Xie S., Girshick R., Dollár P., Tu Z., He K. Aggregated Residual Transformations for Deep Neural Networks // Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 1492-1500. DOI: https://doi.org/10.1109/CVPR.2017.634.
13. Tan, M. and Le, Q.V. (2019) Efficient Net: Rethinking Model Scaling for Convolutional Neural Networks. Proceedings of the 36th International Conference on Machine Learning, ICML 2019, Long Beach, 9-15 June 2019, pp. 6105-6114. URL: http://proceedings.mlr.press/v97/tan19a.html. 
14. Yurushkin M. A Complete Review of the OpenCV Object Tracking Algorithms // Brouton Lab. URL: https://broutonlab.com/blog/opencv-object-tracking. 

Download