Convert object coordinates from image pixels to world coordinates using Python and Unity

Аuthors

Gataulin A. ^*, Babchinetsky S. ^**

Saint Petersburg State University of Aerospace Instrumentation, 67, Bolshaya Morskaya str., Saint Petersburg, 190000, Russia

*e-mail: aleksandrgataulin745@gmail.com
**e-mail: lnpt@guap.ru

Abstract

The article adduces description of development and implementation of an algorithm for transforming coordinates of objects captured in an image from the camera installed on an unmanned aerial vehicle, as well as the development of a simulation employing the Unity engine to test the written algorithm and verify the obtained world coordinates.
The purpose of the presented article consists in developing and implementing an algorithm for converting coordinates of objects in pixels into the world geographic coordinates of the EPSG3857 (Google Mercator) and EPSG4326 (WGS84) formats, which are the most common in the modern world. The developed algorithm will allow computing geographic coordinates of detected objects, knowing their coordinates in pixels and geographic coordinates of an unmanned aerial vehicle equipped with a GPS sensor for its location monitoring.
The authors employ mathematical foundations of coordinate transformation, as well as the Python programming language to perform basic mathematical operations, and the Unity game engine to simulate the flight process of an unmanned aerial vehicle and transfer images from the camera to a Python script. A scene for the algorithm testing was built with the Unity engine. Using the C# language, the information necessary to the coordinates computing is collected and transferred to a microservice running on the Flask and processing images with a neural network that is trained for the task of road damage detecting.
The work was performed employing well-known and easy-to-learn libraries and software provided with enough tutorials for quick implementation of the project. Computer vision and machine learning technologies are applied as well, which emphasizes the relevance and scientific novelty of the research being conducted. Implementation of the algorithm allows efficient processing of the data received from unmanned systems and enhances the possibilities of their application in geoinformation and research tasks. The developed algorithm may be applied in various fields, including navigation, cartography, geographic information systems, as well as in the research related to the unmanned systems and aerial photography.

Keywords:

Unmanned aerial vehicles, coordinate transformation, geographic coordinates, EPSG3857, EPSG4326, machine learning, Unity, Python, flight simulation, neural networks

References

1. Dzhavadov N.G., Agaev F.G., Guseinov G.A., Zul'fugarly P.R. Trudy MAI, 2022, no. 127. URL: https://trudymai.ru/eng/published.php?ID=170350. DOI: 10.34759/trd-2022-127-20
2. Ol'kina D.S. Trudy MAI, 2023, no. 130. URL: https://trudymai.ru/eng/published.php?ID=174617. DOI: 10.34759/trd-2023-130-18
3. Gumelar O. et al. Remote sensing image transformation with cosine and wavelet method for SPACeMAP Visualization, IOP Conference Series: Earth and Environmental Science. IOP Publishing, 2020, vol. 500, no. 1, pp. 012079. DOI: 10.1088/1755-1315/500/1/012079
4. Harris C. R. et al. Array programming with NumPy, Nature, 2020, vol. 585, no. 7825, pp. 357-362. DOI: 10.1038/s41586-020-2649-2
5. Juliani A. et al. Unity: A general platform for intelligent agents, arXiv preprint arXiv:1809.02627, 2018. URL: https://doi.org/10.48550/arXiv.1809.02627
6. Nenashev V.A., Afanas'eva V.I., Zalishchuk A.A. et al. Trudy MAI, 2023, no. 131. URL: https://trudymai.ru/eng/published.php?ID=175921. DOI: 10.34759/trd-2023-131-15
7. Presnetsov A.M., Tyurin A.P. Intellektual'nye sistemy v proizvodstve, 2023, vol. 21, no. 2, pp. 140-151. DOI: 10.22213/2410-9304-2023-2-140-151
8. He Y. et al. Bounding box regression with uncertainty for accurate object detection, Proceedings of the ieee/cvf conference on computer vision and pattern recognition, 2019, pp. 2888-2897. DOI: 10.1109/CVPR.2019.00300
9. Kalinichenko G.A., Skorokhod S.V. XXXVII Mezhdunarodnaya nauchno-prakticheskaya konferentsiya «Actual scientific research 2018»: sbornik trudov, Moscow, Nauchnyi tsentr "Olimp", 2018, pp. 143-145.
10. Czogalla O., Naumann S. Pedestrian guidance for public transport users in indoor stations using smartphones, IEEE 18th International Conference on Intelligent Transportation Systems, IEEE, 2015, pp. 2539-2544. DOI: 10.1109/ITSC.2015.403
11. Polyantseva K.A. Avtomatizatsiya v promyshlennosti, 2022, no. 5, pp. 32-37. DOI: 10.25728/avtprom.2022.05.09
12. Zaretska I., Kulankhina O., Mykhailenko H., Butenko T. Consistency of UML Design, International Journal of Information Technology and Computer Science, 2018, vol. 10, no. 9, pp. 47-56. DOI: 10.5815/ijitcs.2018.09.06
13. Naugol'nykh E.A., Bartolomei I.L. Modernizatsiya i nauchnye issledovaniya v transportnom komplekse, 2022, vol. 1, pp. 323-325.
14. Zhigalov K.Yu. Ispol'zovanie igrovykh vizualizatorov grafiki v sovremennykh geoinformatsionnykh sistemakh, Cloud of Science, 2016, vol. 3, no. 1, pp. 71-80.
15. Vuksanovic I.P., Sudarevic B. Use of web application frameworks in the development of small applications, 2011 Proceedings of the 34th International Convention MIPRO, IEEE, 2011, pp. 458-462.
16. Kuznetsova S.V. Trudy MAI, 2022, no. 125. URL: https://trudymai.ru/eng/published.php?ID=168193. DOI: 10.34759/trd-2022-125-21
17. Ngo H.H. Vehicle-detection-based traffic density estimation at road intersections, International Journal of Open Information Technologies, 2023, vol. 11, no. 7, pp. 39-46.
18. B.K. Choi, J.Uk. Park, K. Min Roh, S.J. Lee. Comparison of GPS receiver DCB estimation methods using a GPS network, Earth, Planets and Space, 2013, vol. 65, no. 7, pp. 707-711. DOI: 10.5047/eps.2012.10.003
19. N. Ravi, S. Naqvi, M. El-Sharkawy. BIoU: An Improved Bounding Box Regression for Object Detection, Journal of Low Power Electronics and Applications, 2022, vol. 12, no. 4, pp. 51. DOI: 10.3390/jlpea12040051
20. OPENCV. Poluchit' koordinaty mirovoi sistemy koordinat iz koordinat pikselei. URL: https://russianblogs.com/article/9367131104/#_11

Download