Simulation model of high-speed Angara communication network with kD-tor topology


Simonov A. S.

Scientific Research Center of Electronic Computing Technology, 125, Varshavskoye shosse, Moscow, 117587, Russia



The article describes the implementation and experience in application of a simulation model of the Angara high-speed communication network, designed to build multiprocessor computing systems with high performance scalability on its basis.

The studies performed with the simulation model allowed working-out basic solution on the router micro-architecture; algorithms of its components operation; a batch format; basic operation; routing and arbitration algorithms; perform optimization of the router architecture parameters, including basic relation on throughput capabilities, network connectiveness, memory buffers size; balance throughput capabilities of the router components, and perform verification of the custom ASIC. Based on the results of the simulation, the final technical appearance of the router was formed, implemented both in custom ASIC and in Angara communication network equipment based on it.

The experience gained can be employed to create perspective samples of network equipment.


simulation modelling, communication network


  1. Slutskin A.I., Simonov A.S., Zhabin I.A., Makagon D.V., Syromyatnikov E.L. Uspekhi sovremennoi radioelektroniki, 2012, no. 1, pp. 6 – 10.

  2. Zhabin I.A., Makagon D.V., Polyakov D.A., Simonov A.S., Syromyatnikov E.L., Shcherbak A.N. Naukoemkie tekhnologii, 2014, vol. 5, no. 1, pp. 21 – 27.

  3. Simonov A.S., Zhabin I.A., Kushtanov E.R., Makagon D.V., Semenov A.S., Shcherbak A.N. Voprosy kiberbezopasnosti, 2019, no. 4 (32), pp. 46 – 53.

  4. Ravi S., Raghunathan A., Chakradhar S.T. U.S. Patent No. 7,134,100. Washington, DC: U.S. Patent and Trademark Office. 2006, available at:

  5. Mathur A., Wang Q. Power reduction techniques and flows at RTL and system level, Processing of 2009 22nd International Conference on VLSI Design, IEEE, 2009, pp. 28 – 29, doi:10.1109/VLSI.Design.2009.113

  6. Baida Yu.V., Butuzov A.V., Efimov A.G., Tsvetkov M.S. Trudy MFTI, 2012, vol. 4, no. 3, pp. 114 – 121.

  7. Matafonov D.E. Trudy MAI, 2018, no. 103, available at:

  8. Kim D., Celio C., Biancolin D., Bachrach J., Asanovic K. Evaluation of RISC-V RTL with FPGA-accelerated simulation, First Workshop on Computer Architecture Research with RISC-V, 2017, available at:

  9. Pellauer M., Vijayaraghavan M., Adler M. et al. A-Port networks: preserving the timed behavior of synchronous systems for modeling on FPGAs, ACM Transactions on reconfigurable technology and systems, 2009, vol. 2, no. 3, pp. 1 – 26.

  10. Zheng G., Kakulapati G., Kalé L.V. Bigsim: A parallel simulator for performance prediction of extremely large parallel machines, 18th International Parallel and Distributed Processing Symposium, 2004, Santa Fe, NM, pp. 78.

  11. Paul D., Nakhla N.M., Achar R., Nakhla M.S. Parallel simulation of massively coupled interconnect networks, IEEE Transactions on Advanced Packaging, 2009, vol. 33, no. 1, pp. 115 – 127.

  12. Mubarak M., Carothers C.D., Ross R.B., Carns P. Enabling parallel simulation of large-scale HPC network systems, IEEE Transactions on Parallel and Distributed Systems, 2016, vol. 28, no. 1, pp. 87 – 100.

  13. Cerutti I., Corvera J.A., Dumlao S.M., Reyes R., Castoldi P., Andriolli N. Simulation and FPGA-based implementation of iterative parallel schedulers for optical interconnection networks, IEEE/OSA Journal of Optical Communications and Networking, 2017, vol. 9, no. 4, pp. 76 – 87.

  14. Jain N., Bhatele A., White S., Gamblin T., Kale L.V. Evaluating HPC networks via simulation of parallel workloads, Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2016, pp. 154 – 165.

  15. Fujimoto R. Parallel and distributed simulation, Winter Simulation Conference (WSC), IEEE, 2015, pp. 45 – 59.

  16. Simonov A.S. Svidetel’stvo o gosudarstvennoi registratsii programmy dlya EVM № 2014662998 (Certificate of state registration of computer programs “’Angara’Communication network simulation program” no. 2014662998), 12.12.2014.

  17. Kermani P., Kleinrock L. Virtual cut-through: A new computer communication switching technique, Computer networks, 1979, vol. 3, pp. 267 – 286.

  18. Weinberg V. The Apex-MAP Benchmark. PRACE Workshop «New Languages & Future Technology Prototypes», LRZ, 1-2, March 2010, available at:

  19. Dongarra J., Luszczek P., Petitet A. The LINPACK benchmark: past, present and future, Concurrency and Computation: practice and experience, 2003, vol. 15, no. 9, pp. 803 – 820.

  20. Dongarra J., Heroux A., Luszczek P. High-performance conjugate-gradient benchmark: A new metric for ranking high-performance computing systems, The International Journal of High Performance Computing Applications, 2016, vol. 30, no. 1, pp. 3 – 10.

Download — informational site MAI

Copyright © 2000-2020 by MAI