Vestnik КRAUNC. Fiz.-Mat. nauki. 2023. vol. 43. no. 2. P. 69-86. ISSN 2079-6641

INFORMATION AND COMPUTATION TECHNOLOGIES
https://doi.org/10.26117/2079-6641-2023-43-2-69-86
Research Article
Full text in Russian
MSC 68T99

Contents of this issue

Read Russian Version

Applicability of Genetic Algorithms for Determining the Weighting Coefficients of an Artificial Neural Network with One Hidden Layer

A. D. Smorodinov¹²^\ast, T. V. Gavrilenko¹², V. A. Galkin¹²

¹Surgut Branch of SRISA 628426, Surgut, Energetikov st., 4, Russia

²Surgut State University, 628412, Surgut, Lenina st., 1, Russia

Abstract. In the training of an artificial neural network, one of the central problems is the initial initialization and adjustment of weighting coefficients associated with pseudo-random initialization of weighting coefficients. The article describes a basic genetic algorithm, as well as a method for determining weight coefficients using this algorithm. A combined method for determining weighting coefficients is also presented, which provides for initial initialization using a genetic algorithm at the first stage and the use of stochastic gradient descent at the second stage of training, the proposed methods are tested on a number of artificial neural networks of direct propagation for various tasks of binary classification of real and synthetic data, as well as for unambiguous multiclass classification of handwritten digits on images from the database MNIST data. Artificial neural networks are constructed on the basis of the Kolmogorov-Arnold theorem. This article presents a comparative analysis of two methods for determining weight coefficients – using a genetic algorithm and gradient descent. Based on the results of the comparative analysis, it is concluded that a genetic algorithm can be used to determine the weighting coefficients both as an algorithm for the initial initialization of an artificial neural network and as an algorithm for adjusting the weighting coefficients.

Key words: artificial neural networks, genetic algorithm, Kolmogorov-Arnold theorem, neural network training.

Received: 21.04.2023; Revised: 08.06.2023; Accepted: 10.06.2023; First online: 29.06.2023

For citation. Smorodinov A. D., Gavrilenko T. V., Galkin V. A. Applicability of Genetic Algorithms for Determining
the Weighting Coefficients of an Artificial Neural Network with One Hidden Layer. Vestnik KRAUNC. Fiz.-mat. nauki.
2023, 43: 2, 69-86. EDN: YWFIZE. https://doi.org/10.26117/2079-6641-2023-43-2-69-86.

Funding. The publication was made within the framework of the state task of the Federal State Institution FNTs NIISI
RAS (Performance of fundamental scientific research GP 47) on topic No. 0580-2021-0007 «Development of methods for mathematical modeling of distributed systems and corresponding calculation methods» here

Competing interests. There are no conflicts of interest regarding authorship and publication.

Contribution and Responsibility. All authors contributed to this article. Authors are solely responsible for providing
the final version of the article in print. The final version of the manuscript was approved by all authors.

^\astCorrespondence: E-mail: Sachenka_1998@mail.ru

The content is published under the terms of the Creative Commons Attribution 4.0 International License

© Smorodinov A. D., Gavrilenko T. V., Galkin V. A., 2023

© Institute of Cosmophysical Research and Radio Wave Propagation, 2023 (original layout, design, compilation)

References

  1. Galkin V. A., Gavrilenko T. V., Smorodinov A. D. Some aspects of approximation and interpolation of functions artificial neural networks. Vestnik KRAUNC. Fiz.-mat. nauki. 2022, 38: 1, 54-73. DOI: 10.26117/2079-6641-2022-38-1-54-73, (In Russian).
  2. Holland J. H. Adaptation in Natural and Artificial Systems: An Introductory Analysis With Applications to Biology, Control, and Artificial Intelligence. The MIT Press, Cambridge, 1992.
  3. Goldberg D. Genetic Algorithms in Search, Optimization, and Machine Learning. Massachusetts: Addison-Wesley, 1989.
  4. Chollet F. Deep Learning with Python. Simon and Schuster, N.Y., USA. 2021, p. 384.
  5. Zakharov I. V. , Shushakov A. O., Zykova S. S. The Choice of Structures of Heterogeneous Information-Computer Systems Based on the Apparatus of Genetic Algorithms. Transportda intellektual texnologiyalar , 2022, 3(31), 46-51, DOI: 10.24412/2413-2527-2022-331-46-51
  6. Bedin D. A., Ivanov A. G. Ispol’zovaniye geneticheskogo algoritma dlya opredeleniya parametrov mnogogipoteznogo algoritma vosstanovleniya trayektorii vozdushnogo sudna. XXVI Sankt-Peterburgskaya mezhdunarodnaya konferentsiya po integrirovannym navigatsionnym sistemam. Sbornik materialov, Sankt-Peterburg, 27–29 maya 2019 goda, Sankt-Peterburg, “Kontsern “Tsentral’nyy nauchno-issledovatel’skiy institut “Elektropribor 2019, 87-90, (In Russian).
  7. Alhussain A. H. Symmetric encryption algorithm using genetic algorithm and pseudorandom number generators. Yestestvennyye i tekhnicheskiye nauki, 2015, 7(85), 75-81 (In Russian).
  8. Stefanyuk V. L., Alkhussayn A. KH. Kriptografiya s simmetrichnym klyuchom s ispol’zovaniyem geneticheskogo algoritma, KII-2014. Chetyrnadtsataya natsional’naya konferentsiya po iskusstvennomu intellektu s mezhdunarodnym uchastiyem. T. 1, Kazan’: RITS «Shkola», 2014, 267-275 (In Russian).
  9. Trokoz D. A. Machine learning algorithm for wide neural networks using hyperdimensional binary vector algebra and genetic algorithms. Yuzhno-Sibirskiy nauchnyy vestnik, 2020, 6(34), 148-154 (In Russian).
  10. Nair V., Hinton G.E. Rectified Linear Units Improve Restricted Boltzmann Machines. 27th International Conference on International Conference on Machine Learning. USA: Omnipress, 2010, 807–814.
  11. Andrew L. Maas, Awni Y. Hannun, Andrew Y. Ng. Rectifier nonlinearities improve neural network acoustic models, Proc. ICML. 2013, 30, 1.
  12. Tieleman T., Hinton G. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Networks for Machine Learning. 2012.
  13. Kolmogorov A. N. On the representation of continuous functions of many variables by superposition of continuous functions of one variable and addition, Doklady Akademii Nauk SSSR, 1957, 114, 5, P. 953–956, (In Russian).
  14. Klambauer G., Unterthiner T., Mayr A., Hochreiter S. Self-Normalizing Neural Networks, Advances in Neural Information Processing Systems. 2017, 30, 2017.
  15. Clover Y.-A., Untertiner T., Hochreiter S. Fast and accurate deep network learning using exponential linear units, arXiv preprint, arXiv:1511.07289, 2015.

Smorodinov Aleksandr Denisovich – Surgut State University, Postgraduate Student of the Department of Applied Mathematics, Lecturer of the Department of ASOIU, Branch of SRISA, Engineer of the Department of Biophysics and Neurocybernetics. ORCID 0000-0002-9324-1844.


Gavrilenko Taras Vladimirovich – PhD (Tech.), Surgut State University, docent, Branch of SRISA, Deputy Director. ORCID 0000-0002-3243-2751.


Galkin Valery Alekseevich – D. Sci. (Phys. & Math.), Professor, Surgut State University, Professor, Doctor of Physical and Mathematical Sciences. Branch of SRISA, Director. ORCID 0000-0002-9721-4026.