Publication: Methods for Speeding Up the Retraining of Neural Networks
Дата
2022
Авторы
Varykhanov, S. S.
Sinelnikov, D. M.
Odintsev, V. V.
Rovnyagin, M. M.
Mingazhitdinova, E. F.
Journal Title
Journal ISSN
Volume Title
Издатель
Аннотация
© 2022 IEEE.Nowadays, machine learning is widespread and is becoming more complex. Developing and debugging neural networks is becoming more and more time-consuming. Distributed solutions are often used to speed up the learning process. But these solutions do not solve the problem of retraining model from zero if the learning fails. This paper presents a new approach to training models on a large datasets, which can save time and resources during the development. This approach is splitting the model's learning process into separate layers. Each of these layers can be modified and reused for the next layers. The implementation of this approach is based on transfer learning and distributed machine learning techniques. To create reusable network layers, it is proposed to use the methods of automating code parallelization for hybrid computing systems described in the article. These methods include: tracking the readiness and dependencies in the data, speculative execution at the kernel level, creating a DSL
Описание
Ключевые слова
Цитирование
Methods for Speeding Up the Retraining of Neural Networks / Varykhanov, S.S. [et al.] // Proceedings of the 2022 Conference of Russian Young Researchers in Electrical and Electronic Engineering, ElConRus 2022. - 2022. - P. 478-481. - 10.1109/ElConRus54750.2022.9755557