Publication: Cross-Modal Transfer Learning for Image and Sound
| dc.contributor.author | Soroka, A. A. | |
| dc.contributor.author | Trofimov, A. G. | |
| dc.contributor.author | Сорока, Артем Александрович | |
| dc.contributor.author | Трофимов, Александр Геннадьевич | |
| dc.date.accessioned | 2024-12-26T07:22:48Z | |
| dc.date.available | 2024-12-26T07:22:48Z | |
| dc.date.issued | 2022 | |
| dc.description.abstract | © 2022, The Author(s), under exclusive license to Springer Nature Switzerland AG.Recently the research on transfer learning between similar domains has become increasingly common. However, the fields of cross-domain and cross-modal knowledge transfers are more complicated and have been studied less. We propose the new transfer learning strategy between tasks on essentially different domains called as cross-modal transfer learning and consider its ideas and the algorithm. The key element of cross-modal transfer pipeline is cross-modal adapter, i.e. a neural network that transforms the target domain features to the source domain features that can be efficiently processed by a pre-trained neural network. In the experiments the dataset ImageNet and audio dataset ESC-50 are chosen as source domain and target domain respectively. It is shown that a fairly simple neural cross-modal adapter makes it possible to achieve high classification accuracy on target domain using the knowledge obtained by pre-trained neural network on the source domain. Our experiments also show that cross-modal transfer learning noticeably reduces the training time in comparison with the building target model “from scratch”. | |
| dc.format.extent | С. 238-245 | |
| dc.identifier.citation | Soroka, A. A. Cross-Modal Transfer Learning for Image and Sound / Soroka, A.A., Trofimov, A.G. // Studies in Computational Intelligence. - 2022. - 1008 SCI. - P. 238-245. - 10.1007/978-3-030-91581-0_32 | |
| dc.identifier.doi | 10.1007/978-3-030-91581-0_32 | |
| dc.identifier.uri | https://www.doi.org/10.1007/978-3-030-91581-0_32 | |
| dc.identifier.uri | https://www.scopus.com/record/display.uri?eid=2-s2.0-85121585879&origin=resultslist | |
| dc.identifier.uri | https://openrepository.mephi.ru/handle/123456789/28628 | |
| dc.relation.ispartof | Studies in Computational Intelligence | |
| dc.title | Cross-Modal Transfer Learning for Image and Sound | |
| dc.type | Conference Paper | |
| dspace.entity.type | Publication | |
| oaire.citation.volume | 1008 SCI | |
| relation.isAuthorOfPublication | 123f88b2-6d69-42f6-a1e5-c478e56f5e6e | |
| relation.isAuthorOfPublication | d0db4f08-f07a-423f-8847-02b0ea79e6ff | |
| relation.isAuthorOfPublication.latestForDiscovery | 123f88b2-6d69-42f6-a1e5-c478e56f5e6e | |
| relation.isOrgUnitOfPublication | 010157d0-1f75-46b2-ab5b-712e3424b4f5 | |
| relation.isOrgUnitOfPublication.latestForDiscovery | 010157d0-1f75-46b2-ab5b-712e3424b4f5 |