Publication:
Data Handling Optimization in Russian Data Lake Prototype

Простая страница элемента
dc.contributor.authorAlekseev, A.
dc.contributor.authorKiryanov, A.
dc.contributor.authorKlimentov, A.
dc.contributor.authorSmirnov, S.
dc.contributor.authorСмирнов, Сергей Юрьевич
dc.date.accessioned2024-12-28T09:30:13Z
dc.date.available2024-12-28T09:30:13Z
dc.date.issued2023
dc.description.abstractAbstract CERN experiments are preparing for the HL-LHC era, which will bring an unprecedented volume of scientific data. These data will need to be stored and processed by thousands of physicists, but expected resource growth is nowhere near the extrapolated requirements of existing models, in terms of both storage volume and compute power. Opportunistic CPU resources such as HPCs and university clusters can provide extra CPU cycles, but there is no opportunistic storage. In this article, we will present the main architectural ideas, deployment details, and test results, with emphasis on our research to build a prototype of a distributed data processing and storage system with a focus on optimizing the efficiency of resources by reducing overhead costs for accessing the data. The described prototype was built using the geographically distributed WLCG sites and university clusters in Russia.
dc.identifier.citationData Handling Optimization in Russian Data Lake Prototype / Alekseev, A. [et al.] // Journal of Physics: Conference Series. - 2023. - 2438. - № 1. - 10.1088/1742-6596/2438/1/012021
dc.identifier.doi10.1088/1742-6596/2438/1/012021
dc.identifier.urihttps://www.doi.org/10.1088/1742-6596/2438/1/012021
dc.identifier.urihttps://www.scopus.com/record/display.uri?eid=2-s2.0-85149722430&origin=resultslist
dc.identifier.urihttp://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcAuth=Alerting&SrcApp=Alerting&DestApp=WOS_CPL&DestLinkType=FullRecord&UT=WOS:001026601300021
dc.identifier.urihttps://openrepository.mephi.ru/handle/123456789/30034
dc.relation.ispartofJournal of Physics: Conference Series
dc.subjectTask Scheduling
dc.subjectParallel Computing
dc.subjectDistributed Storage
dc.subjectHigh-Performance Computing
dc.subjectComputational Grids
dc.titleData Handling Optimization in Russian Data Lake Prototype
dc.typeConference Paper
dspace.entity.typePublication
oaire.citation.issue1
oaire.citation.volume2438
relation.isAuthorOfPublication127ddeae-2341-4a83-a3bb-202d7ed259ec
relation.isAuthorOfPublication.latestForDiscovery127ddeae-2341-4a83-a3bb-202d7ed259ec
relation.isOrgUnitOfPublicationba0b4738-e6bd-4285-bda5-16ab2240dbd1
relation.isOrgUnitOfPublication.latestForDiscoveryba0b4738-e6bd-4285-bda5-16ab2240dbd1
Файлы
Original bundle
Теперь показываю 1 - 1 из 1
Уменьшенное изображение
Name:
W4320915898.pdf
Size:
778.04 KB
Format:
Adobe Portable Document Format
Description:
Download
Коллекции
Публикации