|
ИСТИНА |
Войти в систему Регистрация |
ФНКЦ РР |
||
Optimization of Network Interaction on a High-Performance Cluster Using Graph Scheduling //Редакционная коллегиястатья
- Авторы: Timokhin I.S., Shaikhislamov D.I., Teplov A.M.
- Сборник: PROCEEDINGS OF THE 5TH INTERNATIONAL WORKSHOP ON INFORMATION, COMPUTATION, AND CONTROL SYSTEMS FOR DISTRIBUTED ENVIRONMENTS (ICCS-DE 2023)
- Год издания: 2023
- Место издания: Федеральное государственное бюджетное учреждение науки Институт динамики систем и теории управления имени В.М. Матросова Сибирского отделения Российской академии наук Иркутск
- Первая страница: 141
- Последняя страница: 148
- Аннотация: Data access time become large problem for data processing in distributed environments with a growthof system scale and data size and source of software optimization. In this paper, we research the problemof optimizing I/O and processing operations for a distributed Hadoop cluster that process data in HPCparadigm framework. To increase the efficiency of the framework that processes queries in HDFS, agraph algorithm were developed with respect to optimal use of resources using HDFS data locality ona processing graph. The graph uses data file blocks and replicas, hosts and workers as a nodes, andlinks between them as edges. The results of reading stage optimization phase was implemented andperformance improvements measured comparing baseline and Spark as Industry standard framework.
- Добавил в систему: Теплов Алексей Михайлович