Adaptive Request Scheduling for the I/O Forwarding Layer - Grid'5000 Accéder directement au contenu
Pré-Publication, Document De Travail Année : 2019

Adaptive Request Scheduling for the I/O Forwarding Layer

Résumé

In this paper, we present an approach to adapt the HPC I/O forwarding layer to the application access patterns. I/O forwarding is a technique used in most supercomputers today to alleviate contention in the access to the shared storage infrastructure. Because of its location, between processing nodes and parallel file system servers, it has been used to propose optimization techniques such as request reordering, aggregation, and scheduling. Such techniques can usually provide good results only for some of the situations, or depend on the right choice of parameter values. Our case study for this work is the TWINS request scheduling algorithm, which aims at coordinating the access of intermediate I/O nodes to the data servers. Our approach uses a neural network to classify application access patterns, and a reinforcement learning technique to empower the scheduler to learn the best parameter values to each access pattern during its execution, without the need of a previous training phase. Our evaluation of the access pattern detection neural network shows average precision of 98% during write experiments, and minimum precision of 98% during reads. The latter is an important result as most performance improvements by TWINS were observed for read experiments. Furthermore, we demonstrate that our contextual bandit strategy is able to learn the best value for the window size, achieving approximately 75% of precision-98% of the performance provided by the best window size-in the first hundreds of steps.
Fichier principal
Vignette du fichier
CCGrid_2018___Adaptive_TWINS (1).pdf (768.69 Ko) Télécharger le fichier
Origine : Fichiers produits par l'(les) auteur(s)
Loading...

Dates et versions

hal-01994677 , version 1 (25-01-2019)
hal-01994677 , version 2 (26-02-2019)
hal-01994677 , version 3 (16-10-2019)

Identifiants

  • HAL Id : hal-01994677 , version 1

Citer

Jean Luca Bez, Francieli Zanon Boito, Ramon Nou, Alberto Miranda, Toni Cortes, et al.. Adaptive Request Scheduling for the I/O Forwarding Layer. 2019. ⟨hal-01994677v1⟩
496 Consultations
488 Téléchargements

Partager

Gmail Facebook X LinkedIn More