Pedro Miguel Marcos Joaquim
Leveraging Transient Resources for Incremental Graph Processing on Heterogeneous Infrastructures
Tese submetida para provas de mestrado em Engenharia
Informática e de Computadores Instituto Superior
Técnico, Universidade de Lisboa.
Abstract
Incremental graph processing as emerged as a key technology with
applications in many different areas. This task is typically executed
in public cloud infrastructures that offer a wide variety of
services. The heterogeneity of the provided resources and the
different renting services available, such as reserved or
transient-lived resources, offer a good opportunity to match cost with
intended deployment characteris- tics. In this paper we present
Hourglass, a deployment engine for existing incremental graph
processing systems that aims at leveraging the existing heterogeneity
in such deployment environments, as well as transient resource markets
to significantly reduce costs. The system exploits features of the
workload, such as the imbalance in vertex communication patterns, to
do a resource-aware distribution of graph partitions among the
existing machines. Hourglass constantly analyses market conditions to
find the best possible deployment in specific moments in
time. Knowledge about the computation being per- formed allows
Hourglass to choose the right moment to transition between deployments
and allow the underlying system to operate on a fraction of the cost
of a system composed exclusively of reserved machines, with minimal
impact on the system performance.
Publicações
- Leveraging Transient Resources for Incremental Graph
Processing on Heterogeneous Infrastructures
- Pedro
Miguel Marcos Joaquim
- MSc Thesis. Instituto Superior
Técnico, Universidade de Lisboa.
- November, 2017.
- Available BibTeX, MSC Thesis, and extended abstract, and mid-term report.
- Processamento Incremental de Grafos em Sistemas
Heterogêneos.
- P. Joaquim, M. Bravo,
R. Rodrigues, M. Matos and L. Rodrigues.
- Actas do nono
Simpósio de Informática (Inforum), Lisboa, Portugal,
Oct. 2017.
- (Poster)
- Available BibTeX, extended report
(pdf).
- Hourglass: Leveraging Transient Resources for
Time-Constrained Graph Processing in the Cloud.
- P. Joaquim, M. Bravo, L. Rodrigues, M. Matos
- In Proceedings
of the EuroSys 2019, Dresden, Germany, March 2019.
Luís Rodrigues