Modeling I/O interference for data intensive distributed applications

Sven Groot, Kazuo Goda, Daisaku Yokoyama, Miyuki Nakano, Masaru Kitsuregawa

Research output: Chapter in Book/Report/Conference proceedingConference contribution

13 Citations (Scopus)

Abstract

Data intensive applications such as MapReduce can have large performance degradation from the effects of I/O interference when multiple processes access the same I/O resources simultaneously, particularly in the case of disks. It is necessary to understand this effect in order to improve resource allocation and utilization for these applications. In this paper, we propose a model for predicting the impact of I/O interference on MapReduce application performance. Our model takes basic parameters of the workload and hardware environment, and knowledge of the I/O behavior of the application to predict how I/O interference affects the scalability of an application. We compare the model's predictions for several workloads (TeraSort, WordCount, PFP Growth and PageRank) against the actual behavior of those workloads in a real cluster environment, and confirm that our model can provide highly accurate predictions.

Original languageEnglish
Title of host publication28th Annual ACM Symposium on Applied Computing, SAC 2013
Pages343-350
Number of pages8
DOIs
Publication statusPublished - 27 May 2013
Event28th Annual ACM Symposium on Applied Computing, SAC 2013 - Coimbra, Portugal
Duration: 18 Mar 201322 Mar 2013

Publication series

NameProceedings of the ACM Symposium on Applied Computing

Conference

Conference28th Annual ACM Symposium on Applied Computing, SAC 2013
CountryPortugal
CityCoimbra
Period18/03/1322/03/13

Keywords

  • Cloud computing
  • Data Inte I/O Interference
  • I/O behavior
  • Mapreduce

Fingerprint Dive into the research topics of 'Modeling I/O interference for data intensive distributed applications'. Together they form a unique fingerprint.

  • Cite this

    Groot, S., Goda, K., Yokoyama, D., Nakano, M., & Kitsuregawa, M. (2013). Modeling I/O interference for data intensive distributed applications. In 28th Annual ACM Symposium on Applied Computing, SAC 2013 (pp. 343-350). (Proceedings of the ACM Symposium on Applied Computing). https://doi.org/10.1145/2480362.2480434