Deduplication potential of HPC applications' checkpoints
conference contributionposted on 06.02.2018, 11:34 authored by Jurgen Kaiser, Ramy Gad, Tim Suss, Federico Padua, Lars NagelLars Nagel, Andre Brinkmann
© 2016 IEEE. HPC systems contain an increasing number of components, decreasing the mean time between failures. Checkpoint mechanisms help to overcome such failures for long-running applications. A viable solution to remove the resulting pressure from the I/O backends is to deduplicate the checkpoints. However, there is little knowledge about the potential to save I/Os for HPC applications by using deduplication within the checkpointing process. In this paper, we perform a broad study about the deduplication behavior of HPC application checkpointing and its impact on system design.
This work was partially supported by the German Ministry for Education and Research (BMBF) under project grant 01IH13004 (Project FAST) and the Intel Parallel Computing Centers (IPCC) project.
- Computer Science