- Erweiterung von h5bench um I/O-Zugriffsmuster in gängigen KI-Anwendungen
Djebarov, Dlyaver; Müller, Matthias S. (Thesis advisor); Neuwirth, Sarah (Thesis advisor); Liem, Radita Tapaning Hesti (Consultant)
Aachen : RWTH Aachen University (2024)
Bachelor Thesis
Bachelorarbeit, RWTH Aachen University, 2024
Abstract
Rapid artificial intelligence (AI) adoption in scientific computing requires new tools to evaluate I/O performance effectively. HDF5 is one of the data formats commonly used not only in HPC applications but also in modern AI applications. However, the existing benchmarks are insufficient to address the current challenges posed by AI workloads. This thesis introduces an extension to the existing HDF5 benchmark, called h5bench, by incorporating the same workload from the MLPerf Storage - DLIO Benchmark. This extension allows users to test AI workloads without the need to install machine learning libraries, reducing complexity and enhancing the usability of the benchmark. The experimental analysis demonstrates that the extension managed to replicate the existing I/O patterns with easy-to-adjust configurations to perform various scalability tests.
Institutions
- IT Center [022000]
- Department of Computer Science [120000]
- Chair of High Performance Computing (Computer Science 12) [123010]