Virtual Institute — High Productivity Supercomputing

Process-oriented Performance Engineering (ProPE)

DFG (2017-2019)

This project is part of the DFG Program “Performance Engineering for Scientific Software”. The official project website can be found here.

HPC competence in German HPC centers is distributed across the country. The Gauss-Allianz is an initiative to integrate and organize Tier 2/3 HPC landscape in Germany. Furthermore, there are multiple local efforts: bwHPC, KONWIHR, HKHLR, HLRN and JARA-HPC. Our contribution is to integrate with and built on already existing efforts and further drive the final goal of a hierarchical and yet integrated German HPC infrastructure with an emphasis on Performance-Engineering (PE).

Structured PE Process

The core activity of analyzing and optimizing application performance is guided by a systematic PE-Process. At its core are typical performance limiting settings called performance patterns. To validate and to get a quantitative view of a pattern white box performance models are employed. Identifying a performance pattern is achieved by a set of hardware performance counter metrics but might also involve static code analysis and benchmarking results.

PE Support Infrastructure & Process Management

To quantify the improvement of resource efficiency promised by applying a structured PE process, we investigate cost models that include total ownership costs for human tuning efforts as well as hardware- and performance-related factors. Moreover, we establish a blueprint for a multi-tier distributed process management service and support structures that seamlessly integrate local and remote support levels to provide “in-depth” expertise distributed across sites.

Application Monitoring

Global automatic application performance monitoring is essential to improve efficient usage of HPC systems. Therefore, we establish an infrastructure that creates databases with performance footprints and performance maps to characterize applications and track HPC usage statistics. This enables the identification of applications with high optimization potential or pathological performance behavior and allows us to give users feedback on job runs.

HPC Curriculum

Furthermore, we aim at a national HPC training program (curriculum) for all kind of target audiences. The curriculum is based on existing training activities with focus on target groups and knowledge levels. To foster well-educated trainers in the area of performance engineering, we include train-the-trainer tracks, give trainings and establish concepts for the promotion of trainings.

Documentation & Dissemination

To increase the awareness and provide information on performance engineering, we build a central hub including a knowledgebase with HPC and performance engineering materials, sorted to addressing different target groups. This includes downloadable teaching material, FAQs and case studies with exemplary application of our structured PE process.

Partners

Associated Partners

  • KONWIHR
  • TU Munich (Prof. Bungartz)
  • Forschungszentrum Jülich
  • Technical University Bergakademie Freiberg

Sponsor

This project is part of the DFG Program “Performance Engineering for Scientific Software”.