HPCTESTS 2025

Third International Workshop on HPC Testing and Evaluation of Systems, Tools, and Software

https://olcf.github.io/hpc-system-test-wg/hpctests/hpctests2025

Details

Description

The HPCTESTS workshop brings together HPC researchers, operators, and vendors from around the globe to present and discuss state-of-the-art HPC system testing methodologies, tools, benchmarks, procedures, and best practices. The increasing complexity of HPC architectures and growing need to leverage HPC for integrated workflows more extensive testing than ever in order to thoroughly evaluate the status of the system after installation or a software upgrade and ensure proper operation before it is transitioned to production. Different methodologies are used to evaluate systems during their lifetime, not only at the beginning during the installation, but also during maintenance windows and alongside regular operations. This workshop provides a venue to present and discuss the latest HPC system testing technologies and methodologies.

The event will include an opening talk focused on current HPC system testing topics, followed by a series of paper presentations from peer-reviewed accepted submissions and concluding with a panel discussion.

Call for Papers

The Third International Workshop on HPC Testing and Evaluation of Systems, Tools, and Software (HPCTESTS) will bring together experts from high performance computing (HPC) centers and vendors around the globe to present and discuss state-of-the-art HPC system testing methodologies, tools, and best practices. The workshop will encourage submissions that highlight current benchmarks, tests, and procedures utilized in today’s HPC systems. This event will provide an avenue to showcase newly developed tools and methodologies, and those that are being actively designed to allow authors to gather feedback from the community that could help guide their project. As machine learning (ML), deep learning (DL), and integrated workflows become more prevalent workloads, HPC centers must provide a wider range of services and more robust and resilient resources in order to support both traditional HPC and emerging workloads. The workshop will also invite submissions that are looking ahead at the post-exascale future of HPC system testing to initiate investigation of alternate mechanisms that could be used to adapt to the evolving and emerging workloads. The event will invite and welcome international participation from HPC centers, academic institutions, and representatives from vendors in the supercomputing space.

In addition to discussing procedures and tools utilized, submissions can describe challenges, lessons learned, and best practices used for regression testing, acceptance testing, and hardware evaluations. Furthermore, the workshop aims to encourage submissions that explore testbed evaluations as a means to gather preliminary results on system readiness to assist system design and deployment efforts.

This half-day workshop will kick off with an opening presentation focused on HPC system testing at scale. The session will be followed by a series of paper talks selected from the open call for submissions that will be held. The workshop will conclude with a panel discussing key topics impacting the HPC system testing community.

The workshop description, program committee, Call for Papers (CFP), papers, and presentations will be archived in the HPC System Test Working Group website (https://olcf.github.io/hpc-system-test-wg/) under a dedicated section for the HPCTESTS workshop.

Due to the wide range of systems currently in production and the rapidly evolving landscape of supercomputer architectures, the community should leverage individual center efforts to develop a collaborative set of best practices that will evolve from the publication of current tools, tests, and benchmarks. The diversity of architectures, compounded with the distinct user workloads supported at each HPC center has resulted in each center developing their own procedures, tests, benchmarks, and tools to conduct acceptance and regression testing. However, as we have learned through our events and workshops, these mechanisms are conceptually similar, and the tests utilized could be leveraged by multiple centers. In the workshop, we hope to attract stakeholders from vendors, HPC center staff, students, and faculty, interested in exploring HPC system testing topics more in depth.

Topics of interest include, but are not limited to:

Paper Submissions

The workshop will publish its proceedings with the SC25 conference. Authors must follow the formatting guidelines from SC25 Papers which are available here. Submissions can be 5-10 two-column pages (U.S. letter – 8.5 inches x 11 inches), excluding the bibliography, using the ACM proceedings template. Latex users, please use the “sigconf” option (use of the “review” option is recommended but not required). Word authors can use the “Interim Layout”. See the templates here.
Please also note the reproducibility initiative, which we follow for HPCTESTS.

Submissions will be accepted through the SC25 Submissions site: https://submissions.supercomputing.org/. After you create an account, you will be able to submit to the HPCTESTS 2025 form.

Workshop Deadlines

Organizing Committees

HPCTESTS 2025 General Chairs

HPCTESTS 2025 Program Chairs

HPCTESTS 2025 Steering Committee

TBC

HPCTESTS 2025 Program Committee