Please note that this workshop will be conducted in Japanese.
Overview
In recent years, the realization of "Open Science" has attracted significant attention. To support this movement, it is important not only to share research outcomes, but also to record and make available the underlying data and analysis procedures that led to those outcomes.
This workshop introduces the use of a provenance management system that records and manages the execution history of simulations and data analyses performed on a supercomputer. Through practical use cases, participants will learn what can be achieved by leveraging provenance information, including the complete workflow from job submission and provenance data search/browsing to linking provenance records with research publications.
What is a Provenance Management System?
A provenance management system records and manages the history (provenance information) of how research results are generated, including the input data, programs, and execution conditions used in the computation. It helps ensure research reproducibility and enables researchers to trace computational processes.
Among The University of Osaka's supercomputer systems, OCTOPUS is equipped with the Data Provenance System for HPC (DPS4H), which automatically collects and visualizes job provenance information.
Simply submitting a job to the "DPS" queue automatically records provenance information, including relationships among input/output files and execution scripts used by the job. The recorded provenance information can be searched and viewed through a web browser or CLI. In addition, a unique URL (permanent link) can be generated and cited in research papers.
For more information on research related to the provenance management system, please refer to the following article.
HPSC-News vol.20 A Provenance Recording System for Research Data Management
Recommended for
- Users who are new to DPS4H
- Users who want to preserve provenance information for their jobs
- Users who want to indicate in papers or reports which job generated a particular dataset
Program
| No. | Program | Description |
|---|---|---|
| 1 | Use Cases |
Introduction to characteristic use cases of DPS4H. Submit a job to the “DPS” queue → Provenance information is automatically recorded → Search and view the information through a web browser → Include a permanent link in an academic paper |
| 2 | DPS4H Overview |
Mechanisms for collecting provenance information Interfaces for displaying collected provenance information |
| 3 | Login Configuration |
What is Keycloak authentication? (Differences from OCTOPUS OTP) Initial OTP setup procedure |
| 4 | Collecting and Searching Provenance Information |
Submitting jobs to the “DPS” queue Converting an NQSV Job ID to a DPS4H JOB_ID Search and graph viewing operations Collecting provenance information from the CLI |
| 5 | Using Permanent Links |
Issuing and sharing links Access control and publication settings How to cite links in academic papers |
| 6 | Q&A Session | Questions and answers from participants |
Notes for Participants
- This workshop will be held online using a web conferencing tool. Please ensure that you have a stable internet connection.
Workshop Materials
The workshop materials will be made available after the workshop.
Date : August 25 (Tue), 2:00 PM - 3:00 PM
Instructor: NEC Corporation
Venue: Online
Type : Lecture
Quota: 30 participants
Application deadline: Applications accepted until the day of the workshop
