Explaining How to Perform Node-to-Node Parallel (Distributed Memory Parallel) Processing with OpenMPI on SQUID

Compilation Method

    $ module load BaseGCC
    $ mpif90 [options] source_file (For Fortran,)
    $ mpicc [options] source_file (For C,)
    $ mpic++ [options] source_file (For C++,)

    If you are using GCC 11.4 + OpenMPI 4.1.5, specify the following instead of BaseGCC:

    $ module load /system/apps/env/gcc/Compiler/gcc/11.4.0
    $ module load /system/apps/env/gcc/mpi/gcc11.4.0/openmpi/4.1.5cu11.8


Execution Script

    The command to execute MPI is as follows:

    mpirun ${NQSV_MPIOPTS} -np (total parallel number) executable_name


    An example script for executing an MPI batch request with a duration of 1 hour, running in parallel across 4 nodes, with 76 parallel executions per node (totaling 304 parallel executions), is as follows:


    Points to Note for Multi-Node Execution

      If you specify options or environment variables in the job script with setenv (option name), they will be set only on the master node and not on the slave nodes.
      If you want to reflect the settings on all nodes, please specify #PBS -v (option name).
      For more details, please refer to the following:

      How to Write a Job Script - Specifying Environment Variables



Reference Materials