In addition to parallelism internode with MPI, automatic parallelization and intra-node parallel execution using OpenMP are possible. Please read the following first, as you need to insert OpenMP directives.
How to use OpenMP
Compile
You can use Intel MPI compiler on OCTOPUS. Please note that commands are different for each programming language.
MPI and OpenMP
$ module load BaseCPU
$ mpiifx -qopenmp [options] source_file (Fortran)
$ mpiicx -qopenmp [options] source_file (C)
$ mpiicpx -qopenmp [options] source_file (C++)
Execution Script
The following is an example of a script for running MPI and OpenMP programs.
This script executes MPI batch request with 4 nodes parallel execution (Intel MPI), 256 parallel execution per node (OpenMP), and 1 hour elapsed time. Please specify the same value for "cpunum_job" (number of cores used per node), specified by -l option, and "OMP_NUM_THREADS" (number of OpenMP processes per node), specified by -v option, must be the same value, otherwise execution performance may be degraded.
|
1 2 3 4 5 6 7 8 9 10 |
#!/bin/bash #PBS -q OCT #PBS --group=[group name] #PBS -l cpunum_job=256,elapstim_req=1:00:00 #PBS -T intmpi #PBS -b 4 #PBS -v OMP_NUM_THREADS=256 cd $PBS_O_WORKDIR module load BaseCPU mpirun ${NQSV_MPIOPTS} -np 4 ./a.out |
Notes on multi-node execution
If you specify options or environment variables with "setenv (option name)" in the job script, they are set only on the master node and not on the slave nodes. If you want the settings to be reflected on all nodes, specify "#PBS -v (option name)". The details are described in the following.
How to write a job script Specifying environment variables
