In this page, We describe how to install and use FLASH code on OCTOPUS.
Download Software
Please downloade FLASH code from this page. User registration required.
FLASH code(Flash Center, University Chicago)
Preparing compile
For compiling FLASH code, please set OpenMPI and install HDF5 and PnetCDF, HYPRE.
setting OpenMPI
-
You should compule FLASH code and other relative software by OpenMPI and GNU compuler. Please set it by the following command before compile.
BASE=/octfs/apl/HPC-X/hpcx-v2.2.0-gcc-MLNX_OFED_LINUX-4.2-1.2.0.0-redhat7.3-x86_64
source ${BASE}/hpcx-init.sh
hpcx_load
Please see the following page for how to use OpenMPI.
How to use OpenMPI
install HDF5
1. Download hdf5-1.8.20 from the following page
2. Please extract tar ball.
tar xvzf hdf5-1.8.20.tar.gz
3. make,make install
cd hdf5-1.8.20
./configure --prefix=/octfs/work/(your group name)/(your user ID)/HDF5/ --enable-fortran CC=mpicc F9X=mpif90 --enable-parallel
make
make install
Install PnetCDF
1. Downloade PnetCDF-1.9.0 from the following page.
2. Please extract tar ball.
tar xvzf parallel-netcdf-1.9.0.tar.gz
3. make,make install
cd parallel-netcdf-1.9.0
./configure --prefix=/octfs/work/(your group name)/(your user ID)/PnetCDF/
make
make install
HYPRE install
1. Download HYPRE-2.11.2 from the following page.
2. Please extract tar ball.
tar -xzvf hypre-2.11.2.tar.gz
3. make,make install
cd hypre-2.11.2/src
./configure --prefix=/octfs/work/(your group name)/(your user ID)/HYPRE/ CC=mpicc F9X=mpif90
make
make install
how to complile
-
In this items, we descrive how to compule LaserSlab Simulation on FLASH code4.5.
Please see the following official manual as well.
FLASH code support page
1.Please extract FLASH code.
2.execute "setup script"
cd FLASH4.5/
./setup -auto LaserSlab
3.modify Makefile
cd object
vi Makefile.h
Please modify as the following
MPI_PATH = /octfs/apl/HPC-X/hpcx-v2.2.0-gcc-MLNX_OFED_LINUX-4.2-1.2.0.0-redhat7.3-x86_64/ompi/
HDF5_PATH = /octfs/work/(your group name)/(your user ID)/HDF5
NCMPI_PATH = /octfs/work/(your group name)/(your user ID)/PnetCDF
HYPRE_PATH = /octfs/work/(your group name)/(your user ID)/HYPRE
* please check your group name by "id -gn" command.
4.set LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/octfs/work/(your group name)/(your user ID)/HDF5/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/octfs/work/(your group name)/(your user ID)/PnetCDF/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH=/octfs/work/(your group name)/(your user ID)/HYPRE/lib:$LD_LIBRARY_PATH
5.make
make
If you re-compile, please execute "make" command after executing "make clean". When compiling success, you get "flash4" execution file.
how to execute
-
We permit only batch request for executing FLASH code.
batch script
The following example is a job script FRASH code executes on 96 porcess(4 nodes, 24 proccess per node).
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 |
#!/bin/bqsh #PBS -q OCTOPUS #PBS -l elapstim_req=1:00:00,stacksz_prc=8388608 #PBS -T openmpi #PBS -b 4 BASE=/octfs/apl/HPC-X/hpcx-v2.2.0-gcc-MLNX_OFED_LINUX-4.2-1.2.0.0-redhat7.3-x86_64 source ${BASE}/hpcx-init.sh hpcx_load export LD_LIBRARY_PATH=/octfs/work/(your group name)/(your user ID)/HDF5/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/octfs/work/(your group name)/(your user ID)/PnetCDF/lib:$LD_LIBRARY_PATH export LD_LIBRARY_PATH=/octfs/work/(your group name)/(your user ID)/HYPRE/lib:$LD_LIBRARY_PATH cd $PBS_O_WORKDIR mpirun ${NQSII_MPIOPTS} -x LD_LIBRARY_PATH=${LD_LIBRARY_PATH} -np 96 ./flash4 |
how to run batch request
% qsub flash.sh