Anaconda is a distribution for Python that was created for data science. This page explains how to create your own python environment (virtual environment) with Anaconda and install the necessary software. Anaconda is distributed at the following sites. Please refer here for detailed usage instructions. Anaconda.com * Due to Anaconda's terms of use, industrial Please note that Anaconda is not available for industrial use.
How to use
List of virtual environments
- Displays the list of virtual environments that have already been built.
conda info -e
# conda environments:
#
base * /system/apps/rhel8/cpu/Anaconda3/2020.11
Creating a new virtual environment
- The following is the procedure for creating a new virtual environment called test-env in the user area.
First, specify the conda_env directory under the work area as the destination of the virtual environment. (If you do not specify any directory, the virtual environment will be created in the .conda directory under the home area, which may overflow the capacity.)
conda config --add envs_dirs /sqfs/work/(group name)/(UserID)/conda_env
conda config --add pkgs_dirs /sqfs/work/(group name)/(UserID)/conda_pkg
Generate the virtual environment test-env.
conda create --name test-env python=3.8
If you view the list of virtual environments, you will see that test-env has been generated.
conda info -e
# conda environments:
#
test-env /sqfs/work/group-name/user-name/conda_env/test-env
base * /system/apps/rhel8/cpu/Anaconda3/2020.11
Enabling the virtual environment
- Enable the virtual environment test-env.
conda activate test-env
Installing software in a virtual environment
- We install the Python library numpy in the virtual environment test-env. Since communication to the outside world is not possible from a compute node, please make sure to do this on the login node.
conda install numpy
Disable the virtual environment
- Disable the virtual environment test-env.
conda deactivate
How to use in SQUID calculation node
Please make the following job script when you enable the virtual environment "test-env" and execute test.py.
|
1 2 3 4 5 6 7 |
#! /bin/bash #PBS -q SQUID #PBS --group=(group name) #PBS -l elapstim_req=1:00:00 cd $PBS_O_WORKDIR source activate test-env python test.py |
If you want to use OpenMPI (mpi4py) deployed in the Anaconda virtual environment on multiple nodes, create a job script and a module file as follows. Use the module file to deploy the MPI configuration on each node. The following is an example of a script that runs sample.py on four nodes using my-conda-env, a virtual environment that deploys openMPI and mpi4py in Anaconda.
the following is the sample for module file:
|
1 2 3 4 5 6 7 8 |
#%Module 1.0 proc ModulesHelp { } { puts stderr "OMP-sample\n" } prepend-path PATH /sqfs/work/(group name)/(UserID)/conda_env/my-conda-env/bin prepend-path LD_LIBRARY_PATH /sqfs/work/(group name)/(UserID)/conda_env/my-conda-env/lib/ setenv MPI_ROOT /sqfs/work/(group name)/(UserID)/conda_env/my-conda-env |
the following is the sample for job script. "omp-modules" is a name of module file.
|
1 2 3 4 5 6 7 8 9 10 |
#!/bin/bash #PBS -q SQUID #PBS --group=(group name) #PBS -l elapstim_req=1:00:00 #PBS -b 4 #PBS -T openmpi #PBS -v NQSV_MPI_MODULE=/sqfs/work/(group name)/(UserID)/omp-modules cd $PBS_O_WORKDIR source activate my-conda-env mpirun ${NQSV_MPIOPTS} -np 4 -npernode 1 -mca btl_tcp_if_include ib0 python ./sample.py |
