GPAW/en

General

GPAW is a density-functional theory (DFT) Python code based on the projector-augmented wave (PAW) method and the atomic simulation environment (ASE).

Creating a GPAW virtual environment

We provide precompiled Python wheels for GPAW that can be installed into a virtual python environment.

1. Check which versions of gpaw are available:

   avail_wheels gpaw
   

   Example output:

   name	version	python	arch
   gpaw	22.8.0	cp39	avx2
   gpaw	22.8.0	cp38	avx2
   gpaw	22.8.0	cp310	avx2

2. Load a Python module (e.g., python/3.10):

   module load python/3.10
   

3. Create a new virtualenv:

   virtualenv --no-download venv_gpaw
   

   Example output:

   created virtual environment CPython3.10.2.final.0-64 in 514ms
   [...]
   

4. Activate the virtualenv (venv):

   source venv_gpaw/bin/activate
   

5. Install gpaw into venv:

   pip install --no-index gpaw
   

   Example output:

   [...]
   Successfully installed ... gpaw-22.8.0+computecanada ...
   

6. Download the data and install it into the SCRATCH filesystem:

   gpaw install-data $SCRATCH
   

   Example output:

   Available setups and pseudopotentials
     [*] https://wiki.fysik.dtu.dk/gpaw-files/gpaw-setups-0.9.20000.tar.gz
   [...]
   Setups installed into /scratch/name/gpaw-setups-0.9.20000.
   Register this setup path in /home/name/.gpaw/rc.py? [y/n] n
   As you wish.
   [...]
   Installation complete.
   

7. Now set GPAW_SETUP_PATH to point to the data directory:

   export GPAW_SETUP_PATH=$SCRATCH/gpaw-setups-0.9.20000
   

8. We can run the tests, which are very fast:

   gpaw test
   

   Example output:

   Key	Value
   python-3.10.2	/home/name/venv_gpaw/bin/python
   gpaw-22.8.0	/home/name/venv_gpaw/lib/python3.10/site-packages/gpaw/
   ase-3.22.1	/home/name/venv_gpaw/lib/python3.10/site-packages/ase/
   numpy-1.23.0	/home/name/venv_gpaw/lib/python3.10/site-packages/numpy/
   scipy-1.9.3	/home/name/venv_gpaw/lib/python3.10/site-packages/scipy/
   libxc-5.2.3	yes
   _gpaw	/home/name/venv_gpaw/lib/python3.10/site-packages/_gpaw.cpython-310-x86_64-linux-gnu.so
   MPI enabled	yes
   OpenMP enabled	yes
   scalapack	yes
   Elpa	no
   FFTW	yes
   libvdwxc	no
   PAW-datasets (1)	/scratch/name/gpaw-setups-0.9.20000

   Doing a test calculation (cores: 1): ... Done
   Test parallel calculation with "gpaw -P 4 test".
   

   gpaw -P 4 test
   

   Example output:

   Key	Value
   python-3.10.2	/home/name/venv_gpaw/bin/python
   gpaw-22.8.0	/home/name/venv_gpaw/lib/python3.10/site-packages/gpaw/
   ase-3.22.1	/home/name/venv_gpaw/lib/python3.10/site-packages/ase/
   numpy-1.23.0	/home/name/venv_gpaw/lib/python3.10/site-packages/numpy/
   scipy-1.9.3	/home/name/venv_gpaw/lib/python3.10/site-packages/scipy/
   libxc-5.2.3	yes
   _gpaw	/home/name/venv_gpaw/lib/python3.10/site-packages/_gpaw.cpython-310-x86_64-linux-gnu.so
   MPI enabled	yes
   OpenMP enabled	yes
   scalapack	yes
   Elpa	no
   FFTW	yes
   libvdwxc	no
   PAW-datasets (1)	/scratch/name/gpaw-setups-0.9.20000

   Doing a test calculation (cores: 4): ... Done
   

Results of the last test can be found in the file test.txt that will be created in the current directory.

Example Jobscript

A jobscript may look something like this for hybrid (OpenMP and MPI) parallelization. This assumes that the virtualenv is in your $HOME directory and the PAW-datasets in $SCRATCH as shown above.

job_gpaw.sh

#!/bin/bash
#SBATCH --ntasks=8
#SBATCH --cpus-per-task=4
#SBATCH --mem-per-cpu=4000M
#SBATCH --time=0-01:00
module load gcc/9.3.0 openmpi/4.0.3
source ~/venv_gpaw/bin/activate

export OMP_NUM_THREADS="${SLURM_CPUS_PER_TASK:-1}"
export GPAW_SETUP_PATH=/scratch/$USER/gpaw-setups-0.9.20000

srun --cpus-per-task=$OMP_NUM_THREADS gpaw python my_gpaw_script.py

This would use a single node with 8 MPI-ranks (ntasks) and 4 OpenMP threads per MPI rank (cpus-per-task), for a total of 32 CPUs. You probably want to adjust those numbers so that the product matches the number of cores of a whole node (i.e., 32 at Graham, 40 at Béluga and Niagara, 48 at Cedar or 64 at Narval).

Setting OMP_NUM_THREADS as shown above makes sure it is always set to the same value as cpus-per-task or 1 in case cpus-per-task is not set. Loading the modules gcc/9.3.0 and openmpi/4.0.3 ensures that the exact MPI library is used for the job, as was used for building the wheels.