EuroHPC 2024 Modules
====================

We present here the modules used to compile and run SIESTA under EuroHPC
machines for the **2024** benchmarks. Be aware that these can be slightly different
as time goes by, but it can provide a good starting point.

**For the GPU partitions, you will also need a GPU-enabled version of ELPA.** This
can be already provided by the HPC center, or you can compile it yourself. You
can find ELPA here: https://gitlab.mpcdf.mpg.de/elpa/elpa.git


Intel-based systems
-------------------
Note that IntelMPI showed a few numerical issues and memory leaks, so we
recommend using OpenMPI instead.

Leonardo DCGP
^^^^^^^^^^^^^

.. code-block:: bash

   module load intel-oneapi-compilers/2023.2.1 intel-oneapi-mkl/2023.2.0 intel-oneapi-mpi/2021.10.0
   module load netcdf-fortran/4.6.1--oneapi--2023.2.0 spack

   spack load /g24wncv

Leonardo Booster (GPU)
^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

   module load gcc/12.2.0 fftw/3.3.10--openmpi--4.1.6--gcc--12.2.0
   module load openblas/0.3.24--gcc--12.2.0 netlib-scalapack/2.2.0--openmpi--4.1.6--gcc--12.2.0
   module load netcdf-fortran/4.6.1--openmpi--4.1.6--gcc--12.2.0 cuda/12.1
   module load cmake ninja

Marenostrum 5 GPP
^^^^^^^^^^^^^^^^^

.. code-block:: bash

    module load intel/2023.2.0 mkl/2023.2.0 ucx/1.15.0 bsc/1.0 ninja/1.11.1 cmake/3.25.1 openmpi/4.1.5
    module load pnetcdf/1.12.3-openmpi hdf5/1.14.1-2-openmpi netcdf/c-4.9.2_fortran-4.6.1_cxx4-4.3.1_hdf5-1.14.1-2_pnetcdf-1.12.3-openmpi
    module load fftw/3.3.10-openmpi

Marenostrum 5 ACC (GPU)
^^^^^^^^^^^^^^^^^^^^^^^

.. code-block:: bash

    module load EB/apps libtool Autoconf Python
    module load ScaLAPACK/2.2.0-gompi-2023b-fb FFTW/3.3.10-GCC-13.2.0 CUDA/12.5.0
    module load netCDF-Fortran GCC/12.3.0
    module load CMake/3.26.3-GCCcore-12.3.0 Ninja/1.11.1-GCCcore-12.3.0


Cray-based systems
------------------

LUMI-C
^^^^^^

.. code-block:: bash

   module load LUMI/24.09 partition/C libreadline PrgEnv-cray
   module load cray-hdf5-parallel cray-parallel-netcdf cray-netcdf-hdf5parallel cray-fftw

LUMI-G (GPU)
^^^^^^^^^^^^

.. code-block:: bash

   module load LUMI/24.09 partition/G PrgEnv-cray
   module load cray-hdf5-parallel cray-parallel-netcdf cray-netcdf-hdf5parallel
   module load cray-fftw rocm buildtools


GNU-based systems
-----------------


Deucalion
^^^^^^^^^

.. code-block:: bash

    module load ScaLAPACK/2.2.0-gompi-2023a-fb netCDF-Fortran/4.6.1-gompi-2023a FFTW/3.3.10-GCC-12.3.0
    module load CMake Ninja libreadline/8.2-GCCcore-12.3.0 Bison

Deucalion (GPU)
^^^^^^^^^^^^^^^

.. code-block:: bash

    module load ScaLAPACK/2.2.0-gompi-2023a-fb netCDF-Fortran/4.6.1-gompi-2023a FFTW/3.3.10-GCC-12.3.0
    module load CMake Ninja libreadline/8.2-GCCcore-12.3.0
    module load Autoconf/2.71 libtool/2.4.7
    module load CUDA/12.4.0

Discoverer
^^^^^^^^^^

.. code-block:: bash

    module load gcc/latest openblas/0/latest-gcc lapack/latest-gcc scalapack/2/latest-gcc-openmpi ucx/1/latest cmake/latest
    module load libfabric/1/latest libaec/1/latest-gcc openmpi/4/gcc/latest zstd/1/latest-gcc zlib/1/latest-gcc fftw/3/latest-gcc-openmpi
    module load hdf5/1/1.14/latest-gcc-openmpi netcdf/c/4.9/latest-gcc-openmpi pnetcdf/1/latest-gcc-openmpi netcdf/fortran/4.6/latest-gcc-openmpi
    export OMP_NUM_THREADS=1 # Required for openblas detection.

Karolina
^^^^^^^^

.. code-block:: bash

    module load GCC CMake OpenBLAS LAPACK ScaLAPACK/2.2.0-gompi-2023b-fb libreadline Bison/3.8.2-GCCcore-13.2.0
    module load PnetCDF/1.12.3-gompi-2022a netCDF-Fortran/4.6.1-gompi-2023b FFTW.MPI/3.3.10-gompi-2023b

Karolina (GPU)
^^^^^^^^^^^^^^

.. code-block:: bash

    module load GCC/12.3.0 CMake/3.26.3-GCCcore-12.3.0 OpenBLAS LAPACK ScaLAPACK/2.2.0-gompi-2023b-fb
    module load libreadline CUDA/12.3.0 Autoconf libtool
    module load PnetCDF/1.12.3-gompi-2022a netCDF-Fortran/4.6.1-gompi-2023b FFTW.MPI/3.3.10-gompi-2023b

Meluxina
^^^^^^^^

.. code-block:: bash

    module load ScaLAPACK/2.2.0-gompi-2023a-fb netCDF-Fortran/4.6.1-gompi-2023a FFTW/3.3.10-GCC-12.3.0
    module load CMake Ninja libreadline/8.2-GCCcore-12.3.0 Bison

Meluxina (GPU)
^^^^^^^^^^^^^^

.. code-block:: bash

    module load ScaLAPACK/2.2.0-gompi-2023a-fb netCDF-Fortran/4.6.1-gompi-2023a FFTW/3.3.10-GCC-12.3.0
    module load CMake Ninja libreadline/8.2-GCCcore-12.3.0
    module load Autoconf/2.71 libtool/2.4.7 Automake foss Perl
    module load CUDA/12.2.0

Vega
^^^^
.. code-block:: bash

    module load netCDF-Fortran PnetCDF/1.12.3-gompi-2021b libreadline
    module load OpenBLAS ScaLAPACK FFTW/3.3.10-GCC-12.3.0
    module load CMake Ninja Bison/3.8.2-GCCcore-12.3.0

Vega (GPU)
^^^^^^^^^^

.. code-block:: bash

    module load netCDF-Fortran PnetCDF/1.12.3-gompi-2021b libreadline
    module load OpenBLAS ScaLAPACK FFTW/3.3.10-GCC-12.3.0
    module load Autoconf libtool
    module load GCC CUDA
    module load CMake Ninja