Overview

Miniforge provides the Conda and Mamba package managers, with the default channel being conda-forge.
(Mamba is a reimplementation of the Conda package manager in C++ that uses a state-of-the-art library “libsolv” for much faster dependency solving.)

We have transitioned from Anaconda to Miniforge on Oct 15, 2024. See here for details.

Available Versions

The current installation of Miniforge
incorporates the most popular packages. To find the available versions and learn how to load them, run:

module spider miniforge

The output of the command shows the available Miniforge
module versions.

For detailed information about a particular Miniforge
module, including how to load the module, run the module spider command with the module’s full version label. For example:

module spider miniforge/24.3.0-py3.11

Installing packages

Packages could be installed via the pip, conda, or mamba package managers

Using pip

Open the bash terminal, and type:

1. module load miniforge
2. pip search package_name (search for a package by name)
3. pip install --user package_name (install a package)
4. pip update package_name --upgrade (upgrade the package to the latest stable version)
5. pip list (list all installed packages)

You are using pip version x.x.x, however version y.y.y is available.
You should consider upgrading via the 'pip install --upgrade pip' command.

However, if you must upgrade pip, please do so in a virtual environment, such as conda.

Using conda

You can specify which version of Python you want to run using conda. This can be done on a project-by-project basis, and is part of what is called a “Virtual Environment”. A Virtual Environment is simply your isolated copy of Python in which you maintain your own version of files and directories. It enables you to keep other projects unaffected. With projects that have similar dependencies, you can freely install different versions of the same package without worry on two different Virtual Environments. In order to jump between two VE’s, you simply activate or deactivate your environment. Follow the steps below:

1. Set up your Virtual Environment:

   conda create -n your_env_name_goes_here (default Python version: use conda info to find out)

   OR

   conda create -n your_env_name_goes_here python=version_goes_here (This command will automatically upgrade pip to the latest version in the environment. To find specific Python versions, use conda search "^python$".)

2. If it asks you for y/n, hit y to proceed. It will start the installation

3. Activate your newly created environment source activate your_env_name_goes_here

4. Install a package in your activated environment

   conda install -n your_env_name_goes_here your_package_name_goes_here

   OR

   conda install -n your_env_name_goes_here \ your_package_name_goes_here=version_goes_here

   OR (even better)

   In your home directory or Conda installation directory, create a file called .condarc (if not already there) Inside the file write the following:

   create_default_packages
   	- your_package_name_goes_here
   	- your_package_name_goes_here
   	- your_package_name_goes_here
   	...
   

   Now everytime you create a new environment, all those packages listed in .condarc will be installed.

5. To end the current environment session:
   conda deactivate

6. Remove an environment:
   conda remove -n your_env_name_goes_here -all

7. To create a JupyterLab tile for your conda environment:

   Install ipykernel inside your activated environment:
   conda install -c conda-forge ipykernel

   Then, create a new kernel:
   python -m ipykernel install --user --name=MyEnvName

   Your new kernel will show up as a tile when you select File-> New Launcher in JupyterLab.

To see all available environments, run conda env list.

Python and MPI

The most widely used MPI library for Python is mpi4py. We recommend creating an environment for it. When installing on the cluster, please do not use conda install since this will install prebuilt binaries, including a version of MPICH that does not communicate correctly with our Slurm resource manager. The best practice is to install from the conda-forge channel using their external versions of an MPI library. These are simply bindings to an MPI package provided by the underlying system. For our system we will use OpenMPI.

First load a compiler and a corresponding version of OpenMPI.

module load gcc openmpi

For the above example, with the versions of gcc available on our system, this is gcc 11.4.0. Now check the version of OpenMPI.

module list

In this example it is OpenMPI 4.1.4. Be aware that specific version numbers will change with time.

Now view the available external versions of OpenMPI from conda-forge

conda search -f openmpi -c conda-forge

Install the bindings

conda install -c conda-forge "openmpi=4.1.4=external_*"

Now you can install mpi4py

conda install -c conda-forge mpi4py

Example Slurm script

Non-MPI

#!/bin/bash
#SBATCH -A mygroup
#SBATCH -p standard
#SBATCH -N 1
#SBATCH -c 1
#SBATCH -t 01:00:00
#SBATCH -o myprog.out
#SBATCH --export=NONE

module purge
module load miniforge
# optional: uncomment next line to use your custom Conda environment; replace 'custom_env' with actual env name
# source activate custom_env

python myscript.py

MPI

#!/bin/bash
#SBATCH -A mygroup
#SBATCH -p standard
#SBATCH -N 1
#SBATCH --ntasks-per-node=10
#SBATCH -t 01:00:00
#SBATCH -o myprog.out
#SBATCH --export=NONE

module purge
module load gcc openmpi

# Replace 'custom_env' with the actual env name.
module load miniforge
source activate custom_env

srun --export=ALL python myscript.py

More Information

Please visit the official website.