Stampede3 Deployment

This page documents known working practices for running Floability on Stampede3.

First, follow the basic flow in Run your first backpack.

Recommended Start

Run from a Stampede3 login node:

floability run --backpack <backpack-root> --batch-type slurm

Notebook/Jupyter runs on the login node. Distributed tasks are submitted through vine_factory to Slurm workers.

Stampede3 uses Slurm, so use --batch-type slurm.

In our runs, the following manager port range worked:

floability run --backpack <backpack-root> --batch-type slurm --manager-ports 35000,40000

You may need to provide explicit Slurm options. Start with:

floability run --backpack <backpack-root> --batch-type slurm \
    --batch-options "-p spr -t 02:00:00"

Try running without this first, or use other batch options required by your allocation/policy.

Stampede3 run documentation: https://docs.tacc.utexas.edu/hpc/stampede3/#running

For TaskVine with Slurm, make sure data is on a shared filesystem visible to both login and worker nodes.

We recommend using /work for data/cache storage.

Example:

floability run --backpack <backpack-root> --batch-type slurm \
    --data-cache-dir /work/<project-or-user>/floability-cache-base-dir