Running

Charm4py includes a launcher called charmrun to run parallel applications on desktops and small clusters. Supercomputers and some clusters provide their own application launchers (these can also be used to launch Charm4py applications).

charmrun

After installing Charm4py as explained in the previous section, you can launch applications like this:

$ python3 -m charmrun.start +p4 myprogram.py

The option +pN specifies how many processes to run the application with.

Alternatively, if charmrun is in your PATH (this depends on where Charm4py was installed and your system configuration):

$ charmrun +p4 myprogram.py

You can launch an interactive shell using the ++interactive option, for example:

$ python3 -m charmrun.start +p4 ++interactive

Charm4py programs accept the same command-line parameters as Charm++.

Running on multiple hosts

charmrun can run an application on multiple hosts (e.g. a network of workstations) by passing it a file containing the list of nodes (nodelist file). Hosts can be specified by IP address or host name. For example, this is a simple nodelist file specifying four hosts:

group mynodes
    host 192.168.0.10
    host 192.168.0.133
    host myhost
    host myhost2

The application can be launched like this:

$ python3 -m charmrun.start +pN myprogram.py ++nodelist mynodelist.txt

With this method, charmrun uses ssh to log into remote machines and spawn processes.

charmrun can also use the cluster’s mpiexec job launcher instead of the built-in ssh method.

See the charmrun manual for more information and alternative ways to work with nodelist files.

Important

If you need to set environment variables on the remote hosts, you can let charmrun do it by adding your export commands to a file called ~/.charmrunrc.

Using charmrun from a Python program

You can launch a Charm4py application from inside a Python application, and wait for it to complete, in this manner:

from charmrun import start

start.start(['+p4', 'myprogram.py'])  # launch parallel application and wait for completion

Note that you can also use Python’s subprocess library and launch the same command as you would via the command line.

Troubleshooting

Issue
Program hangs with no output when launching with charmrun.
Solution
This typically occurs when launching the program on multiple hosts, and an error ocurring before starting charm (e.g. syntax error). To diagnose, launch the program on a single host.

mpirun (or equivalent)

If you have built Charm4py to use MPI, you can launch Charm4py applications using mpirun, mpiexec or other valid method on your system that supports launching MPI applications. For example:

$ mpirun -np 4 /usr/bin/python3 myprogram.py

See Install for instructions on building Charm4py for MPI.

You can launch an interactive shell with mpirun like this (or similar command):

$ mpirun -np 4 /usr/bin/python3 -m charm4py.interactive

Note that the console has limited functionality in terms of tab completion, etc. due to stdin and stdout limitations when using mpirun.

Using system job launchers

Charm4py applications can also be launched using system job launchers (e.g. aprun, ibrun, SLURM). The exact details of how to do so depend on the system, and typically Charm++ has to be built with a specialized network layer like MPI, GNI or OFI (see Charm++ manual build).

In all cases, the mechanism consists in launching one or multiple Python processes on each node, and passing the main application file to Python. Here is a simple script for SLURM on a Cray-based system:

#!/bin/bash -l
#SBATCH -N 8         # number of nodes
#SBATCH -t 00:30:00
#SBATCH -C knl

module load craype-hugepages8M
module load python/3.6-anaconda-4.4

export PYTHONPATH=/path/to/charm4py
PYTHON_EXEC=`which python3`

srun -n 512 -c 1 $PYTHON_EXEC myprogram.py app_param1 app_param2 ...