Running¶
Charm4py includes a launcher called charmrun
to run parallel applications on
desktops and small clusters. Supercomputers and some clusters provide their
own application launchers (these can also be used to launch Charm4py applications).
charmrun¶
After installing Charm4py as explained in the previous section, you can launch applications like this:
$ python3 -m charmrun.start +p4 myprogram.py
The option +pN
specifies how many processes to run the application with.
Alternatively, if charmrun
is in your PATH (this depends on where Charm4py was
installed and your system configuration):
$ charmrun +p4 myprogram.py
You can launch an interactive shell using the ++interactive
option, for
example:
$ python3 -m charmrun.start +p4 ++interactive
Charm4py programs accept the same command-line parameters as Charm++.
Running on multiple hosts¶
charmrun
can run an application on multiple hosts (e.g. a network of workstations)
by passing it a file containing the list of nodes (nodelist file). Hosts can be
specified by IP address or host name. For example, this is a simple nodelist file
specifying four hosts:
group mynodes
host 192.168.0.10
host 192.168.0.133
host myhost
host myhost2
The application can be launched like this:
$ python3 -m charmrun.start +pN myprogram.py ++nodelist mynodelist.txt
With this method, charmrun uses ssh
to log into remote machines and spawn processes.
charmrun can also use the cluster’s mpiexec
job launcher instead of the built-in ssh method.
See the charmrun manual for more information and alternative ways to work with nodelist files.
Important
If you need to set environment variables on the remote hosts, you can let
charmrun do it by adding your export commands to a file called ~/.charmrunrc
.
Using charmrun from a Python program¶
You can launch a Charm4py application from inside a Python application, and wait for it to complete, in this manner:
from charmrun import start
start.start(['+p4', 'myprogram.py']) # launch parallel application and wait for completion
Note that you can also use Python’s subprocess
library and launch the same command
as you would via the command line.
Troubleshooting¶
- Issue
- Program hangs with no output when launching with
charmrun
. - Solution
- This typically occurs when launching the program on multiple hosts, and an error ocurring before starting charm (e.g. syntax error). To diagnose, launch the program on a single host.
mpirun (or equivalent)¶
If you have built Charm4py to use MPI, you can launch Charm4py applications using mpirun, mpiexec or other valid method on your system that supports launching MPI applications. For example:
$ mpirun -np 4 /usr/bin/python3 myprogram.py
See Install for instructions on building Charm4py for MPI.
You can launch an interactive shell with mpirun like this (or similar command):
$ mpirun -np 4 /usr/bin/python3 -m charm4py.interactive
Note that the console has limited functionality in terms of tab completion, etc. due to stdin and stdout limitations when using mpirun.
Using system job launchers¶
Charm4py applications can also be launched using system job launchers (e.g. aprun, ibrun, SLURM). The exact details of how to do so depend on the system, and typically Charm++ has to be built with a specialized network layer like MPI, GNI or OFI (see Charm++ manual build).
In all cases, the mechanism consists in launching one or multiple Python processes on each node, and passing the main application file to Python. Here is a simple script for SLURM on a Cray-based system:
#!/bin/bash -l
#SBATCH -N 8 # number of nodes
#SBATCH -t 00:30:00
#SBATCH -C knl
module load craype-hugepages8M
module load python/3.6-anaconda-4.4
export PYTHONPATH=/path/to/charm4py
PYTHON_EXEC=`which python3`
srun -n 512 -c 1 $PYTHON_EXEC myprogram.py app_param1 app_param2 ...