Yves Caniou wrote:
Wow. I just tried "mpirun -h" and, yes, it claims that
"--bind-to-core" is the default. I believe this is wrong... or at
least "misleading." :^) You should specify --bind-to-core
explicitly. It is the successor to paffinity. Do add
--report-bindings to check what you're getting.
Le Wednesday 28 July 2010 15:05:28, vous avez écrit :
I am confused. I thought all you wanted to do is report out the binding of
the process - yes? Are you trying to set the affinity bindings yourself?
If the latter, then your script doesn't do anything that mpirun wouldn't
do, and doesn't do it as well. You would be far better off just adding
--bind-to-core to the mpirun cmd line.
"mpirun -h" says that it is the default, so there is not even something to do?
I don't even have to add "--mca mpi_paffinity_alone 1" ?
Another option is to use OMPI_COMM_WORLD_LOCAL_RANK. This environment
variable directly gives you the value you're looking for, regardless of
how process ranks are mapped to the nodes.
On Jul 28, 2010, at 6:37 AM, Yves Caniou wrote:
Le Wednesday 28 July 2010 11:34:13 Ralph Castain, vous avez écrit :
On Jul 27, 2010, at 11:18 PM, Yves Caniou wrote:
Le Wednesday 28 July 2010 06:03:21 Nysal Jan, vous avez écrit :
OMPI_COMM_WORLD_RANK can be used to get the MPI rank.
Are processes affected to nodes sequentially, so that I can get the
NODE number from $OMPI_COMM_WORLD_RANK modulo the number of proc per
By default, yes. However, you can select alternative mapping methods.
It reports to stderr, so the $OMPI_COMM_WORLD_RANK modulo the number of
proc per nodes seems more appropriate for what I need, right?
So is the following valid to put memory affinity?
MYVAL=$(expr $MYRANK / 4)
NODE=$(expr $MYVAL % 4)
numactl --cpunodebind=$NODE --membind=$NODE $@
mpiexec ./script.sh -n 128 myappli myparam
I don't know what's "better," but here are some potential issues:
Which is better: using this option, or the cmd line with numactl (if it
works)? What is the difference?
*) Different MPI implementations use different mechanisms for
specifying binding. So, if you want your solution to be "portable"...
well, if you want that, you're out of luck. But, perhaps some
mechanisms (command-line arguments, run-time scripts, etc.) might seem
easier for you to adapt than others.
*) Some mechanisms bind processes at process launch time and some at
MPI_Init time. The former might be better. Otherwise, a process might
place some NUMA memory in a location before MPI_Init and then be moved
away from that memory when MPI_Init is encountered. I believe both the
numactl and OMPI --bind-to-core mechanisms have this characteristic.
(OMPI's older paffinity might not, but I don't remember for sure.)
Mostly, if you're going to use just OMPI, the --bind-to-core
command-line argument might be the simplest.