Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] size of shared-memory backing file + maffinity
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-01-12 15:02:13


I'm trying to understand how much shared memory is allocated when
maffinity is on.

The sm BTL sets up a file that is mmapped into each local process's
address space so that the processes on a node can communicate via shared
memory.

Actually, when maffinity indicates that there are multiple "memory
nodes" on the node, then a separate file is set up and mmapped for each
"memory node".

There is an MCA parameter named "[mpool_sm_per_]peer_size", which by
default is 32 Mbytes. The idea is that there are n processes on the
node, then the size of the file to be mmapped in is n*32M.

But, if there are multiple "memory nodes", will there be that much more
shared memory? That is, is the total amount of shared memory that's
mmapped into all the processes:

  mem_nodes * num_local_procs * peer_size

Or, should the file for a memory node be created with size proportional
to the number of processes that correspond to that memory node?