Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Debugging memory use of Open MPI
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-04-16 17:11:40


Shaun Jackman wrote:

> Eugene Loh wrote:
>
>> What's the rest? I said the shared-memory area is much smaller, but
>> I was confused about which OMPI release I was using. So, the
>> shared-memory area was 128 Mbyte and it was getting mapped in once
>> for each process, and so it was counted doubly.
>
> If there are eight processes running on one host, does each process
> allocate one 128 Mbyte shared memory buffer and map in the other seven
> 128 Mbyte buffers allocated by the other processes?

No. The total size for one, single shared file is computed and the
lowest rank on the node creates the file and mmaps it in. Then, the
other processes mmap the same file in.

The code is set up to have different "memory pools". E.g., look at
https://svn.open-mpi.org/source/xref/ompi-trunk/ompi/mca/btl/sm/btl_sm.c#sm_btl_first_time_init
. So, conceivably you could have different buffers on the same node.
But in practice it's just one buffer and, in any case, they're always
created by the lowest rank. E.g.,
https://svn.open-mpi.org/source/xref/ompi-trunk/ompi/mca/common/sm/common_sm_mmap.c#143
.