Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] memory consumption on rank 0 and btl_openib_receive_queues use
From: Eloi Gaudry (eg_at_[hidden])
Date: 2011-01-24 07:34:16


hi folks,

sorry to insist, i'll be glad if someone could point out the relevant
piece of info I missed.

these huge memory consumption cannot be explained easily on the user
side, especially since it's only seen on the master process node...

thanks,
éloi

On 01/03/2011 09:57 AM, Eloi Gaudry wrote:
> hi,
>
> i'd like to know if someone had a chance to check at the issue I
> reported.
>
> thanks and happy new year !
> éloi
>
> On 12/21/2010 10:58 AM, Eloi Gaudry wrote:
>> hi,
>>
>> when launching a parallel computation on 128 nodes using openib and
>> the "-mca btl_openib_receive_queues P,65536,256,192,128" option, i
>> observe a rather large memory consumption (2GB: 65336*256*128) on the
>> process with rank 0 (only).
>>
>> this memory seems to be initialized and in use as the resident memory
>> indicator matches the virtual one. at the time of observation, a call
>> to MPI_Init has been done, but no communication (user side) has been
>> done.
>>
>> i'd like to know why the other processes doesn't behave the same:
>> - other processes located on the same nodes don't use that amount of
>> memory
>> - all others processes (i.e. located on any other nodes) neither
>>
>> i'm using OpenMPI-1.4.2, built with gcc-4.3.4 and
>> '--enable-cxx-exceptions --with-pic --with-threads=posix' options.
>> the cluster is based on eight-core nodes using mellanox hca.
>>
>> thanks for your help,
>> éloi