Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] [OMPI users] huge VmRSS on rank 0 after MPI_Init when using "btl_openib_receive_queues" option
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2011-07-09 07:13:04


Sorry for the delayed reply.

I'm guessing that there's some kind of rooted collective operation occurring during MPI_INIT. Try doing a 1 byte MPI_GATHER to another rank in MCW (e.g., to rank 5) and see if the VmSS goes to the same size as MCW rank 0.

On Jul 8, 2011, at 5:17 AM, Eloi Gaudry wrote:

> what i cannot understand is the reason why this extra memory would be initialized on proc 0 only.
> as far as i know, this doesn't make sense.
> éloi
>
>> On 22/04/2011 08:52, Eloi Gaudry wrote:
>>> it varies with the receive_queues specification *and* with the number of mpi processes: memory_consumed = nb_mpi_process * nb_buffers * (buffer_size + low_buffer_count_watermark + credit_window_size )
>>>
>>> éloi
>>>
>>>
>>> On 04/22/2011 12:26 AM, Jeff Squyres wrote:
>>>> Does it vary exactly according to your receive_queues specification?
>>>>
>>>> On Apr 19, 2011, at 9:03 AM, Eloi Gaudry wrote:
>>>>
>>>>> hello,
>>>>>
>>>>> i would like to get your input on this:
>>>>> when launching a parallel computation on 128 nodes using openib and the "-mca btl_openib_receive_queues P,65536,256,192,128" option, i observe a rather large resident memory consumption (2GB: 65336*256*128) on the process with rank 0 (and only this process) just after a call to MPI_Init.
>>>>>
>>>>> i'd like to know why the other processes doesn't behave the same:
>>>>> - other processes located on the same nodes don't use that amount of memory
>>>>> - all others processes (i.e. located on any other nodes) neither
>>>>>
>>>>> i'm using OpenMPI-1.4.2, built with gcc-4.3.4 and '--enable-cxx-exceptions --with-pic --with-threads=posix' options.
>>>>>
>>>>> thanks for your help,
>>>>> éloi
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/