Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] memory consumption on rank 0 and btl_openib_receive_queues use
From: Eloi Gaudry (eg_at_[hidden])
Date: 2011-01-03 03:57:34


i'd like to know if someone had a chance to check at the issue I reported.

thanks and happy new year !

On 12/21/2010 10:58 AM, Eloi Gaudry wrote:
> hi,
> when launching a parallel computation on 128 nodes using openib and
> the "-mca btl_openib_receive_queues P,65536,256,192,128" option, i
> observe a rather large memory consumption (2GB: 65336*256*128) on the
> process with rank 0 (only).
> this memory seems to be initialized and in use as the resident memory
> indicator matches the virtual one. at the time of observation, a call
> to MPI_Init has been done, but no communication (user side) has been
> done.
> i'd like to know why the other processes doesn't behave the same:
> - other processes located on the same nodes don't use that amount of
> memory
> - all others processes (i.e. located on any other nodes) neither
> i'm using OpenMPI-1.4.2, built with gcc-4.3.4 and
> '--enable-cxx-exceptions --with-pic --with-threads=posix' options. the
> cluster is based on eight-core nodes using mellanox hca.
> thanks for your help,
> éloi

Eloi Gaudry
Senior Product Development Engineer
Free Field Technologies
Company Website:
Direct Phone Number: +32 10 495 147