Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] memory consumption on rank 0 and btl_openib_receive_queues use
From: Eloi Gaudry (eg_at_[hidden])
Date: 2010-12-21 04:58:03


when launching a parallel computation on 128 nodes using openib and the
"-mca btl_openib_receive_queues P,65536,256,192,128" option, i observe a
rather large memory consumption (2GB: 65336*256*128) on the process with
rank 0 (only).

this memory seems to be initialized and in use as the resident memory
indicator matches the virtual one. at the time of observation, a call to
MPI_Init has been done, but no communication (user side) has been done.

i'd like to know why the other processes doesn't behave the same:
- other processes located on the same nodes don't use that amount of memory
- all others processes (i.e. located on any other nodes) neither

i'm using OpenMPI-1.4.2, built with gcc-4.3.4 and
'--enable-cxx-exceptions --with-pic --with-threads=posix' options. the
cluster is based on eight-core nodes using mellanox hca.

thanks for your help,

Eloi Gaudry
Senior Product Development Engineer
Free Field Technologies
Company Website:
Direct Phone Number: +32 10 495 147