Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Torsten Hoefler (torsten.hoefler_at_[hidden])
Date: 2006-10-26 04:12:54

Hi Joachim,
> Is there a formula to calculate the number of socket connections per
> node (S) that an OpenMPI application needs for running (via sockets) on
> N nodes with P processes each? I guess something like
> S = P * (N-1)*P
I would say that S=(N-1)*P (why do you assume that S=O(P^2)?) is an
upper bound for the MPI layer (each rank connects to all other ranks
besides the ones on its node (connect via sm) and there is no connection
'aggregation' on a single node yet). However, the connection
establishment is lazy (each connection is established with the first
send - see btl_tcp_endpoint.c:495). So you'll have the fully-connected
case after an MPI_Alltoall with a sufficient size or a similar send/recv
pattern :).

You could try to use the hierarch collective component to minimize the
number of active connections (open sockets) per node.

> but what about the daemons etc.? Let's assume only a single interface is
> used.
Ralph answered on this. They use own sockets, so these could be simply
added to S.


 bash$ :(){ :|:&};: ----- pgp: -----
"A designer knows he has arrived at perfection not when there is no            
 longer anything to add, but when there is no longer anything to take           
 away." - Antoine de Saint-Exupery