Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Joachim Worringen (joachim_at_[hidden])
Date: 2006-10-26 05:42:30

Torsten Hoefler wrote:
> Hi Joachim,
>> Is there a formula to calculate the number of socket connections per
>> node (S) that an OpenMPI application needs for running (via sockets) on
>> N nodes with P processes each? I guess something like
>> S = P * (N-1)*P
> I would say that S=(N-1)*P (why do you assume that S=O(P^2)?) is an

For N nodes running P processes each, each single process sees (N-1)*P
remote processes. I'd say this results in (N-1)*P^2 connections *per node*.

> upper bound for the MPI layer (each rank connects to all other ranks
> besides the ones on its node (connect via sm) and there is no connection
> 'aggregation' on a single node yet). However, the connection
> establishment is lazy (each connection is established with the first
> send - see btl_tcp_endpoint.c:495). So you'll have the fully-connected
> case after an MPI_Alltoall with a sufficient size or a similar send/recv
> pattern :).

O.k. But what happens if a connection can not be established due to
resource limits? Are existing connections closed? O

> You could try to use the hierarch collective component to minimize the
> number of active connections (open sockets) per node.

Yes, this also can help to increase throughput for small data size.

  regards, Joachim

Joachim Worringen, Software Architect, Dolphin Interconnect Solutions
phone ++49/(0)228/324 08 17 -