Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Timothy S. Woodall (twoodall_at_[hidden])
Date: 2006-02-08 20:17:41


>
> On Feb 8, 2006, at 7:06 PM, Jean-Christophe Hugly wrote:
>
>> But should I understand from all this that the "direct" mode will
>> never
>> actually work ? It seems that if you need at least two transports,
>> then
>> none of them can be the hardwired unique one, right ? Unless there's a
>> built-in switch between a built-in self and the built-in other
>> transport.
>
> Some of the transport layers are able to handle the messages to
> "self". However, as we decide to let "self" do this type of work no
> effort was spending on making sure they do it. Our first concern was/
> is/will be about performance, and "self" really do a great job. So
> the quick answer to your question is no, there is no way to limit the
> number of transports to one.
>
> Long ago, before the latest version of the BTL (byte transport
> layer), we had something called the PTL. They were used with another
> set of PML (protocol management layer). I wrote a specific PML
> (called uniq) that was able to handle only one device (plus "self").
> The latency went down by a little bit (around 0.3 micro-seconds).
> Anyway, the old openib PTL never reached a stable state so this will
> not help you :(. As we plan to drop all support for the old
> generation of PML/PTL, I don't think is a wise idea to spend time on
> the openib PTL to make it working with uniq ...
>
> Thanks,
> george.
>

With the change to ob1/BTLs, there was also a refactoring of data
structures that reduced the overall latency through the stack. As
Galen indicated, if you do a direct comparison w/ send/recv semantics,
I think you will find the overall latency through the stack is lower
than other implementations (on the order of 0.5 us).