On Wed, 4 Mar 2009, George Bosilca wrote:
>> I'm churning a lot and not making much progress, but I'll try chewing on
>> that idea (unless someone points out it's utterly ridiculous). I'll look
>> into having PML ignore sendi functions altogether and just make the
>> "send-immediate" path work fast with normal send functions. If that works,
>> then we can get rid of sendi functions and hopefully have a solution that
>> makes sense for everyone.
> This is utterly ridiculous (I hope you really expect someone to say it). As I
> said before, SM is only one of the networks supported by Open MPI.
> Independent on how much I would like to have better shared memory
> performance, I will not agree with any PML modifications that are SM
> oriented. We did that in the past with other BTLs and it turned out to be a
> bad idea, so I'm clearly not in favor of doing the same mistake twice.
> Regarding the sendi there are at least 3 networks that can take advantage of
> it: Portals, MX and Sicortex. Some of them do this right now, some others in
> the near future. Moreover, for these particular networks there is no way to
> avoid extra overhead without this feature (for very obscure reasons such as
> non contiguous pieces of memory only known by the BTL that can decrease the
> number of network operations).
How about removing the MCA parameter from my earlier proposal and just
having r2 filter out the sendi calls if there are multiple BTLs with
heterogeneous BTLs (ie, some with sendi and some without) to the same
peer. That way, the early sendi will be bypassed in that case. But for
the cases of BTLs that support sendi in common usage scenarios
(homogeneous nics), we'll get the optimization? Does that offend you