Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] [OMPI svn-full] svn:open-mpi r25445
From: Barrett, Brian W (bwbarre_at_[hidden])
Date: 2011-11-07 17:34:45


On 11/7/11 3:27 PM, "George Bosilca" <bosilca_at_[hidden]> wrote:

>
>On Nov 7, 2011, at 10:37 , Jeff Squyres wrote:
>
>> On Nov 7, 2011, at 10:16 AM, Nathan T. Hjelm wrote:
>>
>>> Yes, and I completely agree. I was simply trying to keep it consistent
>>>in
>>> case there is something I don't know about the heterogeneous case.
>>>
>>> I increased the size of the 64 bit member because there is no uint128
>>>type.
>>
>> Ah, I see.
>>
>> I would put the other sizes back, at a minimum. There should be no
>>need to increase those.
>>
>> George -- comments? Should this be a new key fields (128, with 2
>>uint64_t's)? If this is only for large messages, is the extra 8 bytes a
>>concern?
>
>Without the vader documentation it is difficult to asses the needs for
>the 128 bits key. I tried to find the documentation online, but every
>this I found they use a __s64 type.

I'm not sure why they called it vader, but vader is a fairly straight
forward shared memory BTL. It differs from sm in two important ways: 1)
it uses the XPMEM driver instead of SysV for shared memory and 2) it uses
the the Nemesis queue structure from MPICH instead of ring buffers. XPMEM
allows you to map large quantities of memory from other processes into
your memory space, so you can do single copy for long messages, and the
Nemesis queue seems to give better scaling on our larger SMPs. The Vader
BTL does not require the 128 bit keys.

>Which function exactly requires 128bits integers? Where is the call to
>this function in the vader BTL?

A number of OMPI developer institutions are working on a new BTL
(different from vader) for the Cray XE series using the uGNI upper layer.
The rkeys in uGNI are 128 bytes.

Brian

-- 
  Brian W. Barrett
  Dept. 1423: Scalable System Software
  Sandia National Laboratories