Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] eager limits?
From: Brock Palen (brockp_at_[hidden])
Date: 2008-03-07 12:40:40


On Mar 3, 2008, at 3:51 PM, Brian W. Barrett wrote:

> On Mon, 3 Mar 2008, Brock Palen wrote:
>
>> I was writing a simple code to demonstrate deadlock. I create an
>> array of some number of doubles, when sizeof(double)*SIZE is >
>> btl_sm_eager_limit (on a smp machine) I expect the code to deadlock
>> but not until the buffer size is greater than the eager limit.
>>
>> Now in 1.2.3 sm_eager_limit is 4096 (bytes?) So i expect a buffer of
>> doubles created with malloc to have up to 512 before the code
>> deadlocks. (remember this is for demonstrations only) The code
>> deadlocks at 504 doubles, does the eager limit not include the
>> envelope overhead? Is it really 64 Kbytes?
>>
>> Thus does every message take up 64K no matter the size of buffer* ?
>
> I think you missed an order of magnitude somewhere :). 504 doubles
> would
> take 4032 bytes, so that would leave 64 bytes for header.
>
> The eager limit is the total packet size, so headers are also
> included in
> that space (explaining why you wouldn't be able to pack a full 512
> doubles) into a single eager message.
>
> There are two headers sent in the SM case - one for the PML (16
> bytes) and
> one for the SM BTL (16 bytes for 32 bit builds, 24 bytes for 64 bit
> builds). So the grand total of header space for an SM message is
> 32 or 40
> bytes, depending on build size. I'm not sure where the other 24 bytes
> would be going in your case.

Ok cool, yes my math sucked that day :-) It all adds up now.

>
> Hope this helps,
>
> Brian
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>