Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-04-19 20:39:22

On Apr 19, 2007, at 1:45 PM, pooja_at_[hidden] wrote:

> I want to put semaphore in bml.h--- mca_bml_send before and after
> calling
> btl_send.
> SO that when a process call btl_send it first lock a global
> variable X and
> then proceeds.Also if an external Tcp function wants to send data it
> should first lock global variable X and then proceed.
> Can anyone tell me only changing bml.h is enough or are there any
> other
> files where I need to make changes.

This is likely to be a complex issue because there's the put and get
functions as well. ob1 uses a fairly complex algorithm to decide
when to call the bml interface functions -- I doubt that the use of a
semaphore in a single location is going to do what you want.

(why a semaphore, anyway -- why not a mutex?)

> (As I tried doing this and run mpi program it gave me ORTE time out
> error
> also when I changed file back to normal it was not compiling and
> giving me
> error in etc...unfortunately I deleted entire folder and
> downloaded new version.)

Changing bml.h should have zero effect on the ORTE layer. ORTE is a
whole different abstraction and wholly below the OMPI layer. There
are a few places in the OMPI layer that interact with the lower ORTE
later, but the bml is not one of them.

I'm guessing that you had some other problem.

If you're going to be working continually with Open MPI, you might
want to get a subversion checkout.

> Can any one please help me and tell me how should I go about
> implementing
> locks/semaphore in bml layer so that all mpi process access lock
> (of same
> priority ) and continue working while Tcp acquire only when network is
> free(or there is lot of serial operation between 2 mpi sends).

I want to emphasize again that this won't give you what you have
described in previous mails: the PML interface is designed to be
asynchronous. So when you call send/put/get, it only (possibly)
*starts* the communication transfer. When you unlock upon return,
you're allowing the alternate communication mechanism to come in and
start another communication method (via a different BTL, perhaps),
but it does not change that there may still be activity occurring
down in the kernel and/or hardware. Also, this scheme does not
account for received message contention -- it only [tries to] account
for sending contention.

So even if you get the locking working the way that you want, I don't
think that you're going to get the overlap and multiplexing that you

Jeff Squyres
Cisco Systems