----- Original Message ----
From: Brian W. Barrett <brbarret@open-mpi.org>
To: Open MPI Developers <devel@open-mpi.org>
Sent: Wednesday, February 6, 2008 2:57:31 AM
Subject: Re: [OMPI devel] xensocket - callbacks through OPAL/libevent

On Mon, 4 Feb 2008, Muhammad Atif wrote:

> I am trying to port xensockets to openmpi. In principle, I have the
> framework and everything, but there seems to be a small issue, I cannot
> get libevent (or OPAL) to give callbacks for receive (or send) for
> xensockets. I have tried to implement native code for xensockets with
> libevent library, again the same issue.  No call backs! . With normal
> sockets, callbacks do come easily.
>
> So question is, do the socket/file descriptors have to have some special
> mechanism attached to them to support callbacks for libevent/opal? i.e
> some structure/magic?. i.e. maybe the developers of xensockets did not
> add that callback/interrupt thing at the time of creation. Xensockets is
> open source, but my knowledge about these issues is limited. So I though
> some pointer in right direction might be useful.

Yes and no :).  As you discovered, the OPAL interface just repackages a
library called libevent to handle its socket multiplexing.  Libevent can
use a number of different mechanisms to look for activity on sockets,
including select() and poll() calls.  On Linux, it will generally use
poll().  poll() requires some kernel support to do its thing, so if
Xensockets doesn't implement the right magic to trigger poll() events,
then libevent won't work for Xensockets.  There's really nothing you can
do from the Open MPI front to work around this issue -- it would have to
be fixed as part of Xensockets.

> Second question is, what if we cannot have the callbacks. What is the
> recommended way to implement the btl component for such a device? Do we
> need to do this with event timers?

Have a look at any of the BTLs that isn't TCP -- none of them use libevent
callbacks for progress.  Instead, they provide a progress function as part
of the BTL interface, which is called on a regular basis whenever progress
needs to be made.

Brian
_______________________________________________
devel mailing list
devel@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/devel