Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] vader on SGI UV?
From: Paul Hargrove (phhargrove_at_[hidden])
Date: 2014-01-23 16:38:53


I've answered this one for myself:
  NO: the vader blt does not build on an SGI UV
However, xpmem support isn't detected at configure time either.
So, there is no "problem" here.

It might be nice to clarify in README that vader is for Cray's variant of
XPMEM only.

++ Everything below this point is for the "future" milestone. ++

If one ever does want vader/xpmem support on the SGI UV, I see 2 issues to
overcome:

issue 1)
SGI puts the header in /usr/include/sn/xpmem.h
There is no --with-xpmem= value that will let configure find the header in
that location.
This is a "good thing" because if configure did find it, a build would fail
be default due to issue #2.
To support SGI's xpmem will require additional configure logic to look for
EITHER xpmem.h or sn/xpmem.h

To work past issue #1 I did:
  $ mkdir $HOME/xpmem
  $ ln -s /usr/include/sn $HOME/xpmem/include
and configured ompi using --with-xpmem=$HOME/xpmem
That allowed be to see the second issue...

issue 2)
There are some minor API differences in types in SGI's "flavor" of xpmem
which cause the build to fail.
In GASNet we support both variants and the following snippet shows how we
deal with the differences:
  #if defined(HAVE_XPMEM_H)
   /* Cray XPMEM */
   #include <xpmem.h>
   typedef struct xpmem_addr gasneti_xpmem_addr_t;
   typedef xpmem_segid_t gasneti_xpmem_segid_t;
   typedef xpmem_apid_t gasneti_xpmem_apid_t;
   #define gasneti_xpmem_apid apid
  #elif defined(HAVE_SN_XPMEM_H)
   /* SGI XPMEM */
   #include <sn/xpmem.h>
   typedef xpmem_addr_t gasneti_xpmem_addr_t;
   typedef int64_t gasneti_xpmem_segid_t;
   typedef int64_t gasneti_xpmem_apid_t;
   #define gasneti_xpmem_apid id
  #endif

The differences:
+ Cray's "struct xpmem_addr" vs SGI's "xpmem_addr_t"
+ SGI's uses int64_t instead of defining xpmem_segid_t and xpmem_apid_t
+ SGI uses a struct member name of "id" vs Cray's "apid"

Note that the different locations for the header has worked to our benefit
here by providing the variant detection mechanism without the need for
configure probes for the types and members (though one could go that route
if sufficiently paranoid about a variant between the two).

-Paul

On Thu, Jan 23, 2014 at 12:11 PM, Paul Hargrove <phhargrove_at_[hidden]> wrote:

> Nathan,
>
> Is the vader BTL known to work or not work on an SGI UV (w/ XPMEM support,
> of course)?
> I can easily attempt the build, but any test runs would enter a queue that
> is about 1 week deep.
> So, I am wondering if the attempt is worth pursuing.
>
> Additionally, does one need an explicit "-mca btl self,vader" or "-mca btl
> ^sm"?
>
> -Paul
>
> --
> Paul H. Hargrove PHHargrove_at_[hidden]
> Future Technologies Group
> Computer and Data Sciences Department Tel: +1-510-495-2352
> Lawrence Berkeley National Laboratory Fax: +1-510-486-6900
>

-- 
Paul H. Hargrove                          PHHargrove_at_[hidden]
Future Technologies Group
Computer and Data Sciences Department     Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory     Fax: +1-510-486-6900