Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] RFC: Linuxes shipping libibverbs
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-05-28 08:02:41


Ok. With lots more off-list discussion, how's this pseudocode for a
proposal:

   # Main assumption: if the kernel drivers are loaded, the user wants
RDMA
   # hardware support in OMPI.

   $sysfsdir = ibv_get_sysfs_path();
   # Avoid printing "Fatal: couldn't read uverbs ABI version" message.
   if (! -r "$sysfsdir/class/infiniband_verbs/abi_version") {
       if ($always_want_to_see_warnings)
           print "Warning: verbs ABI version unreadable\n";
       return SKIP_THIS_BTL;
   }

   # If sysfs/class/infiniband does not exist, the driver was not
started.
   # Therefore: assume that the user does not want RDMA hardware
support --
   # do *not* print a warning message.
   if (! -d "$sysfsdir/class/infiniband") {
       if ($always_want_to_see_warnings)
           print "Warning: $sysfsdir/class/infiniband does not exist\n";
       return SKIP_THIS_BTL;
   }

   # If we get to this point, the drivers are loaded and therefore we
will
   # assume that there is supposed to be at least one RDMA device
present.
   # Warn if we don't find any.
   $list = ibv_get_device_list();
   if (empty($list)) {
       print "Warning: couldn't find any RDMA devices -- if you have
no RDMA devices, stop the driver to avoid this warning message\n";
       return SKIP_THIS_BTL;
   }

   # ...continue with initialization; warnings and errors are
   # *always* displayed after this point

An overriding assumption here is that if the user requested *only* the
openib BTL in OMPI and it fails to find any devices, OMPI will always
print an error that it was unable to reach remote MPI peers
(regardless of whether the default warning was previously printed or
not).

Note that the two /sys checks may be redundant; I'm not entirely sure
how the two files relate to each other. libibverbs will complain
about the first if it is not present; the second is used to indicate
that the kernel drivers are loaded.

On May 26, 2008, at 5:10 AM, Manuel Prinz wrote:

> Am Samstag, den 24.05.2008, 17:30 +0200 schrieb Manuel Prinz:
>> Am Donnerstag, den 22.05.2008, 17:18 -0400 schrieb Jeff Squyres:
>>> Could you check with some of your other Debian maintainers?
>>
>> I'm sorry that I can't check that before Monday! I'll let you know
>> then but I'm not aware of that.
>
> I just checked on a box with no InfiniBand hardware: /dev/infiniband
> *does not* exist. Loading the IB kernel modules *does not* create the
> device. I seems like it only exists if the hardware is present.
>
> Best regards
> Manuel
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
Cisco Systems