Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] using openib and psm together
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2011-04-21 18:33:30

I believe it was mainly a startup issue -- there's a complicated sequence of events that happens during MPI_INIT. IIRC, the issue was that if OMPI had software support for PSM, it assumed that the lack of PSM hardware was effectively an error.

v1.5 made the startup sequence a little more flexible; the PSM bits in OMPI can say "Oh yes, we have PSM support, but I don't see any PSM hardware, so just ignore me... please move along... nothing to see here..."

OMPI's openib BTL has had this kind of support for a long time, but PSM and verbs are treated a little differently in the startup sequence because they're fundamentally different kinds of transports (abstraction-wise, anyway).

On Apr 21, 2011, at 6:01 AM, Dave Love wrote:

> We have an installation with both Mellanox and Qlogic IB adaptors (in
> distinct islands), so I built open-mpi 1.4.3 with openib and psm
> support.
> Now I've just read this in the OFED source, but I can't see any relevant
> issue in the open-mpi tracker:
> OpenMPI support
> ---------------
> It is recommended to use the OpenMPI v1.5 development branch. Prior versions
> of OpenMPI have an issue with support PSM network transports mixed with standard
> Verbs transport (BTL openib). This prevents an OpenMPI installation with
> network modules available for PSM and Verbs to work correctly on nodes with
> no QLogic IB hardware. This has been fixed in the latest development branch
> allowing a single OpenMPI installation to target IB hardware via PSM or Verbs
> as well as alternate transports seamlessly.
> Do I definitely need 1.5 (and is 1.5.3 good enough?) to have openib and
> psm working correctly? Also what are the symptoms of it not working
> correctly?
> _______________________________________________
> users mailing list
> users_at_[hidden]

Jeff Squyres
For corporate legal information go to: