Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] CentOS 6.3 & OpenMPI 1.6.3
From: Mike Dubman (mike.ompi_at_[hidden])
Date: 2012-12-02 05:56:06


please redownload from
http://mellanox.com/downloads/hpc/mxm/v1.1/mxm-latest.tar
it contains binaries compiled with mofed 1.5.3-3.1.0
M

On Sun, Dec 2, 2012 at 12:13 PM, Mike Dubman <mike.ompi_at_[hidden]> wrote:

>
> It seems that your active mofed is 1.5.3-3.1.0, while installed mxm was
> compiled with 1.5.3-3.0.0
> MOFED is not binary compatible, let me check and send you the link for mxm
> compiled with mofed that you have.
>
> Also, MOFED contains ompi 1.6.0 which is already compiled with mxm
> (/usr/mpi/...)
> On Sun, Dec 2, 2012 at 12:01 PM, Joseph Farran <jfarran_at_[hidden]> wrote:
>
>> Same thing.
>>
>> My new config:
>>
>> CFLAGS="" FCFLAGS="" ./configure \
>> --with-sge \
>> --with-openib=/usr \
>>
>> --enable-openib-connectx-xrc \
>> --enable-mpi-thread-multiple \
>> --with-threads \
>> --with-hwloc \
>> --enable-heterogeneous \
>> --with-fca=/opt/mellanox/fca \
>> --with-mxm-libdir=/opt/mellanox/mxm/lib \
>> --with-mxm=/opt/mellanox/mxm \
>>
>> Fails at the same spot:
>>
>>
>> make[2]: Entering directory
>> `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm'
>> CC mtl_mxm.lo
>> CC mtl_mxm_cancel.lo
>> CC mtl_mxm_endpoint.lo
>> CC mtl_mxm_probe.lo
>> CC mtl_mxm_recv.lo
>>
>> CCLD mca_mtl_mxm.la
>> /bin/grep: /usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la: No such
>> file or directory
>> /bin/sed: can't read /usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la:
>> No such file or directory
>> libtool: link: `/usr/local/mofed-inst/1.5.3-3.0.0/lib/librdmacm.la' is
>> not a valid libtool archive
>> make[2]: *** [mca_mtl_mxm.la] Error 1
>> make[2]: Leaving directory
>> `/data/apps/sources/openmpi-1.6.3/ompi/mca/mtl/mxm'
>> make[1]: *** [all-recursive] Error 1
>> make[1]: Leaving directory `/data/apps/sources/openmpi-1.6.3/ompi'
>> make: *** [all-recursive] Error 1
>>
>>
>>
>> On 12/2/2012 1:37 AM, Mike Dubman wrote:
>>
>> please change "--with-openib" to "--with-openib=/usr" and retry
>> configure/make stage.
>> 10x
>>
>> On Sun, Dec 2, 2012 at 10:36 AM, Joseph Farran <jfarran_at_[hidden]> wrote:
>>
>>> Hi Mike.
>>>
>>> Thanks for the help!
>>>
>>> I am installing OFED on an NFS share partition so that all compute nodes
>>> will have access.
>>>
>>> For the "--with-openib" option, I don't specify one. My config file
>>> looks like this:
>>>
>>> CFLAGS="" FCFLAGS="" ./configure \
>>>
>>> --with-sge \
>>> --with-openib \
>>> --enable-openib-connectx-xrc \
>>> --enable-mpi-thread-multiple \
>>> --with-threads \
>>> --with-hwloc \
>>> --enable-heterogeneous \
>>> --with-fca=/opt/mellanox/fca \
>>> --with-mxm-libdir=/opt/mellanox/mxm/lib \
>>> --with-mxm=/opt/mellanox/mxm \
>>> --prefix=/data/openmpi-1-6.3
>>>
>>> Please advise,
>>> Joseph
>>>
>>>
>>>
>>>
>>>
>>>
>>> On 12/1/2012 11:39 PM, Mike Dubman wrote:
>>>
>>> Hi Joseph,
>>> I guess you install MOFED under /usr, is that right?
>>> Could you please specify "--with-openib=/usr" parameter during ompi
>>> "configure" stage?
>>> 10x
>>> M
>>>
>>> On Fri, Nov 30, 2012 at 1:11 AM, Joseph Farran <jfarran_at_[hidden]> wrote:
>>>
>>>> Hi YK:
>>>>
>>>> Yes, I have those installed but they are newer versions:
>>>>
>>>> # rpm -qa | grep rdma
>>>> librdmacm-1.0.15-1.x86_64
>>>> librdmacm-utils-1.0.15-1.x86_64
>>>> librdmacm-devel-1.0.15-1.x86_64
>>>> # locate librdmacm.la
>>>> #
>>>>
>>>> Here are the RPMs the Mellanox build created for kernel:
>>>> 2.6.32-279.14.1.el6.x86_64
>>>>
>>>> # ls *rdma*
>>>> librdmacm-1.0.15-1.i686.rpm librdmacm-devel-1.0.15-1.i686.rpm
>>>> librdmacm-utils-1.0.15-1.i686.rpm
>>>> librdmacm-1.0.15-1.x86_64.rpm librdmacm-devel-1.0.15-1.x86_64.rpm
>>>> librdmacm-utils-1.0.15-1.x86_64.rpm
>>>>
>>>>
>>>> On 11/29/2012 02:59 PM, Yevgeny Kliteynik wrote:
>>>>
>>>>> Joseph,
>>>>>
>>>>>
>>>>> You're supposed to have librdmacm installed as part of MLNX_OFED
>>>>> installation.
>>>>> What does "rpm -qa | grep rdma" tell?
>>>>>
>>>>> $ rpm -qa | grep rdma
>>>>> librdmacm-devel-1.0.14.1-1.x86_64
>>>>> librdmacm-utils-1.0.14.1-1.x86_64
>>>>> librdmacm-1.0.14.1-1.x86_64
>>>>>
>>>>> $ locate librdmacm.la
>>>>> /usr/local/mofed/1.5.3-4.0.9/lib/librdmacm.la
>>>>>
>>>>> -- YK
>>>>>
>>>>>
>>>>
>>>
>>>
>>
>>
>