Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] hwloc and pmi
From: Mike Dubman (miked_at_[hidden])
Date: 2014-07-12 14:16:03


okay, seems found the reason:

slurm-devel-14.03.4-2.el6.x86_64 comes with this:

$grep hwloc /usr/lib64/*la

/usr/lib64/libpmi.la:dependency_libs=' /usr/lib64/libslurm.la -L/usr/lib64
-ldl -lhwloc -lpthread'

/usr/lib64/libslurmdb.la:dependency_libs=' -L/usr/lib64 -ldl -lhwloc
-lpthread'

/usr/lib64/libslurm.la:dependency_libs=' -L/usr/lib64 -ldl -lhwloc
-lpthread'

After removing "-lhwloc" from la files - ompi can compile.

Please suggest.

On Fri, Jul 11, 2014 at 10:32 PM, Ralph Castain <rhc_at_[hidden]> wrote:

> It's probably being picked up from the PMI check and being added to the
> cppflags for components that call that .m4 (e.g., common/pmi). You might
> print out the cppflags being created in that script and see if that's the
> case.
>
> The slurm check shouldn't be throwing anything into the global cppflags,
> and I don't think common/pmi calls the slurm .m4 check - at least, it
> probably shouldn't
>
> On Jul 11, 2014, at 11:57 AM, Mike Dubman <miked_at_[hidden]>
> wrote:
>
> I think the problem related to new version of SLURM which was upgraded on
> our machines.
> we had 2.6.6 now it is 14.03.4-2
>
> $make V=1
> /bin/sh ../../../../libtool --tag=CC --mode=link gcc -std=gnu99
> -DNDEBUG -O3 -g -finline-functions -fno-strict-aliasing -pthread
> -version-info 2:2:1 -export-dynamic -o libmca_common_pmi.la -rpath
> /hpc/scrap/mtt/scratch/shmem/
> 20140711_210002_6937_8974_sputnik7.vbench.com/installs/vyng/install/lib
> common_pmi.lo -lpmi2 -lpmi -Wl,-rpath= -lrt -lnsl -lutil -lm
> libtool: link: gcc -std=gnu99 -shared -fPIC -DPIC .libs/common_pmi.o
> /usr/lib64/libpmi2.so /usr/lib64/libpmi.so -L/usr/lib64
> /usr/lib64/libslurm.so -ldl -lhwloc -lpthread -lrt -lnsl -lutil -lm -O3
> -pthread -Wl,-rpath= -pthread -Wl,-soname -Wl,libmca_common_pmi.so.1 -o
> .libs/libmca_common_pmi.so.1.1.2
> /usr/bin/ld: cannot find -lhwloc
> collect2: ld returned 1 exit status
> make: *** [libmca_common_pmi.la] Error 1
>
> The Makefile in opal/mca/common/pmi/Makefile has no references to
> "-lhwloc", so it comes as dependancy from outside.
> does it make sense?
>
>
> this is a configure line used:
> $ ./configure --with-platform=contrib/platform/mellanox/optimized
> --with-fca=/opt/mellanox/fca
> --with-mxm=/hpc/local/benchmarks/hpc-stack-gcc/install/mxm --enable-oshmem
> --with-slurm --with-pmi --with-oshmem-param-check
> --with-knem=/opt/knem-1.1.1.90mlnx --prefix=/hpc/scrap/mtt/scratch/shmem/
> 20140711_210002_6937_8974_sputnik7.vbench.com/installs/vyng/install
>
>
> $ldd /usr/lib64/libpmi.so
> linux-vdso.so.1 => (0x00007ffff7ffe000)
> libslurm.so.27 => /usr/lib64/libslurm.so.27 (0x00007ffff7ac6000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffff7897000)
> libc.so.6 => /lib64/libc.so.6 (0x00007ffff7504000)
> libdl.so.2 => /lib64/libdl.so.2 (0x00007ffff7300000)
> libhwloc.so.5 => /usr/lib64/libhwloc.so.5 (0x00007ffff70d7000)
> /lib64/ld-linux-x86-64.so.2 (0x0000003d9de00000)
> libm.so.6 => /lib64/libm.so.6 (0x00007ffff6e53000)
> libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00007ffff6c4a000)
> libpci.so.3 => /lib64/libpci.so.3 (0x00007ffff6a3d000)
> libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00007ffff66eb000)
> libresolv.so.2 => /lib64/libresolv.so.2 (0x00007ffff64d1000)
> libz.so.1 => /lib64/libz.so.1 (0x00007ffff62ba000)
> mtt_at_sputnik7 /hpc/scrap/mtt/scratch/shmem/
> 20140711_210002_6937_8974_sputnik7.vbench.com/mpi-install/NoDd/src/ompi-vendor.git
> $ldd /usr/lib64/libslurm.so
> linux-vdso.so.1 => (0x00007ffff7ffe000)
> libdl.so.2 => /lib64/libdl.so.2 (0x00007ffff7ab6000)
> libhwloc.so.5 => /usr/lib64/libhwloc.so.5 (0x00007ffff788d000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x00007ffff7670000)
> libc.so.6 => /lib64/libc.so.6 (0x00007ffff72dd000)
> /lib64/ld-linux-x86-64.so.2 (0x0000003d9de00000)
> libm.so.6 => /lib64/libm.so.6 (0x00007ffff7058000)
> libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x00007ffff6e4f000)
> libpci.so.3 => /lib64/libpci.so.3 (0x00007ffff6c43000)
> libxml2.so.2 => /usr/lib64/libxml2.so.2 (0x00007ffff68f0000)
> libresolv.so.2 => /lib64/libresolv.so.2 (0x00007ffff66d6000)
> libz.so.1 => /lib64/libz.so.1 (0x00007ffff64c0000)
>
>
>
>
>
>
> On Thu, Jul 10, 2014 at 6:53 PM, Nathan Hjelm <hjelmn_at_[hidden]> wrote:
>
>> Nope, just added a missing file to the tarball.
>>
>> -Nathan
>>
>> On Thu, Jul 10, 2014 at 06:54:19AM -0700, Ralph Castain wrote:
>> > IIRC, I thought I saw a change to that makefile.am flow thru
>> yesterday?
>> > Could be there was an error in it
>> > On Jul 10, 2014, at 5:26 AM, Jeff Squyres (jsquyres) <
>> jsquyres_at_[hidden]>
>> > wrote:
>> >
>> > Shouldn't be - PMI should be linking against the internal hwloc.
>> > I'm AFK and can't look at this. Have a look at other components
>> that use
>> > hwloc and copy their header file setup and make file.am setup.
>> >
>> > Sent from my phone. No type good.
>> > On Jul 10, 2014, at 8:22 AM, "Mike Dubman" <
>> miked_at_[hidden]>
>> > wrote:
>> >
>> > Hi guys,
>> > jenkins node failing on this.
>> > is hwloc-devel now required to be available as part of distro?
>> > Thanks
>> > M
>> >
>> > 15:14:11 make[3]: Leaving directory
>> `/scrap/jenkins/scrap/workspace/hpc-ompi-shmem/label/hpc-test-node/opal'
>> > 15:14:11 make[2]: Leaving directory
>> `/scrap/jenkins/scrap/workspace/hpc-ompi-shmem/label/hpc-test-node/opal'
>> > 15:14:11 Making install in mca/common/pmi
>> > 15:14:11 make[2]: Entering directory
>> `/scrap/jenkins/scrap/workspace/hpc-ompi-shmem/label/hpc-test-node/opal/mca/common/pmi'
>> > 15:14:11 CC common_pmi.lo
>> > 15:14:11 CCLD libmca_common_pmi.la
>> > 15:14:11 /usr/bin/ld: cannot find -lhwloc
>> > 15:14:11 collect2: ld returned 1 exit status
>> > 15:14:11 make[2]: *** [libmca_common_pmi.la] Error 1
>> > 15:14:11 make[2]: Leaving directory
>> `/scrap/jenkins/scrap/workspace/hpc-ompi-shmem/label/hpc-test-node/opal/mca/common/pmi'
>> >
>> > _______________________________________________
>> > devel mailing list
>> > devel_at_[hidden]
>> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> > Link to this post:
>> > http://www.open-mpi.org/community/lists/devel/2014/07/15090.php
>> >
>> > _______________________________________________
>> > devel mailing list
>> > devel_at_[hidden]
>> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> > Link to this post:
>> > http://www.open-mpi.org/community/lists/devel/2014/07/15091.php
>>
>> > _______________________________________________
>> > devel mailing list
>> > devel_at_[hidden]
>> > Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> > Link to this post:
>> http://www.open-mpi.org/community/lists/devel/2014/07/15096.php
>>
>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
>> Link to this post:
>> http://www.open-mpi.org/community/lists/devel/2014/07/15099.php
>>
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/07/15110.php
>
>
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/07/15112.php
>