Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] Configure issue with/without HWLOC when PGI used and CUDA support enabled
From: Filippo Spiga (spiga.filippo_at_[hidden])
Date: 2014-02-14 09:43:38


Dear Open MPI developers,

I just want to point to a weird behavior of the configure procedure I discovered. I am compiling Open MPI 1.7.4 with CUDA support (CUDA 6.0 RC) and PGI 14.1

If I explicitly compile against a self-compiled version of HWLOC (1.8.1) using this configure line
../configure CC=pgcc CXX=pgCC FC=pgf90 F90=pgf90 --prefix=/usr/local/Cluster-Users/fs395/openmpi-1.7.4/pgi-14.1_cuda-6.0RC --enable-mpirun-prefix-by-default --with-fca=$FCA_DIR --with-mxm=$MXM_DIR --with-knem=$KNEM_DIR --with-hwloc=/usr/local/Cluster-Users/fs395/hwlock-1.8.1/gcc-4.4.7_cuda-6.0RC --with-slurm=/usr/local/Cluster-Apps/slurm --with-cuda=/usr/local/Cluster-Users/fs395/cuda/6.0-RC

make fails telling me that it cannot find "-lcudart".

If I compile without HWLOC using this configure line:
../configure CC=pgcc CXX=pgCC FC=pgf90 F90=pgf90 --prefix=/usr/local/Cluster-Users/fs395/openmpi-1.7.4/pgi-14.1_cuda-6.0RC --enable-mpirun-prefix-by-default --with-fca=$FCA_DIR --with-mxm=$MXM_DIR --with-knem=$KNEM_DIR --with-slurm=/usr/local/Cluster-Apps/slurm --with-cuda=/usr/local/Cluster-Users/fs395/cuda/6.0-RC

make succeeds and I have Open MPI compiled properly.

$ mpif90 -show
pgf90 -I/usr/local/Cluster-Users/fs395/openmpi-1.7.4/pgi-14.1_cuda-6.0RC/include -I/usr/local/Cluster-Users/fs395/openmpi-1.7.4/pgi-14.1_cuda-6.0RC/lib -Wl,-rpath -Wl,/usr/local/Cluster-Users/fs395/openmpi-1.7.4/pgi-14.1_cuda-6.0RC/lib -L/usr/local/Cluster-Users/fs395/openmpi-1.7.4/pgi-14.1_cuda-6.0RC/lib -lmpi_usempif08 -lmpi_usempi_ignore_tkr -lmpi_mpifh -lmpi
$ ompi_info --all | grep btl_openib_have_cuda_gdr
                 MCA btl: informational "btl_openib_have_cuda_gdr" (current value: "true", data source: default, level: 5 tuner/detail, type: bool)

I wonder why the configure picks up lib instead of lib64. I will test the build using real codes.

Cheers,
Filippo

--
Mr. Filippo SPIGA, M.Sc.
http://www.linkedin.com/in/filippospiga ~ skype: filippo.spiga
«Nobody will drive us out of Cantor's paradise.» ~ David Hilbert
*****
Disclaimer: "Please note this message and any attachments are CONFIDENTIAL and may be privileged or otherwise protected from disclosure. The contents are not to be disclosed to anyone other than the addressee. Unauthorized recipients are requested to preserve this confidentiality and to advise the sender immediately of any error in transmission."