Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] 1.7.1 Hang with MPI_THREAD_MULTIPLE set
From: Paul Kapinos (kapinos_at_[hidden])
Date: 2013-06-03 08:06:47


Hello,

It is more or less well-known that MPI_THREAD_MULTIPLE disable the OpenFabric /
InfiniBand networking in Open MPI:

http://www.open-mpi.org/faq/?category=supported-systems#thread-support
http://www.open-mpi.org/community/lists/users/2010/03/12345.php

On our system not only the 'openib' BTL is off, but also the IPoIB denies to
work, leading to error.

But I was able to run your programm error-free when completely disabling using
the InfiniBand: either both processes on same node (using shared memory), or use
"-mca btl ^openib -mca btl_tcp_if_exclude ib0,lo" parameter to the 'mpiexec' in
order to disable InfiniBand and IPoIB.
Well; this is disappinting due to some 20x loss of performance using Gigagbit
Ethernet, comparing the actual InfiniBand...

Note: Intel MPI support MPI_THREAD_MULTIPLE when linked with -mt_mpi (Intel and
GCC compilers) or -lmpi_mt instead of -lmpi (other compilers). However, Intel
MPI is not free.

Best,

Paul Kapinos

Also, I recommend to _always_ check what kinda of threading lievel you ordered
and what did you get:
   print *, 'hello, world!', MPI_THREAD_MULTIPLE, provided

On 05/31/13 06:12, W Spector wrote:
> Dear OpenMPI group,
>
> The following trivial program hangs on the mpi_barrier call with 1.7.1. I am
> using gfortran/gcc 4.6.3 on Ubuntu linux. OpenMPI was built with
> --enable-mpi-thread-multiple support and no other options (other than --prefix).
>
> Are there additional options we should be telling configure about? Or have we
> done something very silly? Mpich2 works just fine...
>
> Walter Spector
>
>
> program hang
> use mpi
> implicit none
>
> integer :: me, npes
> integer :: mpierr, provided
> logical :: iampe0
>
> call mpi_init_thread ( &
> MPI_THREAD_MULTIPLE, &
> provided, &
> mpierr)
> print *, 'hello, world!'
>
> ! Hangs here with MPI_THREAD_MULTIPLE set...
> call mpi_barrier (MPI_COMM_WORLD, mpierr)
>
> call mpi_comm_rank (MPI_COMM_WORLD, me, mpierr)
> iampe0 = me == 0
> call mpi_comm_size (MPI_COMM_WORLD, npes, mpierr)
> print *, 'pe:', me, ', total comm size:', npes
> print *, 'I am ', trim (merge ('PE 0 ', 'not PE 0', iampe0))
>
> call mpi_finalize (mpierr)
>
> end program
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>

-- 
Dipl.-Inform. Paul Kapinos   -   High Performance Computing,
RWTH Aachen University, Center for Computing and Communication
Seffenter Weg 23,  D 52074  Aachen (Germany)
Tel: +49 241/80-24915