Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI_Allgather(): Which algorithm is used?
From: Rainer Keller (keller_at_[hidden])
Date: 2009-08-22 22:14:04


Hello,
On Saturday 22 August 2009 12:05:00 pm Dr. Hans Ekkehard Plesser wrote:
> OpenMPI 1.3.3 comes with a range of algorithms for MPI_Allgather(), but I
> have not been able to find out how to check which algorithms is actually
> used.
You may get this information using the debug output using flag --mca
coll_base_verbose 1, however this is turned on only, when build in debugging
mode (aka configure --enable-debug), see variable ompi_coll_tuned_stream.

So, e.g. with:
mpirun -np 4 --mca coll_base_verbose 1 --mca coll_tuned_use_dynamic_rules 1
Collective tests Allgather (31/1), comm MPI_COMM_WORLD (1/13), type MPI_INT
(7/1)
[lap79756:05981] coll:tuned:allgather_intra_recursivedoubling rank 0, size 4
[lap79756:05981] ompi_coll_tuned_allgather_intra_dec_dynamic
[lap79756:05981] ompi_coll_tuned_allgather_intra_dec_fixed rank 0 com_size 4
msg_length 16000
[lap79756:05982] coll:tuned:allgather_intra_recursivedoubling rank 1, size 4
[lap79756:05982] coll:tuned:allgather_intra_recursivedoubling rank 1, size 4
[lap79756:05982] coll:tuned:module_init called.
[lap79756:05982] coll:tuned:module_init MCW & Dynamic
[lap79756:05982] coll:tuned:module_init Tuned is in use
[lap79756:05982] coll:tuned:module_query using intra_dynamic
[lap79756:05982] coll:tuned:module_tuned query called
[lap79756:05982] ompi_coll_tuned_allgather_intra_dec_dynamic
[lap79756:05982] ompi_coll_tuned_allgather_intra_dec_dynamic
[lap79756:05982] ompi_coll_tuned_allgather_intra_dec_fixed rank 1 com_size 4
msg_length 16000
[lap79756:05982] ompi_coll_tuned_allgather_intra_dec_fixed rank 1 com_size 4
msg_length 16000
[lap79756:05984] coll:tuned:allgather_intra_recursivedoubling rank 3, size 4
[lap79756:05984] coll:tuned:allgather_intra_recursivedoubling rank 3, size 4
[lap79756:05984] coll:tuned:module_init called.
[lap79756:05984] coll:tuned:module_init MCW & Dynamic
[lap79756:05984] coll:tuned:module_init Tuned is in use
[lap79756:05984] coll:tuned:module_query using intra_dynamic
[lap79756:05984] coll:tuned:module_tuned query called
[lap79756:05984] ompi_coll_tuned_allgather_intra_dec_dynamic
[lap79756:05984] ompi_coll_tuned_allgather_intra_dec_dynamic
[lap79756:05984] ompi_coll_tuned_allgather_intra_dec_fixed rank 3 com_size 4
msg_length 16000
[lap79756:05984] ompi_coll_tuned_allgather_intra_dec_fixed rank 3 com_size 4
msg_length 16000

> From the following message on the list I got an idea about how to select
> the algoritm by hand
> http://www.open-mpi.org/community/lists/users/2008/10/6847.php,
> but I should like to be able to find out at runtime which algorithm Open
> MPI has chosen.
You want to use this information in Your application?

> BTW, is one algorithm chosen the first time MPI_Allgather() is called, or
> is a new algorithm chosen for every call based on message properties?
If the user does not override the decision, it is done for every MPI_Allgather
call.
With fixed rules using the tuned collectives, the decision is based on msg.
size and whether the number of procs in the communicator is a power of 2.
With dynamic tuned collectives on, the application may even override the
choice between calls to MPI_Allgathers...
George surely will elaborate on the details!

With best regards,
Rainer

PS: More information on the file format for file driven decision functions in
http://icl.cs.utk.edu/news_pub/submissions/Flex-collective-euro-
pvmmpi-2006.pdf

-- 
------------------------------------------------------------------------
Rainer Keller, PhD                  Tel: +1 (865) 241-6293
Oak Ridge National Lab          Fax: +1 (865) 241-4811
PO Box 2008 MS 6164           Email: keller_at_[hidden]
Oak Ridge, TN 37831-2008    AIM/Skype: rusraink