Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI_Alltoallv and unknown data send sizes
From: George Bosilca (bosilca_at_[hidden])
Date: 2008-09-10 11:10:06


Daniel,

Your understanding of he MPI standard requirement with regard to
MPI_Alltoallv is now 100% accurate. The send count and datatype should
match what the receiver expect. You can always use an MPI_Alltoall
before the MPI_Alltoallv to exchange the lengths that you expect.

   george.

On Sep 10, 2008, at 1:46 PM, Daniel Spångberg wrote:

> Dear all,
>
> First some background, the real question is at the end of this
> (longish) mail.
>
> I have a problem where I need to exchange data between all
> processes. The data is unevenly distributed and I thought at first I
> could use MPI_Alltoallv to transfer the data. However, in my case,
> the receivers do not know how many data items the senders will send,
> but it is relatively easy to set up so the receiver can figure out
> the maximum number of items the sender will send, so I set the
> recvcounts to the maximum possible, and the sendcounts to the actual
> number of elements (smaller than recvcounts).
>
> The mpi-forum description (from http://www.mpi-forum.org/docs/mpi21-report/node99.htm)
> describes the following:
>
> MPI_ALLTOALLV(sendbuf, sendcounts, sdispls, sendtype, recvbuf,
> recvcounts, rdispls, recvtype, comm)
> IN sendbuf starting address of send buffer (choice)
> IN sendcounts integer array equal to the group size specifying the
> number of elements to send to each processor
> IN sdispls integer array (of length group size). Entry j specifies
> the displacement (relative to sendbuf) from which to take the
> outgoing data destined for process j
> IN sendtype data type of send buffer elements (handle)
> OUT recvbuf address of receive buffer (choice)
> IN recvcounts integer array equal to the group size specifying the
> number of elements that can be received from each processor
> IN rdispls integer array (of length group size). Entry i specifies
> the displacement (relative to recvbuf) at which to place the
> incoming data from process i
> IN recvtype data type of receive buffer elements (handle)
> IN comm communicator (handle)
>
> In particular the wording is "the number of elements that can be
> received from each processor" for recvcounts, and does not say that
> this must be exactly the same as the number of elements sent.
>
> It also mentions that it should work similarly as a number of
> independent MPI_Send/MPI_Recv calls. The amount of data sent in such
> a case does not need to exactly match the amount of data received.
>
> I, unfortunately, missed the following:
>
> The type signature associated with sendcounts[j], sendtypes[j] at
> process i must be equal to the type signature associated with
> recvcounts[i], recvtypes[i] at process j. This implies that the
> amount of data sent must be equal to the amount of data received,
> pairwise between every pair of processes. Distinct type maps between
> sender and receiver are still allowed.
>
> And the openmpi man page shows
> When a pair of processes exchanges data, each may pass
> different ele-
> ment count and datatype arguments so long as the sender
> specifies the
> same amount of data to send (in bytes) as the receiver
> expects to
> receive.
>
> I did test my program on different send/recv counts, and while it
> sometimes works, sometimes it does not. Even if it worked I would
> not be comfortable using it anyway.
>
>
> The question is: If there is no way of determining the length of the
> data sent by the sender on the receiving end, I see two options:
> Either always transmit too much data using MPI_Alltoall(v) or cook
> up my own routine based on PTP calls, probably MPI_Sendrecv is the
> best option. Am I missing something?
>
> --
> Daniel Spångberg
> Materials Chemistry
> Uppsala University
> Sweden
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users