Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
From: Terry Dontje (terry.dontje_at_[hidden])
Date: 2011-07-15 23:25:01


I have to agree with Jeff, we really need a complete program to really
debug this. Note, without really seeing what the structures look like
it is hard to determine if maybe there is some type of structure
mismatch going between recv_packet and load_packet. Also the output you
show seems incomplete in that not all data transfers are being shown so
it is kind of hard to determine if packets are possibly being dropped or
what.

I agree the output looks suspicious but it still leaves a lot to
interpretation without really seeing a complete code.

Sorry,

--td

On 7/15/2011 3:44 PM, Jeff Squyres wrote:
> Can you write this up in a small, complete program that shows the problem, and that we can compile and run?
>
>
> On Jul 15, 2011, at 3:36 PM, Mudassar Majeed wrote:
>
>> *id is same as myid
>>
>> I am comparing the results by seeing the printed messages, given by the printfs....
>>
>> the recv_packet.rank is the rank of the sender that should be equal to status.MPI_SOURCE but it is not.
>>
>> I have updated the code a little bit, here is it.
>>
>> if( (is_receiver == 1)&& (is_sender != 1) )
>> {
>> printf("\nP%d>> Receiver only ...!!", myid);
>> printf("\n");
>> MPI_Recv(&recv_packet, 1, loadDatatype, MPI_ANY_SOURCE, MPI_TAG_LOAD, comm,&status);
>> printf("\nP%d>> Received from P%d, packet contains rank: %d", myid, status.MPI_SOURCE, recv_packet.rank);
>> printf("\n");
>> }
>> else if( (is_sender == 1)&& (is_receiver != 1) )
>> {
>> load_packet.rank = myid;
>> load_packet.ld = load;
>> printf("\nP%d>> Sender only ...!! P%d", myid, rec_rank);
>> printf("\n");
>> MPI_Ssend(&load_packet, 1, loadDatatype, rec_rank, MPI_TAG_LOAD, comm);
>> }
>> else if( (is_receiver == 1)&& (is_sender == 1) )
>> {
>> load_packet.rank = myid;
>> load_packet.ld = load;
>> printf("\nP%d>> Both ...!! P%d", myid, rec_rank);
>> printf("\n");
>> MPI_Sendrecv(&load_packet, 1, loadDatatype, rec_rank, MPI_TAG_LOAD,
>> &recv_packet, 1, loadDatatype, MPI_ANY_SOURCE, MPI_TAG_LOAD, comm,&status);
>> printf("\nP%d>> Received from P%d, packet contains rank: %d", myid, status.MPI_SOURCE, recv_packet.rank);
>> printf("\n");
>> }
>>
>> here is the output
>>
>> P11>> Sender only ...!! P2
>>
>> P14>> Sender only ...!! P6
>>
>> P15>> Neither ...!!
>>
>> P15>> I could reach here ...!!
>>
>> P8>> Neither ...!!
>>
>> P8>> I could reach here ...!!
>>
>> P1>> Receiver only ...!!
>>
>> P9>> Sender only ...!! P0
>>
>> P2>> Receiver only ...!!
>>
>>
>> P10>> Sender only ...!! P1
>>
>> P3>> Receiver only ...!!
>>
>> P3>> Received from P13, packet contains rank: 14
>>
>>
>> P0>> Receiver only ...!!
>>
>> P0>> Received from P3, packet contains rank: 9
>>
>> P4>> Receiver only ...!!
>>
>> P12>> Neither ...!!
>>
>> P12>> I could reach here ...!!
>>
>> P5>> Both ...!! P3
>>
>> P13>> Sender only ...!! P4
>>
>> P13>> I could reach here ...!!
>>
>> P6>> Both ...!! P5
>>
>> P7>> Neither ...!!
>>
>> P7>> I could reach here ...!!
>>
>> P14>> I could reach here ...!!
>>
>> P1>> Received from P7, packet contains rank: 11
>>
>> P1>> I could reach here ...!!
>>
>> P9>> I could reach here ...!!
>> P2>> Received from P11, packet contains rank: 13
>>
>> P2>> I could reach here ...!!
>>
>> P0>> I could reach here ...!!
>>
>> P11>> I could reach here ...!!
>> P3>> I could reach here ...!!
>>
>>
>> regards,
>> Mudassar
>>
>> From: Terry Dontje<terry.dontje_at_[hidden]>
>> To: Mudassar Majeed<mudassarm30_at_[hidden]>
>> Cc: "users_at_[hidden]"<users_at_[hidden]>
>> Sent: Friday, July 15, 2011 9:06 PM
>> Subject: Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
>>
>>
>>
>> On 7/15/2011 2:35 PM, Mudassar Majeed wrote:
>>> Here is the code
>>>
>>> if( (is_receiver == 1)&& (is_sender != 1) )
>>> {
>>> printf("\nP%d>> Receiver only ...!!", myid);
>>> printf("\n");
>>> MPI_Recv(&recv_packet, 1, loadDatatype, MPI_ANY_SOURCE, MPI_TAG_LOAD, comm,&status);
>>> printf("\nP%d>> Received from P%d", myid, status.MPI_SOURCE);
>>> printf("\n");
>>> }
>>> else if( (is_sender == 1)&& (is_receiver != 1) )
>>> {
>>> load_packet.rank = *id;
>>> load_packet.ld = load;
>>> printf("\nP%d>> Sender only ...!! P%d", myid, rec_rank);
>>> printf("\n");
>>> MPI_Ssend(&load_packet, 1, loadDatatype, rec_rank, MPI_TAG_LOAD, comm);
>>> }
>>> else if( (is_receiver == 1)&& (is_sender == 1) )
>>> {
>>> load_packet.rank = *id;
>>> load_packet.ld = load;
>>> printf("\nP%d>> Both ...!! P%d", myid, rec_rank);
>>> printf("\n");
>>> MPI_Sendrecv(&load_packet, 1, loadDatatype, rec_rank, MPI_TAG_LOAD,
>>> &recv_packet, 1, loadDatatype, MPI_ANY_SOURCE, MPI_TAG_LOAD, comm,&status);
>>> printf("\nP%d>> Received from P%d", myid, status.MPI_SOURCE);
>>> printf("\n");
>>> }
>>>
>>> A process can be a message sender, or receiver or both. There are 16 ranks. "rec_rank" contains the rank of the receiver. It is displayed before the message is sent.
>>> Every sender displays this "rec_rank" and it should correctly. But on the receiver sides, status.MPI_SOURCE is displayed (after receiving message), but the value
>>> is not matching with the expected sender's rank.
>> Sorry, but I still don't see how you are detecting the mismatch. I assume load_packet_rank some how relates to load_packet. But why are you setting it to *id instead of myid? Also on the receive side I see no place where you pull out the rank from the recv_packet to compare with status.MPI_SOURCE.
>>> I did not understand about kernel that you were talking about.
>>>
>> A "kernel" that I am talking about is a small piece of code someone can build and run to see the problem.
>>> See the code is very clear and it sends the message to "rec_rank" that was displayed before sending the message. But on the receiver side the MPI_SOURCE comes to be wrong.
>>> This shows to me that messages on the receiving sides are captured on the basis of MPI_ANY_SOURCE, that seems like it does not see the destination of message while capturing it from message queue of the MPI system.
>>>
>>> regards,
>>> Mudassar
>>>
>>> From: Terry Dontje<terry.dontje_at_[hidden]>
>>> To: Mudassar Majeed<mudassarm30_at_[hidden]>
>>> Cc: "users_at_[hidden]"<users_at_[hidden]>
>>> Sent: Friday, July 15, 2011 7:10 PM
>>> Subject: Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
>>>
>>>
>>>
>>> On 7/15/2011 12:49 PM, Mudassar Majeed wrote:
>>>> Yes, processes receive messages that were not sent to them. I am receiving the message with the following call
>>>>
>>>> MPI_Recv(&recv_packet, 1, loadDatatype, MPI_ANY_SOURCE, MPI_TAG_LOAD, comm,&status);
>>>>
>>>> and that was sent using the following call,
>>>>
>>>> MPI_Ssend(&load_packet, 1, loadDatatype, rec_rank, MPI_TAG_LOAD, comm);
>>>>
>>>> What problem it can have ?. All the parameters are correct, I have seen them by printf. What I am thinking is that, the receive is done with MPI_ANY_SOURCE, so the process is getting any message (from any source). What should be done so that only that message is captured that had the destination as this process.
>>>>
>>> By virtue of MPI the MPI_Recv call should only return messages destined for that rank. What makes you think that is not happening? Can you make some sort of kernel of code that proves your theory that your MPI_Recv is receiving another rank's message? If you can and then post that code maybe we'll be able to figure out what the issue is.
>>>
>>> Right now, it seems we are at a deadlock of you claiming something is happening that really cannot be happening. So unless we have more than a broad description of the problem it is going to be nearly impossible for us to tell you what is wrong.
>>>
>>> --td
>>>> regards,
>>>> Mudassar
>>>>
>>>> Date: Fri, 15 Jul 2011 07:04:34 -0400
>>>> From: Terry Dontje<terry.dontje_at_[hidden]>
>>>> Subject: Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
>>>> To: users_at_[hidden]
>>>> Message-ID:<4E201EC2.803_at_[hidden]>
>>>> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
>>>>
>>>> Well MPI_Recv does give you the message that was sent specifically to
>>>> the rank calling it by any of the processes in the communicator. If you
>>>> think the message you received should have gone to another rank then
>>>> there is a bug somewhere. I would start by either adding debugging
>>>> printf's to your code to trace the messages. Or narrowing down the
>>>> code to a small kernel such that you can prove to yourself that MPI is
>>>> working the way it should and if not you can show us where it is going
>>>> wrong.
>>>>
>>>> --td
>>>>
>>>> On 7/15/2011 6:51 AM, Mudassar Majeed wrote:
>>>>> I get the sender's rank in status.MPI_SOURCE, but it is different than
>>>>> expected. I need to receive that message which was sent to me, not any
>>>>> message.
>>>>>
>>>>> regards,
>>>>>
>>>>> Date: Fri, 15 Jul 2011 06:33:41 -0400
>>>>> From: Terry Dontje<terry.dontje_at_[hidden]
>>>>> <mailto:terry.dontje_at_[hidden]>>
>>>>> Subject: Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
>>>>> To: users_at_[hidden]<mailto:users_at_[hidden]>
>>>>> Message-ID:<4E201785.6010101_at_[hidden]
>>>>> <mailto:4E201785.6010101_at_[hidden]>>
>>>>> Content-Type: text/plain; charset="iso-8859-1"; Format="flowed"
>>>>>
>>>>> Mudassar,
>>>>>
>>>>> You can do what you are asking. The receiver uses MPI_ANY_SOURCE for
>>>>> the source rank value and when you receive a message the
>>>>> status.MPI_SOURCE will contain the rank of the actual sender not the
>>>>> receiver's rank. If you are not seeing that then there is a bug
>>>>> somewhere.
>>>>>
>>>>> --td
>>>>>
>>>>> On 7/14/2011 9:54 PM, Mudassar Majeed wrote:
>>>>>> Friend,
>>>>>> I can not specify the rank of the sender. Because only
>>>>>> the sender knows to which receiver the message is to be sent. The
>>>>>> receiver does not know from which sender the message will come. I am
>>>>>> trying to do a research work on load balancing in MPI application
>>>>>> where load is redistributed, so in that I require a receiver to
>>>>>> receive a load value from a sender that it does not know. On the other
>>>>>> hand, the sender actually calculates, to which receiver this load
>>>>>> value should be sent. So for this, I want sender to send a message
>>>>>> containing the load to a receiver, but receiver does not know from
>>>>>> which sender the message will come. See, it is like send receiver in
>>>>>> DATAGRAM sockets. The receiver, receives the message on the IP and
>>>>>> port, the message which was directed for it. I want to have same
>>>>>> behavior. But it seems that it is not possible in MPI. Isn't it?
>>>>>>
>>>>>> regards,
>>>>>> Mudassar
>>>>>>
>>>>>> ------------------------------------------------------------------------
>>>>>> *From:* Jeff Squyres<jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>>
>>>>>> *To:* Mudassar Majeed<mudassarm30_at_[hidden]
>>>>> <mailto:mudassarm30_at_[hidden]>>
>>>>>> *Cc:* Open MPI Users<users_at_[hidden]<mailto:users_at_[hidden]>>
>>>>>> *Sent:* Friday, July 15, 2011 3:30 AM
>>>>>> *Subject:* Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
>>>>>>
>>>>>> Right. I thought you were asking about receiving *another* message
>>>>>> from whomever you just received from via ANY_SOURCE.
>>>>>>
>>>>>> If you want to receive from a specific sender, you just specify the
>>>>>> rank you want to receive from -- not ANY_SOURCE.
>>>>>>
>>>>>> You will always only receive messages that were sent to *you*.
>>>>>> There's no MPI_SEND_TO_ANYONE_WHO_IS_LISTENING functionality, for
>>>>>> example. So your last statement: "But when it captures with ..
>>>>>> MPI_ANY_SOURCE and MPI_ANY_TAG, the receiver will capture any message
>>>>>> (even not targetted for it)" is incorrect.
>>>>>>
>>>>>> I guess I still don't understand your question...?
>>>>>>
>>>>>>
>>>>>> On Jul 14, 2011, at 9:17 PM, Mudassar Majeed wrote:
>>>>>>
>>>>>>> I know this, but when I compare status.MPI_SOURCE with myid, they
>>>>>> are different. I guess you need to reconsider my question. The
>>>>>> MPI_Recv function seems to capture message from the queue with some
>>>>>> search parameters like source, tag etc. So in case the receiver does
>>>>>> not know the sender and wants to receive only that message which was
>>>>>> sent for this receiver. But when it captures with source as
>>>>>> MPI_ANY_SOURCE and MPI_ANY_TAG, the receiver will capture any message
>>>>>> (even not targetted for it).
>>>>>>> regards,
>>>>>>> Mudassar
>>>>>>>
>>>>>>>
>>>>>>> From: Jeff Squyres<jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>
>>>>> <mailto:jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>>>
>>>>>>> To: Mudassar Majeed<mudassarm30_at_[hidden]
>>>>> <mailto:mudassarm30_at_[hidden]>
>>>>>> <mailto:mudassarm30_at_[hidden]<mailto:mudassarm30_at_[hidden]>>>; Open
>>>>> MPI Users<users_at_[hidden]<mailto:users_at_[hidden]>
>>>>>> <mailto:users_at_[hidden]<mailto:users_at_[hidden]>>>
>>>>>>> Sent: Friday, July 15, 2011 1:58 AM
>>>>>>> Subject: Re: [OMPI users] Urgent Question regarding, MPI_ANY_SOURCE.
>>>>>>>
>>>>>>> When you use MPI_ANY_SOURCE in a receive, the rank of the actual
>>>>>> sender is passed back to you in the status.MPI_SOURCE.
>>>>>>> On Jul 14, 2011, at 7:55 PM, Mudassar Majeed wrote:
>>>>>>>
>>>>>>>> Hello people,
>>>>>>>> I am trapped in the following problem plz
>>>>>> help me. Suppose a process A sends a message to process B. The process
>>>>>> B will receive the message with MPI_Recv with MPI_ANY_SOURCE in the
>>>>>> source argument. Let say process B does not know that A is the sender.
>>>>>> But I want B to receive message from process A (the one who actually
>>>>>> sends the message to process B). But if I use MPI_ANY_SOURCE, then any
>>>>>> message from any source is captured by process B (let say there are
>>>>>> other processes sending messages). Instead of MPI_ANY_SOURCE I cannot
>>>>>> use A in the source argument as B does not know about the sender. What
>>>>>> should I do in this situation ?
>>>>>>>> regards,
>>>>>>>> Mudassar Majeed
>>> --
>>> <Mail Attachment.gif>
>>> Terry D. Dontje | Principal Software Engineer
>>> Developer Tools Engineering | +1.781.442.2631
>>> Oracle - Performance Technologies
>>> 95 Network Drive, Burlington, MA 01803
>>> Email terry.dontje_at_[hidden]
>>>
>>>
>>>
>>>
>>>
>> --
>> <Mail Attachment.gif>
>> Terry D. Dontje | Principal Software Engineer
>> Developer Tools Engineering | +1.781.442.2631
>> Oracle - Performance Technologies
>> 95 Network Drive, Burlington, MA 01803
>> Email terry.dontje_at_[hidden]
>>
>>
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Oracle
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.781.442.2631
Oracle *- Performance Technologies*
95 Network Drive, Burlington, MA 01803
Email terry.dontje_at_[hidden] <mailto:terry.dontje_at_[hidden]>



picture