Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: [OMPI devel] RGET issue when send is less than receive
From: Rolf vandeVaart (rvandevaart_at_[hidden])
Date: 2013-06-21 11:25:29

I ran into a hang in a test in which the sender sends less data than the receiver is expecting. For example, the following shows the receiver expecting twice what the sender is sending.

Rank 0: MPI_Send(buf, BUFSIZE, MPI_INT, 1, 99, MPI_COMM_WORLD)
Rank 1: MPI_Recv(buf, BUFSIZE*2, MPI_INT, 0, 99, MPI_COMM_WORLD)

This is also reproducible using one of the intel tests and adjusting the eager value for the openib BTL.

Ø mpirun -np 2 -host frick,frack -mca btl_openib_eager_limit 56 MPI_Send_overtake_c

In most cases, this works just fine. However, when the PML protocol used is the RGET protocol, the test hangs. Below is a proposed fix for this issue.
I believe we want to be checking against req_bytes_packed rather than req_bytes_expected as req_bytes_expected is what the user originally told us.
Otherwise, with the current code, we never send a FIN message back to the sender.

Any thoughts?

[rvandevaart_at_sm065 ompi-trunk]$ svn diff ompi/mca/pml/ob1/pml_ob1_recvreq.c
Index: ompi/mca/pml/ob1/pml_ob1_recvreq.c
--- ompi/mca/pml/ob1/pml_ob1_recvreq.c (revision 28633)
+++ ompi/mca/pml/ob1/pml_ob1_recvreq.c (working copy)
@@ -335,7 +335,7 @@
     /* is receive request complete */
     OPAL_THREAD_ADD_SIZE_T(&recvreq->req_bytes_received, frag->rdma_length);
- if (recvreq->req_bytes_expected <= recvreq->req_bytes_received) {
+ if (recvreq->req_recv.req_bytes_packed <= recvreq->req_bytes_received) {

This email message is for the sole use of the intended recipient(s) and may contain
confidential information. Any unauthorized review, use, disclosure or distribution
is prohibited. If you are not the intended recipient, please contact the sender by
reply email and destroy all copies of the original message.