I believe it is definitely a no-no to STORE (write) into a send buffer while a send is posted.  I know there have been debate in the forum to relax LOADS (reads) from a send buffer.  I think OMPI can handle the latter case (LOADS).  On the posted receive side you open yourself up for some race conditions and overwrites if you do STORES or LOADS from a posted receive buffer.


Alberto Canestrelli wrote:
I have a problem with a fortran code that I have parallelized with MPI. I state in advance that I read the whole ebook "Mit Press - Mpi - The Complete Reference, Volume 1" and I took different MPI classes, so I have a discrete MPI knowledge. I was able to solve by myself all the errors I bumped into but now I am not able to find the bug of my code that provides erroneous results. Without entering in the details of my code, I think that the cause of the problem could be reletad to the following aspect highlighted in the above ebook (in the follow I copy and paste from the e-book):

A nonblocking post-send call indicates that the system may start copying data
out of the send buffer. The sender must not access any part of the send buffer
(neither for loads nor for STORES) after a nonblocking send operation is posted until
the complete send returns.
A nonblocking post-receive indicates that the system may start writing data into
the receive buffer. The receiver must not access any part of the receive buffer after
a nonblocking receive operation is posted, until the complete-receive returns.
Rationale. We prohibit read accesses to a send buffer while it is being used, even
though the send operation is not supposed to alter the content of this buffer. This
may seem more stringent than necessary, but the additional restriction causes little
loss of functionality and allows better performance on some systems- consider
the case where data transfer is done by a DMA engine that is not cache-coherent
with the main processor.End of rationale.

I use plenty of nonblocking post-send in my code. Is it really true that the sender must not access any part of the send buffer not even for STORES?  Or was it a MPI 1.0 issue?
users mailing list

Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.650.633.7054
Oracle - Performance Technologies
95 Network Drive, Burlington, MA 01803
Email terry.dontje@oracle.com