Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] overlapping memcpy in ompi_coll_tuned_allgather_intra_bruck
From: George Bosilca (bosilca_at_[hidden])
Date: 2008-02-03 22:53:41

The reason behind the overlapping restriction on the memcpy, is that
the memcpy is supposed to use the fastest available algorithm on the
particular architecture, without restricting itself to a specific copy
direction (forward or backward). On the other side, memmove is
supposed to choose the direction that guarantee the correctness of the

As you discovered, as long as memcpy do the copy in the forward
direction, there will be no problems with in the
ompi_ddt_copy_content_same_ddt. Do you know any operating system where
memcpy is done backward ?

Now, the overlapping case is a real exception. Obviously, it happened
for at least two peoples (as per mailing list search) in about 4
years, but without affecting the correctness of the application. Is
that a reason good enough to effect the overall performance of all
parallel applications using Open MPI ? You can already guess my stance.

However, I can imagine a way to rewrite the last step of the bruck
algorithm to avoid this problem, and without affecting the overall


On Jan 30, 2008, at 9:41 AM, Number Cruncher wrote:

> I'm getting many "Source and destination overlap in memcpy" errors
> when
> running my application on an odd number of procs.
> I believe this is because the Allgather collective is using Bruck's
> algorithm and doing a shift on the buffer as a finalisation step
> (coll_tuned_allgather.c):
> tmprecv = (char*) rbuf;
> tmpsend = (char*) rbuf + (size - rank) * rcount * rext;
> err = ompi_ddt_copy_content_same_ddt(rdtype, rank * rcount,
> tmprecv, tmpsend);
> Unfortunately ompi_ddt_copy_content_same_ddt does a memcpy, instead of
> the memmove which is needed here. For this buffer-left-shift, any
> forward-copying memcpy should actually be OK as it won't overwrite
> itself during the copy, but this violates the precondition of memcpy
> and
> may break for some implementations.
> I think this issue was dismissed too lightly previously:
> Thanks,
> Simon
> _______________________________________________
> users mailing list
> users_at_[hidden]

  • application/pkcs7-signature attachment: smime.p7s