Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] Collective communications may be abend when it use over 2GiB buffer
From: Y.MATSUMOTO (yuki.matsumoto_at_[hidden])
Date: 2012-03-05 04:02:38


Dear All,

Next feedback is about "collective communications".

Collective communication may be abend when it use over 2GiB buffer.
This problem occurs following condition:
-- communicator_size * count(scount/rcount) >= 2GiB
It occurs in even small PC cluster.

The following is one of the suspicious parts.
(Many similar code in ompi/coll/tuned/*.c)

--- in ompi/coll/tuned/coll_tuned_allgather.c (V1.4.X's trunk)---
398 tmprecv = (char*) rbuf + rank * rcount * rext;
-----------------------------------------------------------------

if this condition is met, "rank * rcount" is overflowed.
So, we fixed it tentatively like following:
(cast int to size_t)
--- in ompi/coll/tuned/coll_tuned_allgather.c --------------
398 tmprecv = (char*) rbuf + (size_t)rank * rcount * rext;
------------------------------------------------------------

It needs not only "ompi/coll/tuned" but also other codes to fix this problem.
We try to fix, but following functions have problem (argument may be overflowed):
-"ompi_coll_tuned_sendrecv" may be called when "scount/rcount" sets over 2GiB.
-"ompi_datatype_copy_content_same_ddt" may be called when "count" sets over 2GiB.
-"basic_linear in Allgather": Bcast may be called when "count" sets over 2GiB.

Best Regards,
Yuki Matsumoto
MPI development team,
Fujitsu