On Mon, Nov 09, 2009 at 11:45:29AM +0100, Vincent Loechner wrote:
> > I expect problems with sizes larger than 2^31-1, but these array sizes
> > are still much smaller.
> No, they are bigger, you allocate two arrays of 320 Mdouble :
> 2 * 320M * 8 = 5GB.
> Are your processes limited to 4GB virtual memory ?
No, they aren't. There are 16GB on each node plus swap space; I have
run processes much bigger than this - in fact this program doesn't
even come close to the memory limits on each node.
The reason why there is a problem for counts larger than 2^31-1 is
that the "count" variable in the MPI library is a MPI_INT - a 32bit integer.
BTW: the MPI_Allreduce version fails (i.e., hangs forever) for an
array size larger than 176763240. This is not a smooth transition:
for 176763240 the program finishes in about 3s, for 176763241 it
This looks to me as if there is an internal buffer or some other
limit that I cannot exceed. Is there such a limit? Or: can somebody
confirm that they can run the program with an array size larger