Any error messages?  Maybe the nodes ran out of memory?  I know MPI implement some kind of buffering under the hood, so even though you&#39;re sending array&#39;s over 2^26 in size, it may require more than that for MPI to actually send it.<br>

<br><div class="gmail_quote">On Mon, Apr 4, 2011 at 2:16 PM, Michael Di Domenico <span dir="ltr">&lt;<a href="mailto:mdidomenico4@gmail.com">mdidomenico4@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">

Has anyone seen an issue where OpenMPI/Infiniband hangs when sending<br>
messages over 2^26 in size?<br>
<br>
For a reason i have not determined just yet machines on my cluster<br>
(OpenMPI v1.5 and Qlogic Stack/QDR IB Adapters) is failing to send<br>
array&#39;s over 2^26 in size via the AllToAll collective. (user code)<br>
<br>
Further testing seems to indicate that an MPI message over 2^26 fails<br>
(tested with IMB-MPI)<br>
<br>
Running the same test on a different older IB connected cluster seems<br>
to work, which would seem to indicate a problem with the infiniband<br>
drivers of some sort rather then openmpi (but i&#39;m not sure).<br>
<br>
Any thoughts, directions, or tests?<br>
_______________________________________________<br>
users mailing list<br>
<a href="mailto:users@open-mpi.org">users@open-mpi.org</a><br>
<a href="http://www.open-mpi.org/mailman/listinfo.cgi/users" target="_blank">http://www.open-mpi.org/mailman/listinfo.cgi/users</a><br>
</blockquote></div><br><br clear="all"><br>-- <br>David Zhang<br>University of California, San Diego<br>