Dear OpenMPI users,<div><br></div><div>i&#39;m using OpenMPI 1.3.3 on Infiniband 4x interconnnection network. My parallel application use intensive MPI_Reduce communication over communicator created with MPI_Comm_split.</div>
<div><br></div><div>I&#39;ve noted strange behaviour during execution. My code is instrumented with Scalasca 1.3 to report subroutine execution time. First execution shows elapsed time with 128 processors ( job_communicator is created with MPI_Comm_split). In both cases is composed to the same ranks of MPI_COMM_WORLD:</div>
<div><br></div><div>MPI_Reduce(.....,job_communicator)</div><div><br></div><div>The elapsed time is 2671 sec.</div><div><br></div><div>Second run use MPI_BARRIER before MPI_Reduce:</div><div><br></div><div>MPI_Barrier(job_communicator..)</div>
<div>MPI_Reduce(.....,job_communicator)</div><div><br></div><div>The elapsed time of Barrier+Reduce is 2167 sec, (about 8 minutes less). </div><div><br></div><div>So, im my opinion, it is better put MPI_Barrier before any MPI_Reduce to mitigate &quot;asynchronous&quot; behaviour of MPI_Reduce in OpenMPI. I suspect the same for others collective communications. Someone can explaine me why MPI_reduce has this strange behaviour?</div>
<meta http-equiv="content-type" content="text/html; charset=utf-8"><div><br></div><div>Thanks in advance.</div><div><br></div><div><br></div><div><br></div><meta http-equiv="content-type" content="text/html; charset=utf-8"><div>
<br>-- <br>Ing. Gabriele Fatigati<br><br>Parallel programmer<br><br>CINECA Systems &amp; Tecnologies Department<br><br>Supercomputing Group<br><br>Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy<br><br><a href="http://www.cineca.it">www.cineca.it</a>                    Tel:   +39 051 6171722<br>
<br>g.fatigati [AT] <a href="http://cineca.it">cineca.it</a>           <br>
</div>