Hi Josh,<div>Thanks for the reply. I did not use the &#39;--enable-ft-thread&#39; option. Here is my build options:</div><div><br><div>CFLAGS=-g \</div><div>./configure \</div><div>--with-ft=cr \</div><div>--enable-mpi-threads \</div>
<div>--with-blcr=/home/nguyen/opt/blcr \</div><div>--with-blcr-libdir=/home/nguyen/opt/blcr/lib \</div><div>--prefix=/home/nguyen/opt/openmpi \</div><div>--with-openib \</div><div>--enable-mpirun-prefix-by-default</div><div>
<br></div><div>My application requires lots of communication in every loop, focusing on MPI_Isend, MPI_Irecv and MPI_Wait. Also I want to make only one checkpoint per application execution for my purpose, but the unknown overhead exists even when no checkpoint was taken.</div>
<div><br></div><div>Do you have any other idea?</div><div><br></div><div>Regards,</div><div>Nguyen Toan</div><div><br></div><div><br></div><div class="gmail_quote">On Wed, Feb 9, 2011 at 12:41 AM, Joshua Hursey <span dir="ltr">&lt;<a href="mailto:jjhursey@open-mpi.org">jjhursey@open-mpi.org</a>&gt;</span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">There are a few reasons why this might be occurring. Did you build with the &#39;--enable-ft-thread&#39; option?<br>
<br>
If so, it looks like I didn&#39;t move over the thread_sleep_wait adjustment from the trunk - the thread was being a bit too aggressive. Try adding the following to your command line options, and see if it changes the performance.<br>

  &quot;-mca opal_cr_thread_sleep_wait 1000&quot;<br>
<br>
There are other places to look as well depending on how frequently your application communicates, how often you checkpoint, process layout, ... But usually the aggressive nature of the thread is the main problem.<br>
<br>
Let me know if that helps.<br>
<br>
-- Josh<br>
<div><div></div><div class="h5"><br>
On Feb 8, 2011, at 2:50 AM, Nguyen Toan wrote:<br>
<br>
&gt; Hi all,<br>
&gt;<br>
&gt; I am using the latest version of OpenMPI (1.5.1) and BLCR (0.8.2).<br>
&gt; I found that when running an application,which uses MPI_Isend, MPI_Irecv and MPI_Wait,<br>
&gt; enabling C/R, i.e using &quot;-am ft-enable-cr&quot;, the application runtime is much longer than the normal execution with mpirun (no checkpoint was taken).<br>
&gt; This overhead becomes larger when the normal execution runtime is longer.<br>
&gt; Does anybody have any idea about this overhead, and how to eliminate it?<br>
&gt; Thanks.<br>
&gt;<br>
&gt; Regards,<br>
&gt; Nguyen<br>
</div></div>&gt; _______________________________________________<br>
&gt; users mailing list<br>
&gt; <a href="mailto:users@open-mpi.org">users@open-mpi.org</a><br>
&gt; <a href="http://www.open-mpi.org/mailman/listinfo.cgi/users" target="_blank">http://www.open-mpi.org/mailman/listinfo.cgi/users</a><br>
<br>
------------------------------------<br>
Joshua Hursey<br>
Postdoctoral Research Associate<br>
Oak Ridge National Laboratory<br>
<a href="http://users.nccs.gov/~jjhursey" target="_blank">http://users.nccs.gov/~jjhursey</a><br>
<br>
<br>
_______________________________________________<br>
users mailing list<br>
<a href="mailto:users@open-mpi.org">users@open-mpi.org</a><br>
<a href="http://www.open-mpi.org/mailman/listinfo.cgi/users" target="_blank">http://www.open-mpi.org/mailman/listinfo.cgi/users</a><br>
</blockquote></div><br></div>