Do you know if is there another patch available so my application treats the fail of one node instead of mpi kill the job? This is very important for me, I have a big cluster and I can&#39;t stop my job every time I have some problem with just one node.<div>
<br></div><div>Regards<br><br><div class="gmail_quote">On Fri, Sep 23, 2011 at 4:34 PM, Ralph Castain <span dir="ltr">&lt;<a href="mailto:rhc@open-mpi.org">rhc@open-mpi.org</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div style="word-wrap:break-word"><div class="im"><div>On Sep 23, 2011, at 1:21 PM, Guilherme V wrote:</div></div><div><div><div class="im"><br><blockquote type="cite"><p><span>I&#39;m using version 1.4.3 and I forgot to tell that I have made a change in the orterun.c line 792:</span></p>
<p><span>    if (ORTE_JOB_STATE_TERMINATED != exit_state) {<br>                    exit(0); /* patch*/<br></span></p></blockquote><div><br></div></div>I don&#39;t see how that change can keep your job running - we should still have terminated it. All this does is suppress the error reporting.</div>
<div><br></div><div><div>Regardless, openib will cause the process to fail under the described circumstances, which should cause OMPI to terminate all running procs. I&#39;m not sure what you are doing with tcp, but it could be that there are alternative paths available - e.g., you have multiple NICs and remove one cable, but the other paths remain viable.</div>
<div><br><div></div></div><blockquote type="cite"><div class="im"><p><span>
</span></p><p>Regards<br><span></span></p><p><span><br></span></p><p><span>&gt;</span>
What version of OMPI are you using? The job should terminate in either 
case - what did you do to keep it running after node failure with tcp?
<br>
</p><p><span>&gt;</span>On Sep 23, 2011, at 12:34 PM, Guilherme V wrote:
<br>
</p><span>&gt;&gt; Hi,
</span><br>
<span>&gt;</span><span>&gt; I want to know if anybody is having 
problems with fault tolerant job using infiniband. When I run my job 
with tcp if anything happens with one node, my job keeps running, but if
 I change my job to use infiniband if anything happens with the 
infiniband (i.e cable problems) my job fails.
</span><br>
<span>&gt;</span><span>&gt; 
</span><br>
<span>&gt;</span><span>&gt; Anybody knows if there is something different that need to be done when using openib instead tcp?  
</span><br>
<span>&gt;</span><span>&gt;</span><span> 
</span><br>
<span>&gt;</span><span>&gt; Bellow a example of the message I&#39;m receiving from the mpi.
</span><br>
<span>&gt;</span><span>&gt; 
</span><br>
<span>&gt;</span><span>&gt; Regards,
</span><br>
<span>&gt;&gt; Guilherme
</span><br>
<span></span><br>
<span><br></span><span><a href="http://www.open-mpi.org/mailman/listinfo.cgi/users" target="_blank"></a>
</span></div>
_______________________________________________<br>users mailing list<br><a href="mailto:users@open-mpi.org" target="_blank">users@open-mpi.org</a><br><a href="http://www.open-mpi.org/mailman/listinfo.cgi/users" target="_blank">http://www.open-mpi.org/mailman/listinfo.cgi/users</a></blockquote>
</div><br></div></div><br>_______________________________________________<br>
users mailing list<br>
<a href="mailto:users@open-mpi.org">users@open-mpi.org</a><br>
<a href="http://www.open-mpi.org/mailman/listinfo.cgi/users" target="_blank">http://www.open-mpi.org/mailman/listinfo.cgi/users</a><br></blockquote></div><br></div>