I actually tried both: -- in the interactive mode, as soon as I hit enter, the PBS sends me to a worker node (WN2) that does not have tm support. I guess if I added the head node to the list of PBS nodes, I would not run into the problem. However, I am glad I did run into the problem. You guys helped me understand how mpirun interacts with PBS. -- in the batch mode, the PBS chooses WN2 as the root node (rank 0) to run mpirun. So, Both behaved in the same in my case.
With many thanks Ralph for the clarification. I am now getting into the openMPI spirit. By the way, I love it :)
From: firstname.lastname@example.org Date: Tue, 1 Dec 2009 19:44:38 -0700 To: email@example.com Subject: Re: [OMPI users] mpirun is using one PBS node only
Just to further show my confusion (since I wrote much of the TM support):
If you get an interactive allocation and then type "mpirun ....", mpirun will execute on the node upon which you are sitting. Jeff's statement is -only- true if you "qsub" the job - i.e., you run it in batch mode.
From your prior notes, I was under the impression you were running interactively. If this is true, then you only need OMPI to be built with TM support on the node which you are interactively working on.
If you are running it in batch mode, then Jeff is correct - you will need OMPI built with TM support on all nodes since you (in general) cannot know which node PBS will launch you on.
Hope that is clear
On Dec 1, 2009, at 7:35 PM, Belaid MOA wrote:
> Yes, this page is definitely incorrect if you want to run with PBS/TM > support -- you definitely need to install with TM support on all nodes. > > The reason is that PBS will launch your script (and therefore > "mpirun") on the first node of the job. This node must have an Open > MPI mpirun with TM support so that it can query PBS to get the nodes > to launch on, use the TM API to launch processes on the nodes > allocated to the job, etc. That's exactly the sound logic I understood from OpenMPI documentation but I thought the website is reliable in the sense it configured the server node with tm support.
I am very grateful to subscribe to this list. It is one of the most active list I ever saw.