> If you run your cmd with the hostfile option and add
> --display-allocation, what does it say?
Thank you, Ralph.
This is the command I used inside my submission script:
mpirun --display-allocation -np 4 -hostfile hosts ./program
And this is the output I got.
Data for node: Name: node03 Num slots: 4 Max slots: 0
Data for node: Name: node02 Num slots: 4 Max slots: 0
Data for node: Name: node04 Num slots: 4 Max slots: 0
Data for node: Name: node01 Num slots: 4 Max slots: 0
If I run the same mpirun command on the cluster head node "clhead" then
this is what I get:
Data for node: Name: clhead Num slots: 0 Max slots: 0
Data for node: Name: node01 Num slots: 1 Max slots: 0
Data for node: Name: node02 Num slots: 1 Max slots: 0
Data for node: Name: node03 Num slots: 1 Max slots: 0
Data for node: Name: node04 Num slots: 1 Max slots: 0
The content of the 'hosts' file:
node01 slots=1
node02 slots=1
node03 slots=1
node04 slots=1
= Serge
On Apr 6, 2010, at 12:18 PM, Serge wrote:
> Hi,
>
> OpenMPI integrates with Sun Grid Engine really well, and one does not
> need to specify any parameters for the mpirun command to launch the
> processes on the compute nodes, that is having in the submission script
> "mpirun ./program" is enough; there is no need for "-np XX" or
> "-hostfile file_name".
>
> However, there are cases when being able to specify the hostfile is
> important (hybrid jobs, users with MPICH jobs, etc.). For example, with
> Grid Engine I can request four 4-core nodes, that is total of 16 slots.
> But I also want to specify how to distribute processes on the nodes, so
> I create the file 'hosts'
>
> node01 slots=1
> node02 slots=1
> node03 slots=1
> node04 slots=1
>
> and modify the line in the submission script to:
> mpirun -hostfile hosts ./program
>
> With Open MPI 1.2.x everything worked properly, meaning that Open MPI
> could count the number of slots specified in the 'hosts' file - 4 (i.e.
> effectively supplying the mpirun command with the -np parameter), as
> well as properly distribute processes on the compute nodes (one process
> per host).
>
> It's different with Open MPI 1.4.1. It cannot process the 'hosts' file
> properly at all. All the processes get launched on just one node -- the
> shepherd host.
>
> The format of the 'hosts' file does not matter. It can be, say
>
> node01
> node01
> node02
> node02
>
> meaning 2 slots on each node. Open MPI 1.2.x would handle that with no
> problem, however Open MPI 1.4.x would not.
>
> The problem appears with OMPI 1.4.1, SGE 6.1u6. It was also tested with
> OMPI 1.3.4 and SGE 6.2u4.
>
> It's important to notice that if the mpirun command is run
> interactively, not from inside the Grid Engine script, then it
> interprets the content of the host file just fine.
>
> I am wondering what changed from OMPI 1.2.x to OMPI 1.4.x that prevents
> expected behavior, and is it possible to get it from OMPI 1.4.x by, say,
> tuning some parameters?
>
> = Serge
>
|
![[Flag]](../../../../../community/mirrors/flag-us.gif){kind=small}