a) Yes, I know I could use only IB by "--mca btl openib", but just
want to make sure I am using IB interfaces. I am seeking an option
to mpirun to print out the actual interconnect protocol, like --prot to
mpirun in MPICH2.
b) Yes, my default shell is bash, but I run a c-shell script from bash
terminal, mpirun is invoked inside this c-shell script. I am using rsh
launcher, exactly as your guess. I try different mpirun command in
the c-shell, one of them is
/path/to/bin/mpirun --mca btl openib --app appfile
and mpirun and orted are under /path/to/bin, and necessary libs are
under /path/to/lib. I tried the -x, --prefix, and -path, all does not work
as expected to propagate the PATH and LD_LIBRARY_PATH,
since orted is not found on slave nodes, although it shoud since it
on the shared NFS partition.
On Jun 28, 2011, at 9:05 AM, yanyg_at_[hidden] wrote:
> Hello All,
> I installed Open MPI 1.4.3 on our new HPC blades, with Infiniband
> My system environments are as:
> 1)uname -a output:
> Linux gulftown 2.6.18-194.el5 #1 SMP Tue Mar 16 21:52:39 EDT
> 2010 x86_64 x86_64 x86_64 GNU/Linux
> 2) /home is mounted over all nodes, and mpirun is started under
> Open MPI and application codes are compiled with intel(R)
> compilers V11. Infiniband stack is Mellanox OFED 1.5.2.
> I have two questions about mpirun:
> a) how could I get to know what is the network interconnect
> protocol used by the MPI application?
> I specify "--mca btl openib,self,sm,tcp" to mpirun, but I want to
> make sure it really uses infiniband interconnect.
Why specify tcp if you don't want it used? Just leave that off and it
will have no choice but to use IB.
> b) when I run mpirun, I get the following message:
> It seems orted is not found on slave nodes. If I set the PATH and
> LD_LIBRARY_PATH through --prefix to mpirun, or --path, or -x
> options to mpirun, to make the orted and related dynamic libs
> available on slave nodes, it does not work as expected from
> manual page. The only working case is that I set PATH and
> LD_LIBRARY_PATH in ~/.bashrc for mpirun, and this .bashrc is
> invoked by slave nodes too for login shell. I do not want to set
> and LD_LIBRARY_PATH in ~/.bashrc, but instead to set options
> mpirun directly.
Should work with either prefix or -x options, assuming the right
syntax with the latter.
I take it your default shell is bash, and that you are using the rsh
launcher (as opposed to something like torque)? Are you launching
from your default shell, or did you perhaps change shell?
Can you send the actual mpirun command you typed?