Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Infiniband Question
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-02-05 17:40:00


Yep -- it's normal.

Those IP addresses are used for bootstrapping/startup, not for MPI traffic. In particular, that "HNP URI" stuff is used by Open MPI's underlying run-time environment. It's not used by the MPI layer at all.

On Feb 5, 2010, at 2:32 PM, Mike Hanby wrote:

> Howdy,
>
> When running a Gromacs job using OpenMPI 1.4.1 on Infiniband enabled nodes, I'm seeing the following process listing:
>
> \_ -bash /opt/gridengine/default/spool/compute-0-3/job_scripts/97037
> \_ mpirun -np 4 mdrun_mpi -v -np 4 -s production-Npt-323K_4CPU -o production-Npt-323K_4CPU -c production-Npt-323K_4CPU -x production-Npt-323K_4CPU -g production-Npt-323K_4CPU.log
> \_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V compute-0-4.local orted -mca ess env -mca orte_ess_jobid 945881088
> -mca orte_ess_vpid 1 -mca orte_ess_num_procs 4 --hnp-uri "945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
> \_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V compute-0-2.local orted -mca ess env -mca orte_ess_jobid 945881088
> -mca orte_ess_vpid 2 -mca orte_ess_num_procs 4 --hnp-uri "945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
> \_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V compute-0-1.local orted -mca ess env -mca orte_ess_jobid 945881088
> -mca orte_ess_vpid 3 -mca orte_ess_num_procs 4 --hnp-uri "945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
> \_ mdrun_mpi -v -np 4 -s production-Npt-323K_4CPU -o production-Npt-323K_4CPU -c production-Npt-323K_4CPU
> -x production-Npt-323K_4CPU -g production-Npt-323K_4CPU.log
>
> Is it normal for these tcp addresses to be listed if the job is using Infiniband?
>
> The 192.168.20.x subnet is the eth0 GigE network
> And the 192.168.21.x subnet is the ib0 IPoverIB network
>
> Or is this job actually using TCPIP over Infiniband / GigE?
>
> I'm running mpirun without any special fabric includes / excludes.
>
> ompi_info lists openib as a valid fabric:
> $ ompi_info |grep openib
> MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.1)
>
> Thanks for any insight,
>
> Mike
> =================================
> Mike Hanby
> mhanby_at_[hidden]
> Information Systems Specialist II
> IT HPCS / Research Computing
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/