Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] Infiniband Question
From: Mike Hanby (mhanby_at_[hidden])
Date: 2010-02-05 14:32:54


Howdy,

When running a Gromacs job using OpenMPI 1.4.1 on Infiniband enabled nodes, I'm seeing the following process listing:

\_ -bash /opt/gridengine/default/spool/compute-0-3/job_scripts/97037
    \_ mpirun -np 4 mdrun_mpi -v -np 4 -s production-Npt-323K_4CPU -o production-Npt-323K_4CPU -c production-Npt-323K_4CPU -x production-Npt-323K_4CPU -g production-Npt-323K_4CPU.log
        \_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V compute-0-4.local orted -mca ess env -mca orte_ess_jobid 945881088
            -mca orte_ess_vpid 1 -mca orte_ess_num_procs 4 --hnp-uri "945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
        \_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V compute-0-2.local orted -mca ess env -mca orte_ess_jobid 945881088
            -mca orte_ess_vpid 2 -mca orte_ess_num_procs 4 --hnp-uri "945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
        \_ /opt/gridengine/bin/lx26-amd64/qrsh -inherit -nostdin -V compute-0-1.local orted -mca ess env -mca orte_ess_jobid 945881088
            -mca orte_ess_vpid 3 -mca orte_ess_num_procs 4 --hnp-uri "945881088.0;tcp://192.168.20.252:39440;tcp://192.168.21.252:39440"
        \_ mdrun_mpi -v -np 4 -s production-Npt-323K_4CPU -o production-Npt-323K_4CPU -c production-Npt-323K_4CPU
            -x production-Npt-323K_4CPU -g production-Npt-323K_4CPU.log

Is it normal for these tcp addresses to be listed if the job is using Infiniband?

The 192.168.20.x subnet is the eth0 GigE network
And the 192.168.21.x subnet is the ib0 IPoverIB network

Or is this job actually using TCPIP over Infiniband / GigE?

I'm running mpirun without any special fabric includes / excludes.

ompi_info lists openib as a valid fabric:
$ ompi_info |grep openib
                 MCA btl: openib (MCA v2.0, API v2.0, Component v1.4.1)

Thanks for any insight,

Mike
=================================
Mike Hanby
mhanby_at_[hidden]
Information Systems Specialist II
IT HPCS / Research Computing