On Thu, Nov 26, 2009 at 7:07 PM, Chuck Ritter <cfr100@psu.edu> wrote:

On Thu, Nov 26, 2009 at 3:16 AM, Vivek Satpute <vivekonline86@gmail.com> wrote:

>> > My understanding/assumption is, Infiniband also work without using
>> > IPoIB. So, I want to find out application/utility which work without
>> > IPoIB module. All the traffic generated by such xyz application
>> > should go to remote machine without IPoIB so for achieving this I can
>> > use LID/GID of IB devices.
>>
>> These programs are written with Infiniband in mind and do not use
>> sockets but instead the ibverbs API.
>>
> Exactly.. I am looking for these sample programs.
>

Look for ibstat and ibhosts. They should be in the OFED packages somewhere.

Linpack is also widely available and frequently used.

> Ok. OFED-1.4 has few sample programs in libibverbs-1.1.2 package. But
> all those programs expect IP address of remote machine. So those are not
> I am looking for as they will pass through IPoIB module. Could you please
> point me to sample applications/programs which uses purely ibverbs APIs ?
>
> And why those examples in libibverbs-1.1.2 package takes IP address ?
> Do you have any idea about that ?

Why not?

I think hostnames and ips were used because they uniquely identify a
node. Why create a parallel naming scheme? It just adds more data to
track.

I agree with this that why there should be parallel naming scheme. But
Infiniband devices already have LID and GID naming system to identify devices
So instead of using IP address to uniquely identify, we can use those LID/GID.
Am I right ?

Basically, some programs are both IB and IP capable. The interface is
a choice. IB/iverb capable application look for /etc/mv2.conf for
interface definition. Most will quietly fallback to ethernet if they
cannot detect IB devices (nice feature ehh. when in doubt just run).

Some mpi implementations allow the user to specify the interface:

mpiexec -np n -use-interface-something-something ib
.-no-fallback-or-something /prog

Then the application will print error messages instead of using IP. I
don't recall if mvapich is one of these.

>> > My this assumption can be wrong so please correct me if I am wrong ..

You are wrong. Read the documentation. :-)

By the way... IPoIB can be quite useful for applications that don't
support IB directly. Some filesystems come to mind.

Happy Thanksgiving.