Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

From: Sridhar Chirravuri (sridhar_at_[hidden])
Date: 2005-08-09 08:48:56


Does r6774 has lot of changes that are related to 3rd generation point-to-point? I am trying to run some benchmark tests (ex: pallas) with Open MPI stack and just want to compare the performance figures with MVAPICH 095 and MVAPICH 092.

In order to use 3rd generation p2p communication, I have added the following line in the /openmpi/etc/openmpi-mca-params.conf


I also exported (as double check) OMPI_MCA_pml=ob1.

Then, I have tried running on the same machine. My machine has got 2 processors.

Mpirun -np 2 ./PMB-MPI1

I still see the following lines

Request for 0 bytes (coll_basic_reduce_scatter.c, 79)
Request for 0 bytes (coll_basic_reduce.c, 193)
Request for 0 bytes (coll_basic_reduce_scatter.c, 79)
Request for 0 bytes (coll_basic_reduce.c, 193)

In the output of ompi_info, I could see that ob1 component is being built.

Do I have to configure any other stuff? Please let me know


-----Original Message-----
From: devel-bounces_at_[hidden] [mailto:devel-bounces_at_[hidden]] On Behalf Of Jeff Squyres
Sent: Tuesday, August 09, 2005 6:35 PM
To: Open MPI Developers
Subject: Re: [O-MPI devel] Fwd: Regarding MVAPI Component in Open MPI

On Aug 9, 2005, at 7:24 AM, Sridhar Chirravuri wrote:

> I have fixed the timing issue between the server and client, and now I
> could build Open MPI successfully.


> Here is the output of ompi_info....
> [root_at_micrompi-2 ompi]# ompi_info
>                 Open MPI: 1.0a1r6760M

Note that as of this morning (US Eastern time), the current head is
r6774. Also be wary of any local mods you have put in the tree (as
noted by the "M"). Check "svn status" to see which files you have
modified, and "svn diff" to see the exact changes.

> This time, I could see that btl mvapi component is built.
> But I am still seeing the same problem while running Pallas Benchmark
> i.e., I still see that the data is passing over TCP/GigE and NOT over
> Infiniband.

Please note that the 2nd generation point-to-point implementation is
still the default (where we have no IB support) -- all the IB support,
both mVAPI and Open IB, is in the 3rd generation point-to-point
implementation. You must explicitly request the 3rd generation
point-to-point implementation at run time to get IB support. Check out
slide 48, "Example: Forcing ob1/BTL" in the slides that we discussed on
the teleconference (were you on the teleconference? I attached copies
if you were not). The short version is that you need to tell Open MPI
to use the "ob1" pml component (3rd gen), not the default "teg" pml
component (2nd gen).

We'll eventually make the 3rd gen stuff be the default, and likely
remove all the 2nd gen stuff (i.e., definitely before release) -- we
just haven't done it yet because Tim and Galen are still polishing up
the 3rd gen stuff.

> I have disabled building OpenIB and to do so I have touched
> .ompi_ignore. This should not be a problem for MVAPI.

If the Open IB headers / libraries are not located in compiler-known
locations, then you shouldn't need to .ompi_ignore the tree (i.e.,
configure won't find the Open IB headers / libraries, and will
therefore automatically skip those components).

Again, it is our intention that users will neither know about nor have
to touch files in the distribution -- they only need use appropriate
options to "configure" and then "make".

I'm not sure if we have explicit options to disable a component in
configure -- Brian, can you comment here?

> I have run, configure and make all. The output of
>, configure and make all commands are <<ompi_out.tar.gz>>
> gzip'ed in ompi_out.tar.gz file which is attached in this mail. This
> gzip file also contains the output of Pallas Benchmark results. At the
> end of Pallas Benchmark output, you can find the error
> Request for 0 bytes (coll_basic_reduce.c, 193)
> Request for 0 bytes (coll_basic_reduce.c, 193)
> Request for 0 bytes (coll_basic_reduce.c, 193)
> Request for 0 bytes (coll_basic_reduce.c, 193)
> Request for 0 bytes (coll_basic_reduce.c, 193)
> Request for 0 bytes (coll_basic_reduce_scatter.c, 79)
> Request for 0 bytes (coll_basic_reduce.c, 193)
> Request for 0 bytes (coll_basic_reduce_scatter.c, 79)
> Request for 0 bytes (coll_basic_reduce.c, 193)
> ..and Pallas just hung.
> I have no clue about the above errors which are coming from Open MPI
> source code.

The 2nd generation component has fallen into some disrepair -- I'd try
re-running with ob1 and see what happens. I have not seen such errors
when running PMB before, but I can try running it again to see if we've
broken something recently.

> Is there any thing that I am missing while building btl mvapi? Also,
> is anyone built for mvapi and tested this OMPI stack. Please let me
> know.

Galen Shipman and Tim Woodall are doing all the IB work.

{+} Jeff Squyres
{+} The Open MPI Project
devel mailing list