Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Scalability issue
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-12-07 11:54:46


On Dec 7, 2010, at 8:33 AM, Gus Correa wrote:

> Did I understand you right?
>
> Are you saying that one can effectively double the counting
> capability (i.e. the "count" parameters in MPI calls) of OpenMPI
> by compiling it with 8-byte integer flags?

Yes and no.

If you increase the size of INTEGER *and* int, then hypothetically yes -- although I literally just got a report from someone today that tried a compiler flag to increase the size of C int to 8 bytes and something didn't work right (I don't think we've ever tried this before, so it's not surprising that there are likely some bugs in there).

We have previously tested the increase-the-sizeof-INTEGER-to-8-bytes compiler flags and AFAIK, that's still working fine.

When you call MPI_SEND with an INTEGER count, OMPI will truncate it down to the size of a C int (if we had 8 byte C ints working, this might be a different story). But keep in mind that increasing the size of C ints will likely cause problems in other areas -- are OS system calls that take int parameters firmly sized (i.e., int32 and the like)? I'm not so sure -- indeed, that might even be (one of the) problem(s) with the report that I got earlier today...

> And long as one consistently uses the same flags to compile
> the application, everything would work smoothly?

I always recommend using the same flags for compiling OMPI as compiling your application. Of course, you can vary some flags that don't matter (e.g., compiling your app with -g and compiling OMPI with -Ox). But for "significant" behavior changes (like changing the size of INTEGER), they should definitely match between your app and OMPI.

> As per several previous discussions here in the list,
> I was persuaded to believe that MPI_INT / MPI_INTEGER is written
> in stone to be 4-bytes (perhaps by MPI standard,
> perhaps the configure script, maybe by both),

Neither, actually. :-)

The MPI spec is very, very careful not to mandate the size of int or INTEGER at all.

> and that "counts" in [Open]MPI would also be restricted to that size
> i.e., effectively up to 2147483647, if I counted right.

*Most* commodity systems (excluding the embedded world) have 4 byte int's these days, in part because most systems are this way (i.e., momentum).
Hence, when we talk about the 2B count limit, we're referring to the fact that most systems where MPI is used default to 4 byte int's.

> I may have inadvertently misled Benjamin, if this perception is wrong.
> I will gladly stand corrected, if this is so.
>
> You are the OpenMPI user's oracle (oops, sorry Cisco),
> so please speak out.

Please buy Cisco stuff! :-p

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/