Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Using MPI derived datatypes
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-08-07 15:18:09

On Aug 3, 2012, at 7:36 AM, Grzegorz Maj wrote:

> I would like my MPI processes to exchange some structural data. That
> data is represented by plain structures containing basic datatypes. I
> would like to use MPI derived datatypes, because of its portability
> and good performance.
> I would like to be able to send/receive any of my structures in the
> same part of code. In the low-level network programming it is usually
> done by having each struct of this pattern:
> struct S1 {
> int structType;
> ...
> }
> And then you first read structType and know what bytes to expect next.
> Is there a good way to do it using MPI derived datatypes?

No. MPI is about atomic messages -- not streams. So if you're going to send a struct, you send the whole struct. You could choose to send the first int/enum and then do a second send with the rest of the struct, but that's a different pattern.

Alternatively, you might want to explore using that first int/enum as the tag in your MPI_SEND/MPI_RECV. I.e., you can use the tag to indicate the type of data to receive.

> I was thinking of having separate MPI_Request for each of my
> structures and calling multiple MPI_Irecv + MPI_Waitany.

Yes, that would probably work fine (i.e., one MPI_Irecv for each tag/struct type).

> But then, how
> to do this for MPI_Bcast?

You can't currently do this with MPI_Bcast -- OMPI currently only has blocking versions of the collectives. MPI-3.0 defines non-blocking collectives (e.g., MPI_Ibcast and friends), but they still have no tags (for very complicated reasons :-) ). You might need a different communicators to be able to post a non-blocking, receiving MPI_Ibcast for different struct types.

These non-blocking collectives will be included in the upcoming Open MPI v1.7 -- probably in a few months.

> My second question is about having arbitrary size structures, i.e. the
> ones having 'char buf[0]' as the last field, where you allocate memory
> of size 'sizeof(S) + bufLen'. Is there a way to convert such a struct
> into MPI derived datatype?

Not really. MPI datatypes represent fixed sizes. That being said, if your last data member is "large enough", you could just send that as a blob in a 2nd send, and latency difference wouldn't matter (i.e., the blob is large enough such that the overhead of a 2nd send is dwarfed by the transmission / receipt time).

Alternatively, if you're looking to send blobs of unknown sizes without associated meta data, you might want to look at MPI_MPROBE. MPROBE was specifically introduced in MPI-3.0 for, among other reasons, receiving messages of unknown size. See for an explanation of MPROBE.

MPROBE, too, will be included in the upcoming OMPI v1.7 release.

Jeff Squyres
For corporate legal information go to: