Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Using MPI derived datatypes
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-08-07 15:18:09


On Aug 3, 2012, at 7:36 AM, Grzegorz Maj wrote:

> I would like my MPI processes to exchange some structural data. That
> data is represented by plain structures containing basic datatypes. I
> would like to use MPI derived datatypes, because of its portability
> and good performance.
>
> I would like to be able to send/receive any of my structures in the
> same part of code. In the low-level network programming it is usually
> done by having each struct of this pattern:
> struct S1 {
> int structType;
> ...
> }
> And then you first read structType and know what bytes to expect next.
>
> Is there a good way to do it using MPI derived datatypes?

No. MPI is about atomic messages -- not streams. So if you're going to send a struct, you send the whole struct. You could choose to send the first int/enum and then do a second send with the rest of the struct, but that's a different pattern.

Alternatively, you might want to explore using that first int/enum as the tag in your MPI_SEND/MPI_RECV. I.e., you can use the tag to indicate the type of data to receive.

> I was thinking of having separate MPI_Request for each of my
> structures and calling multiple MPI_Irecv + MPI_Waitany.

Yes, that would probably work fine (i.e., one MPI_Irecv for each tag/struct type).

> But then, how
> to do this for MPI_Bcast?

You can't currently do this with MPI_Bcast -- OMPI currently only has blocking versions of the collectives. MPI-3.0 defines non-blocking collectives (e.g., MPI_Ibcast and friends), but they still have no tags (for very complicated reasons :-) ). You might need a different communicators to be able to post a non-blocking, receiving MPI_Ibcast for different struct types.

These non-blocking collectives will be included in the upcoming Open MPI v1.7 -- probably in a few months.

> My second question is about having arbitrary size structures, i.e. the
> ones having 'char buf[0]' as the last field, where you allocate memory
> of size 'sizeof(S) + bufLen'. Is there a way to convert such a struct
> into MPI derived datatype?

Not really. MPI datatypes represent fixed sizes. That being said, if your last data member is "large enough", you could just send that as a blob in a 2nd send, and latency difference wouldn't matter (i.e., the blob is large enough such that the overhead of a 2nd send is dwarfed by the transmission / receipt time).

Alternatively, if you're looking to send blobs of unknown sizes without associated meta data, you might want to look at MPI_MPROBE. MPROBE was specifically introduced in MPI-3.0 for, among other reasons, receiving messages of unknown size. See http://blogs.cisco.com/performance/mpi-forum-roundup/ for an explanation of MPROBE.

MPROBE, too, will be included in the upcoming OMPI v1.7 release.

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/