Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Question about senting short MPI messages using RDMA
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2010-01-04 17:33:17


It's complicated, but the short answer is that a "short" message is defined as one where the cost of a memory copy doesn't matter.

You could always use mpi_alloc_mem to get registered memory. But I don't recall offhand if we check to see if the memory is already registered for short messages (because the cost of the memcpy is insignificant).

Yes, the cost of mem reg is one of the main drivers for these kinds of optimizations.

-jms
Sent from my PDA. No type good.

----- Original Message -----
From: users-bounces_at_[hidden] <users-bounces_at_[hidden]>
To: users_at_[hidden] <users_at_[hidden]>
Sent: Mon Jan 04 11:09:10 2010
Subject: [OMPI users] Question about senting short MPI messages using RDMA

Hi, guys.
As I understand, to send short MPI messages, OpenMPI copies the
messages to preallocated buffer and then uses RDMA.

I was wondering if we can avoid the overhead of memory copy. If the
user buffers for short messages are reused a lot, we can just register
the user buffer instead of using preallocated buffer. Then we can do
RDMA directly from the user buffer instead of the preallocated buffer.
But if the user buffers are not reused, we will suffer from the
overhead of memory registration.

Besides the overhead of memory registration, is there any other reason
that prevent you to do RDMA directly from the user buffer for short
messages?

Thank you.
_______________________________________________
users mailing list
users_at_[hidden]
http://www.open-mpi.org/mailman/listinfo.cgi/users