Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] importing to MPI data already in memory from another process
From: George Bosilca (bosilca_at_[hidden])
Date: 2014-06-27 10:19:13


The One-Sided Communications from the Chapter 11 of the MPI standard?
For processes on the same node you might want to look at
MPI_WIN_ALLOCATE_SHARED.

  George.

On Fri, Jun 27, 2014 at 9:53 AM, Brock Palen <brockp_at_[hidden]> wrote:
> Is there a way to import/map memory from a process (data acquisition) such that an MPI program could 'take' or see that memory?
>
> We have a need to do data acquisition at the rate of .7TB/s and need todo some shuffles/computation on these data, some of the nodes are directly connected to the device, and some will do processing.
>
> Here is the proposed flow:
>
> * Data collector nodes runs process collecting data from device
> * Those nodes somehow pass the data to an MPI job running on these nodes and a number of other nodes (cpu need for filterting is greater than what the 16 data nodes can provide).
>
> One thought is to have the data collector processes be threads inside the MPI job running across all nodes, but was curious is there is a way to pass data still in memory (to much to hit disk) to the running MPI filter job.
>
> Thanks!
>
> Brock Palen
> www.umich.edu/~brockp
> CAEN Advanced Computing
> XSEDE Campus Champion
> brockp_at_[hidden]
> (734)936-1985
>
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24716.php