Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] importing to MPI data already in memory from another process
From: Brock Palen (brockp_at_[hidden])
Date: 2014-06-27 10:30:05


But this is within the same MPI "universe" right?

Brock Palen
www.umich.edu/~brockp
CAEN Advanced Computing
XSEDE Campus Champion
brockp_at_[hidden]
(734)936-1985

On Jun 27, 2014, at 10:19 AM, George Bosilca <bosilca_at_[hidden]> wrote:

> The One-Sided Communications from the Chapter 11 of the MPI standard?
> For processes on the same node you might want to look at
> MPI_WIN_ALLOCATE_SHARED.
>
> George.
>
> On Fri, Jun 27, 2014 at 9:53 AM, Brock Palen <brockp_at_[hidden]> wrote:
>> Is there a way to import/map memory from a process (data acquisition) such that an MPI program could 'take' or see that memory?
>>
>> We have a need to do data acquisition at the rate of .7TB/s and need todo some shuffles/computation on these data, some of the nodes are directly connected to the device, and some will do processing.
>>
>> Here is the proposed flow:
>>
>> * Data collector nodes runs process collecting data from device
>> * Those nodes somehow pass the data to an MPI job running on these nodes and a number of other nodes (cpu need for filterting is greater than what the 16 data nodes can provide).
>>
>> One thought is to have the data collector processes be threads inside the MPI job running across all nodes, but was curious is there is a way to pass data still in memory (to much to hit disk) to the running MPI filter job.
>>
>> Thanks!
>>
>> Brock Palen
>> www.umich.edu/~brockp
>> CAEN Advanced Computing
>> XSEDE Campus Champion
>> brockp_at_[hidden]
>> (734)936-1985
>>
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24716.php
> _______________________________________________
> users mailing list
> users_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24717.php