Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] importing to MPI data already in memory from another process
From: George Bosilca (bosilca_at_[hidden])
Date: 2014-06-27 11:35:32


MPI "universe" yes, but not necessarily MPI "world". You could have
the two worlds connect/accept or join
(https://www.open-mpi.org/doc/v1.8/man3/MPI_Comm_join.3.php) and then
you should be able to take advantage of the RMA. At least, that is
what is written in the book ...

  George.

On Fri, Jun 27, 2014 at 10:30 AM, Brock Palen <brockp_at_[hidden]> wrote:
> But this is within the same MPI "universe" right?
>
> Brock Palen
> www.umich.edu/~brockp
> CAEN Advanced Computing
> XSEDE Campus Champion
> brockp_at_[hidden]
> (734)936-1985
>
>
>
> On Jun 27, 2014, at 10:19 AM, George Bosilca <bosilca_at_[hidden]> wrote:
>
>> The One-Sided Communications from the Chapter 11 of the MPI standard?
>> For processes on the same node you might want to look at
>> MPI_WIN_ALLOCATE_SHARED.
>>
>> George.
>>
>> On Fri, Jun 27, 2014 at 9:53 AM, Brock Palen <brockp_at_[hidden]> wrote:
>>> Is there a way to import/map memory from a process (data acquisition) such that an MPI program could 'take' or see that memory?
>>>
>>> We have a need to do data acquisition at the rate of .7TB/s and need todo some shuffles/computation on these data, some of the nodes are directly connected to the device, and some will do processing.
>>>
>>> Here is the proposed flow:
>>>
>>> * Data collector nodes runs process collecting data from device
>>> * Those nodes somehow pass the data to an MPI job running on these nodes and a number of other nodes (cpu need for filterting is greater than what the 16 data nodes can provide).
>>>
>>> One thought is to have the data collector processes be threads inside the MPI job running across all nodes, but was curious is there is a way to pass data still in memory (to much to hit disk) to the running MPI filter job.
>>>
>>> Thanks!
>>>
>>> Brock Palen
>>> www.umich.edu/~brockp
>>> CAEN Advanced Computing
>>> XSEDE Campus Champion
>>> brockp_at_[hidden]
>>> (734)936-1985
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24716.php
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
>> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24717.php
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24718.php