Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] mpi_leave_pinned is dangerous
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-11-08 09:55:25


Nope, that wasn't it.

...oh, I see, Linus' reply didn't go to LKML; it just went to a bunch of individuals. Here's part of his reply:

----
The interface claims to be generic, but is really just a hack for a single
use case that very few people care about. I find the design depressingly
stupid, even if the code itself is at least small and simple.
Last update I got was that people who can't even agree on some other
interface but still are otherwise doing the same RDMA crap want to then
use this crud too - but no explanation of why they couldn't agree on the
other interface.
----
Here's another relevant bit:
-----
Can't you crazy RDMA people just agree on an RDMA interface, and making it
part of that? It still makes _zero_ sense outside of that small niche as
far as I can tell.
-----
...it goes downhill from there.  :-)
On Nov 8, 2012, at 9:48 AM, Brice Goglin wrote:
> My understanding of the upstreaming failure was more like:
> * Linus was going to be OK
> * Some perf (or trace?) guys came late and said "oh your code should be
> integrated into our more general stuff" but they didn't do it, and
> basically vetoed anything that didn't do what they said
> 
> Brice
> 
> 
> 
> Le 08/11/2012 15:43, Jeff Squyres a écrit :
>> Note that the saga of trying to push ummunotify upstream to Linux ended up with Linus essentially saying "fix your own network stack; don't put this in the main kernel."
>> 
>> He's was right back then.  With a 2nd "customer" for this kind of thing (cuda), that equation might be changing, but I'll leave that to Nvidia to push on Linus.  :-)
>> 
>> Something like ummunotify should be in the ibcore area in the kernel.  And at least initially, probably something like this should be in the cuda kernel module(s).
>> 
>> Just my $0.02...
>> 
>> 
>> 
>> On Nov 8, 2012, at 9:38 AM, Shamis, Pavel wrote:
>> 
>>> Another good reason for ummunotify kernel module
>>> (http://lwn.net/Articles/345013/)
>>> 
>>> Pavel (Pasha) Shamis
>>> ---
>>> Computer Science Research Group
>>> Computer Science and Math Division
>>> Oak Ridge National Laboratory
>>> 
>>> On Nov 8, 2012, at 9:08 AM, Jeff Squyres wrote:
>>> 
>>> On Nov 8, 2012, at 8:51 AM, Rolf vandeVaart wrote:
>>> 
>>> Not sure.  I will look into this.   And thank you for the feedback Jens!
>>> 
>>> FWIW, I +1 Jens' request.  MPI implementations are able to handle network registration mechanisms via standard memory hooks (their hooks are actually pretty terrible, but for the most part, they are generally functional).
>>> 
>>> If CUDA requires registered memory, then it should also provide hooks so that MPI implementations can "just make it work" from the users' perspective (and please please please provide BETTER hooks than verbs / glibc malloc!).
>>> 
>>> --
>>> Jeff Squyres
>>> jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>
>>> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
>>> 
>>> 
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> 
>>> 
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> 
> 
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/