Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] Notes from mem hooks call today
From: Brian W. Barrett (brbarret_at_[hidden])
Date: 2008-05-28 17:20:29

On Wed, 28 May 2008, Roland Dreier wrote:

> > - gleb asks: don't we want to avoid the system call when possible?
> > - patrick: a single syscall can be/is cheaper than a reg cache
> > lookup in user space
> This doesn't really make sense -- syscall + cache lookup in kernel is
> "obviously" more expensive than cache lookup in userspace with no
> context switch (I don't see any tricks the kernel can do that make the
> cache lookup cheaper there).
> However the solution I proposed a long time ago (when Pete Wyckoff
> originally did his work on having the kernel track this -- and as a side
> note, it's not clear to me whether MMU notifiers really help what Pete
> did) is for userspace to provide a pointer to a flag when registering
> memory with the kernel, and then the kernel can mark the flag if the
> mapping changes -- ie keep the userspace cache but have the kernel
> manage invalidation "perfectly" without any malloc hooks.

I think Patrick's point is that it's not too much more expensive to do the
syscall on Linux vs just doing the cache lookup, particularly in the
context of a long message. And it means that upper layer protocols like
MPI don't have to deal with caches (and since MPI implementors hate
registration caches only slightly less than we hate MPI_CANCEL, that will
make us happy).