I would find this a useful feature. I haven't played with the diff so
I can't comment on it, but the idea of it sounds good to me.
On Nov 29, 2007, at 6:37 PM, Aurelien Bouteiller wrote:
> This patch introduces customisable wait/test for requests as
> discussed at the face-to-face ompi meeting in Paris.
> A new global structure (ompi_request_functions) holding all the
> pointers to the wait/test functions have been added.
> ompi_request_wait* and ompi_request_test* have been #defined to be
> replaced by ompi_request_functions.req_wait. The default
> implementations of the wait/test functions names have been changed
> from ompi_request_% to ompi_request_default_%. Those functions are
> static initializer of the ompi_request_functions structure.
> To modify the defaults, a components 1) copy the
> ompi_request_functions structure (the type ompi_request_fns_t can
> be used to declare a suitable variable), 2) change some of the
> functions according to its needs. This is best done at MPI_init
> time when there is no threads. Should this component be unloaded it
> have to restore the defaults. The ompi_request_default_* functions
> should never be called directly anywhere in the code. If a
> component needs to access the previously defined implementation of
> wait, it should call its local copy of the function. Component
> implementors should keep in mind that another component might have
> already changed the defaults and needs to be called.
> Performance impact on NetPipe -a (async recv mode) does not show
> measurable overhead. Here follows the "diff -y" between original
> and modified ompi assembly code from ompi/mpi/c/wait.c. The only
> significant difference is an extra movl to load the address of the
> ompi_request_functions structure in eax. This obviously explains
> why there is no measurable cost on latency.
> L2: L2:
> movl L_ompi_request_null$non_lazy_ptr-"L00000000001$pb"(%ebx), %
> eax movl L_ompi_request_null$non_lazy_ptr-"L00000000001$pb"(%
> ebx), %eax
> cmpl %eax, (%edi) cmpl %eax, (%edi)
> je L18 je L18
> > movl L_ompi_request_functions
> $non_lazy_ptr-"L00000000001$pb"(%ebx), %eax
> movl %esi, 4(%esp) movl %esi, 4(%esp)
> movl %edi, (%esp) movl %edi, (%esp)
> call L_ompi_request_wait$stub | call *16(%eax)
> Here is the patch for those who want to try themselves.
> If I receive comments outlining the need, thread safe accessors
> could be added to allow components to change the functions at
> anytime during execution and not only during MPI_Init/Finalize.
> Please make noise if you find this useful.
> If comments does not suggest extra work, I expect this code to be
> committed in trunk next week.
> Le 8 oct. 07 à 06:01, Aurelien Bouteiller a écrit :
>> For message logging purpose, we need to interface with wait_any,
>> wait_some, test, test_any, test_some, test_all. It is not possible to
>> use PMPI for this purpose. During the face-to-face meeting in Paris
>> (5-12 october 2007) we discussed this issue and came to the
>> conclusion that the best way to achieve this is to replace direct
>> calls to ompi_request_wait* and test* by indirect calls (same way as
>> PML send, recv, etc).
>> Basic idea is to declare a static structure containing the 8 pointers
>> to all the functions. This structure is initialized at compilation
>> time with the current basic wait/test functions. Before end of
>> MPI_init, any component might replace the basics with specialized
>> Expected cost is less than .01us latency according to preliminary
>> test. The method is consistent with the way we call pml send/recv.
>> Mechanism could be used later for stripping out grequest from
>> critical path when they are not used.
>> Aurelien Bouteiller, PhD
>> Innovative Computing Laboratory - MPI group
>> +1 865 974 6321
>> 1122 Volunteer Boulevard
>> Claxton Education Building Suite 350
>> Knoxville, TN 37996
>> devel mailing list
> Dr. Aurelien Bouteiller, Sr. Research Associate
> Innovative Computing Laboratory - MPI group
> +1 865 974 6321
> 1122 Volunteer Boulevard
> Claxton Education Building Suite 350
> Knoxville, TN 37996
> devel mailing list