Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] Question about MCA_BTL_DES_SEND_ALWAYS_CALLBACK
From: Rolf Vandevaart (Rolf.Vandevaart_at_[hidden])
Date: 2010-01-13 08:42:29


Hello Group:

In the openib BTL, there is a some code that gets executed upon a
completion event. One of the things that happens is we call the
callback function of the descriptor. I notice that with coalesced
fragments, we do not check the MCA_BTL_DES_SEND_ALWAYS_CALLBACK flag
prior to issuing the callback. Does anyone know why we do not do that
check?

In normal operation, I have never seen an issue with this. However,
with some of the failover work I am doing, I ended up triggering a SEGV
when I tried to call a non-existent callback. Here is the code snippet
from btl_openib_component.c in the trunk. (this may not be very
readable) At line 2902 we do not check the flag, but at line 2911 we do.

2896case IBV_WC_SEND:
2897 OPAL_OUTPUT((-1, "Got WC: RDMA_WRITE or SEND"));
2898 if(openib_frag_type(des) == MCA_BTL_OPENIB_FRAG_SEND) {
2899 opal_list_item_t *i;
2900 while((i =
opal_list_remove_first(&to_send_frag(des)->coalesced_frags))) {
2901 btl_ownership = (to_base_frag(i)->base.des_flags &
MCA_BTL_DES_FLAGS_BTL_OWNERSHIP);
2902 to_base_frag(i)->base.des_cbfunc(&openib_btl->super, endpoint,
2903 &to_base_frag(i)->base, OMPI_SUCCESS);
2904 if( btl_ownership ) {
2905 mca_btl_openib_free(&openib_btl->super, &to_base_frag(i)->base);
2906 }
2907 }
2908 }
2909 /* Process a completed send/put/get */
2910 btl_ownership = (des->des_flags & MCA_BTL_DES_FLAGS_BTL_OWNERSHIP);
2911 if (des->des_flags & MCA_BTL_DES_SEND_ALWAYS_CALLBACK) {
2912 des->des_cbfunc(&openib_btl->super, endpoint, des,OMPI_SUCCESS);
2913 }
2914 if( btl_ownership ) {
2915 mca_btl_openib_free(&openib_btl->super, des);
2916 }
2917

-- 
=========================
rolf.vandevaart_at_[hidden]
781-442-3043
=========================