Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

From: Jeff Squyres \(jsquyres\) (jsquyres_at_[hidden])
Date: 2006-04-14 11:52:41

Martin --
We finally figured out the Right solution and have committed it to all
three SVN branches:
- trunk (head of development)
- v1.1 (recently-created branch for the upcoming v1.1)
- v1.0 (stable release)
The fix is in the nightly tarballs for the trunk and v1.1; due to a
different problem, it didn't make it into last night's v1.0 snapshot but
it should be there tonight.
Thanks for finding this!


        From: users-bounces_at_[hidden]
[mailto:users-bounces_at_[hidden]] On Behalf Of Audet, Martin
        Sent: Monday, April 10, 2006 4:34 PM
        To: users_at_[hidden]
        Subject: [OMPI users] Incorrect behavior for attributes attached

        It looks like there is a problem in OpenMPI 1.0.2 with how
MPI_COMM_SELF attributes callback functions are handled by
        The following C program register a callback function associated
with the MPI_COMM_SELF communicator to be called during the first steps
of MPI_Finalize(). As shown in this example, this can be used to make
sure that global MPI_Datatype variables associated to global datatypes
are freed by calling MPI_Type_free() before program exit (and thus
preventing ugly memory leaks/outstanding allocations when run in
valgrind for example). This mechanism is used by the library I'm working
on as well as by PetSc library.
        The program works by taking advantage of the MPI-2 Standard
Section 4.8 "Allowing User Function at Process Termination". As it says,
the MPI_Finalize() function calls the delete callback associated to the
MPI_COMM_SELF attribute "before any other part of MPI are affected". It
also says that "calling MPI_Finalized() will return false in any of
these callback functions".
        Section 4.9 of the MPI-2 Standard: "Determining Whether MPI Has
Finished" moreover says that it can be determined if MPI is active by
calling MPI_Finalized(). It also reaffirm that MPI is active in the
callback functions invoked by MPI_Finalize().
        I think that an "active" MPI library here means that basic MPI
functions like MPI_Type_free() can be called.
        The following small program therefore seems to conform to the
MPI standard.
        However where I run it (compiled with OpenMPI 1.0.2 mpicc), I
get the following message:
        *** An error occurred in MPI_Type_free
        *** after MPI was finalized
        *** MPI_ERRORS_ARE_FATAL (goodbye)
        Note that this program works well with mpich2.
        Please have a look at this problem.
        Martin Audet
        #include <assert.h>
        #include <stddef.h>
        #include <mpi.h>
        static int attr_delete_function(MPI_Comm p_comm, int p_keyval,
void *p_attribute_val, void * p_extra_state)
           assert(p_attribute_val != NULL);
           /* Get a reference on the datatype received. */
           MPI_Datatype *const cur_datatype = (MPI_Datatype
           /* Free it if non null. */
           if (*cur_datatype != MPI_DATATYPE_NULL) {
assert(*cur_datatype == MPI_DATATYPE_NULL);
           return MPI_SUCCESS;
        /* If p_datatype refer to a non null MPI datatype, this function
will register a callback */
        /* function to free p_datatype and set it to MPI_DATATYPE_NULL.
This callback will be */
        /* called during the first steps of the MPI_Finalize() function
when the state of the MPI */
        /* library still allows MPI functions to be called. This is
done by associating an */
        /* attribute to the MPI_COMM_SELF communicator as allowed by
the MPI 2 standard (section 4.8). */
        static void add_type_free_callback(MPI_Datatype *p_datatype)
           int keyval;
           assert(p_datatype != NULL);
           /* First create the keyval.
           /* No callback function will be called when MPI_COMM_SELF is
duplicated */
           /* and attr_delete_function() will be called when
           /* freed (e.g. during MPI_Finalize()).
           /* Since many callback can be associated with MPI_COMM_SELF
to free many */
           /* datatypes, a new keyval has to be created every time.
           MPI_Keyval_create(MPI_NULL_COPY_FN, &attr_delete_function,
&keyval, NULL);
           /* Then associate this keyval to MPI_COMM_SELF and make sure
the pointer */
           /* to the datatype p_datatype is passed to the callback.
           MPI_Attr_put(MPI_COMM_SELF, keyval, p_datatype);
           /* Free the keyval because it is no longer needed.
        typedef struct {
           short ss;
           int ii;
        } glb_struct_t;
        MPI_Datatype glb_dtype = MPI_DATATYPE_NULL;
        static void calc_glb_dtype(void)
           const int NB_MEM = 3;
           static int len_tbl[3] = { 1,
1, 1 };
           static MPI_Aint disp_tbl[3] = { offsetof(glb_struct_t,
ss), offsetof(glb_struct_t, ii), sizeof(glb_struct_t) };
           static MPI_Datatype type_tbl[3] = { MPI_SHORT,
           MPI_Type_struct(NB_MEM, len_tbl, disp_tbl, type_tbl,
        int main(int argc, char *argv[])
           MPI_Init(&argc, &argv);
           return 0;