Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] matching code rewrite in OB1
From: Gleb Natapov (glebn_at_[hidden])
Date: 2007-12-18 10:05:03


On Mon, Dec 17, 2007 at 08:08:02PM -0500, Richard Graham wrote:
> Needless to say (for the nth time :-) ) that changing this bit of code
> makes me
> nervous.
I've noticed it already :)

> However, it occurred to me that there is a much better way to
> test
> this code than setting up an environment that generates some out of order
> events with out us being able to specify the order.
> Since this routine is executed serially, it should be sufficient to set up
> a test
> code that would simulate any out-of-order scenario we want. If one
> specifies
> number of ³messages² to be ³sent², and ³randomly² changes the order they
> arrive (e.g. scramble some input vector), one can check and see if the
> messages
> are ³received² in the correct order. One could even ³drop² messages and
> see
> if matching stops. Using a test code like this, and a code coverage tool,
> one
> should be able to get much better testing that we have to date.
While I sometimes do unit testing for code that I write, in this case it is
easy to generate all reasonable corner case without isolating the code in a
separate unit. I run this code through different specially constructed MPI
application and checked code coverage with gcov. Here is the result:

File '/home/glebn/OpenMPI/ompi.stg/ompi/mca/pml/ob1/pml_ob1_recvfrag.c'
Lines executed:97.58% of 124

Only two lines of the code was never executes, both for error cases that
should cause abort anyway.
The pml_ob1_recvfrag.c.gcov with coverage results is attached for
however is curious enough to look at them. BTW I doubt that previous
code passed this level of testing. At least with gcov it is not possible
to generate meaningful results when most of the code is inside macros.

> What would you think about doing something like this ? Seems like a few
> hours
> of this sort of simulation would be much better than even years of testing
> and
> relying on random fluctuations in the run to thoroughly test out-of-order
> scenarios.
>
> What do you think ?
I think that coverage testing I did is enough for this code.

> Rich
>
>
> On 12/17/07 8:32 AM, "Gleb Natapov" <glebn_at_[hidden]> wrote:
>
> > On Thu, Dec 13, 2007 at 08:04:21PM -0500, Richard Graham wrote:
> >> > Yes, should be a bit more clear. Need an independent way to verify that
> >> > data is matched
> >> > in the correct order ­ sending this information as payload is one way to
> >> do
> >> > this. So,
> >> > sending unique data in every message, and making sure that it arrives in
> >> > the user buffers
> >> > in the expected order is a way to do this.
> >
> > Did that. Encoded sequence number in a payload and sent many eager
> > packets from one rank to another. Many packets were reoredered, but
> > application received everything in a correct order.
> >
> > --
> > Gleb.
> >
> > _______________________________________________
> > devel mailing list
> > devel_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
> >
>
>

> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

--
			Gleb.