Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] "An error occurred in MPI_Recv" with more than 2 CPU
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-05-27 10:41:08


vasilis wrote:

>Thank you Eugene for your suggestion. I used different tags for each variable,
>and now I do not get this error.
>The problem now is that I am getting a different solution when I use more than
>2 CPUs. I checked the matrices and I found that they differ by a very small
>amount of the order 10^(-10). Actually, I am getting a different solution if I
>use 4CPUs or 16CPUs!!!
>Do you have any idea what could cause this behavior?
>
>
Sure.

Rank 0 accumulates all the res_cpu values into a single array, res. It
starts with its own res_cpu and then adds all other processes. When
np=2, that means the order is prescribed. When np>2, the order is no
longer prescribed and some floating-point rounding variations can start
to occur.

If you want results to be more deterministic, you need to fix the order
in which res is aggregated. E.g., instead of using MPI_ANY_SOURCE, loop
over the peer processes in a specific order.

P.S. It seems to me that you could use MPI collective operations to
implement what you're doing. E.g., something like:

call MPI_Reduce(res_cpu, res, total_unknown, MPI_DOUBLE_PRECISION,
MPI_SUM, 0, MPI_COMM_WORLD, ierr)

call MPI_Gather(jacob_cpu, total_elem_cpu * unique, MPI_DOUBLE_PRECISION, &
                jacob , total_elem_cpu * unique,
MPI_DOUBLE_PRECISION, 0, MPI_COMM_WORLD, ierr)
call MPI_Gather( row_cpu, total_elem_cpu * unique, MPI_INTEGER , &
                  row , total_elem_cpu * unique, MPI_INTEGER
, 0, MPI_COMM_WORLD, ierr)
call MPI_Gather( col_cpu, total_elem_cpu * unique, MPI_INTEGER , &
                  col , total_elem_cpu * unique, MPI_INTEGER
, 0, MPI_COMM_WORLD, ierr)

I think the res part is right. The jacob/row/col parts are not quite
right since you don't just want to gather the elements, but add them
into particular arrays. Not sure if you really want to allocate a new
scratch array to use for this purpose or what. Nor would this solve the
res_cpu indeterministic problem you had. I just wanted to make sure you
knew about the MPI collective operations as an alternative to your
point-to-point implementation.