Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Program deadlocks, on simple send/recv loop
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-12-03 12:55:09


Jeff Squyres wrote:
On Dec 3, 2009, at 10:56 AM, Brock Palen wrote:
  
The allocation statement is ok:
allocate(vec(vec_size,vec_per_proc*(size-1)))

This allocates memory vec(32768, 2350)
    
So this allocates 32768 rows, each with 2350 columns -- all stored contiguously in memory, in column-major order.  Does the language/compiler *guarantee* that the entire matrix is contiguous in memory?  Or does it only guarantee that the *columns* are contiguous in memory -- and there may be gaps between successive columns?
  
I think you're getting one big contiguous block of memory and the portions that are passed are contiguous, nonoverlapping pieces.
This means that in the first iteration, you're calling:
call MPI_RECV(vec(1, 2301), 32768, ...)

And in the last iteration, you're calling:
call MPI_RECV(vec(1, 2350), 32768, ...)

That doesn't seem right.  If I'm reading this right -- and I very well may not be -- it looks like successive receives will be partially overlaying the previous receive.
No.  In Fortran, leftmost index varies the fastest.  E.g.,

% cat y.f90
  integer a(2,2)
  a(1,1) = 11
  a(2,1) = 21
  a(1,2) = 12
  a(2,2) = 22
  call sub(a)
end

subroutine sub(a)
  integer a(4)
  write(6,*) a
end
% a.out
 11 21 12 22
%

Here is how I think of Brock's code:

program sendbuf

  include 'mpif.h'

  integer, parameter :: n = 32 * 1024, m = 50

  complex*16 buf(n)

  call MPI_INIT(ierr)
  call MPI_COMM_SIZE(MPI_COMM_WORLD, np, ierr)
  call MPI_COMM_RANK(MPI_COMM_WORLD, me, ierr)

  buf = 0

  if ( me == 0 ) then
     do i = 1, np-1
        do j = 1, m
           call MPI_RECV(buf, n, MPI_DOUBLE_COMPLEX, i, j, MPI_COMM_WORLD, MPI_STATUS_IGNORE, ierr)
        end do
     end do
  else
     do j = 1, m
        call MPI_SEND( x, n, MPI_DOUBLE_COMPLEX, 0, j, MPI_COMM_WORLD, ierr)
     end do
  end if

  call MPI_FINALIZE(ierr)
end


This version reuses send and receive buffers, but that's fine since they're all blocking calls anyhow.