Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Tom Rosmond (rosmond_at_[hidden])
Date: 2006-09-06 13:40:28


Brian,

I notice in the OMPI_INFO output the following parameters that seem
relevant to this problem:

MCA btl: parameter "btl_self_free_list_num" (current value: "0")
MCA btl: parameter "btl_self_free_list_max" (current value: "-1")
MCA btl: parameter "btl_self_free_list_inc" (current value: "32")
MCA btl: parameter "btl_self_eager_limit" (current value: "131072")
MCA btl: parameter "btl_self_max_send_size" (current value: "262144")
MCA btl: parameter "btl_self_max_rdma_size" (current value: "2147483647")
MCA btl: parameter "btl_self_exclusivity" (current value: "65536")
MCA btl: parameter "btl_self_flags" (current value: "2")
MCA btl: parameter "btl_self_priority" (current value: "0")

Specifically the 'self_max_send_size=262144', which I assume is the
maximum size (bytes?) message a processor can send to itself. None of
the messages in my above tests approached this limit. However, I am
puzzled by this, because the program below runs correctly for
ridiculously large message sizes (as shown 200 Mbytes).

program test
!
implicit none
!
include 'mpif.h'
!
integer imjm
parameter (imjm=200000000)
!
integer itype , istrt , itau ,istat , msg , nstat
integer irank,nproc,iwin,i,n,ir
integer isizereal,iwinsize,itarget_disp
!
integer , dimension(:) , allocatable :: len
integer , dimension(:) , allocatable :: loff
real , dimension(:) , allocatable :: x
real , dimension(:) , allocatable :: ximjm
!
!
call mpi_init(istat)
call mpi_comm_rank(mpi_comm_world,irank,istat)
call mpi_comm_size(mpi_comm_world,nproc,istat)
!
allocate(len(nproc))
allocate(loff(nproc))
allocate(x(imjm/nproc))
!
ir = irank + 1
!
if(ir.eq.1)allocate(ximjm(imjm))
!
do 200 n = 1,nproc
len(n) = imjm/nproc
loff(n) = (n-1)*imjm/nproc
200 continue
!
call mpi_type_size(mpi_real,isizereal,istat)
!
iwinsize = imjm*isizereal
call mpi_win_create(ximjm,iwinsize,isizereal,mpi_info_null,
& mpi_comm_world,iwin,istat)
!
if(ir.eq.1)then
do 250 i = 1,imjm
ximjm(i) = i
250 continue
endif
!
itarget_disp = loff(ir)
call mpi_win_fence(0,iwin,istat)
call mpi_get(x,len(ir),mpi_real,0,itarget_disp,len(ir),mpi_real,
& iwin,istat)
call mpi_win_fence(0,iwin,istat)
!
print('(A,i3,8f20.2)'),' x ',ir,x(1),x(len(ir))
stop
end

Tom Rosmond

Brian Barrett wrote:

>On Mon, 2006-09-04 at 11:01 -0700, Tom Rosmond wrote:
>
>
>
>>Attached is some error output from my tests of 1-sided message
>>passing, plus my info file. Below are two copies of a simple fortran
>>subroutine that mimics mpi_allgatherv using mpi-get calls. The top
>>version fails, the bottom runs OK. It seems clear from these
>>examples, plus the 'self_send' phrases in the error output, that there
>>is a problem internally with a processor sending data to itself. I
>>know that your 'mpi_get' implementation is simply a wrapper around
>>'send/recv' calls, so clearly this shouldn't happen. However, the
>>problem does not happen in all cases; I tried to duplicate it in a
>>simple stand-alone program with mpi_get calls and was unable to make
>>it fail. Go figure.
>>
>>
>
>That is an odd failure and at first glance it does look like there is
>something wrong with our one-sided implementation. I've filed a bug in
>our tracker about the issue and you should get updates on the ticket as
>we work on the issue.
>
>Thanks,
>
>Brian
>
>_______________________________________________
>users mailing list
>users_at_[hidden]
>http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>