Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Tom Rosmond (rosmond_at_[hidden])
Date: 2006-09-06 13:40:28


I notice in the OMPI_INFO output the following parameters that seem
relevant to this problem:

MCA btl: parameter "btl_self_free_list_num" (current value: "0")
MCA btl: parameter "btl_self_free_list_max" (current value: "-1")
MCA btl: parameter "btl_self_free_list_inc" (current value: "32")
MCA btl: parameter "btl_self_eager_limit" (current value: "131072")
MCA btl: parameter "btl_self_max_send_size" (current value: "262144")
MCA btl: parameter "btl_self_max_rdma_size" (current value: "2147483647")
MCA btl: parameter "btl_self_exclusivity" (current value: "65536")
MCA btl: parameter "btl_self_flags" (current value: "2")
MCA btl: parameter "btl_self_priority" (current value: "0")

Specifically the 'self_max_send_size=262144', which I assume is the
maximum size (bytes?) message a processor can send to itself. None of
the messages in my above tests approached this limit. However, I am
puzzled by this, because the program below runs correctly for
ridiculously large message sizes (as shown 200 Mbytes).

program test
implicit none
include 'mpif.h'
integer imjm
parameter (imjm=200000000)
integer itype , istrt , itau ,istat , msg , nstat
integer irank,nproc,iwin,i,n,ir
integer isizereal,iwinsize,itarget_disp
integer , dimension(:) , allocatable :: len
integer , dimension(:) , allocatable :: loff
real , dimension(:) , allocatable :: x
real , dimension(:) , allocatable :: ximjm
call mpi_init(istat)
call mpi_comm_rank(mpi_comm_world,irank,istat)
call mpi_comm_size(mpi_comm_world,nproc,istat)
ir = irank + 1
do 200 n = 1,nproc
len(n) = imjm/nproc
loff(n) = (n-1)*imjm/nproc
200 continue
call mpi_type_size(mpi_real,isizereal,istat)
iwinsize = imjm*isizereal
call mpi_win_create(ximjm,iwinsize,isizereal,mpi_info_null,
& mpi_comm_world,iwin,istat)
do 250 i = 1,imjm
ximjm(i) = i
250 continue
itarget_disp = loff(ir)
call mpi_win_fence(0,iwin,istat)
call mpi_get(x,len(ir),mpi_real,0,itarget_disp,len(ir),mpi_real,
& iwin,istat)
call mpi_win_fence(0,iwin,istat)
print('(A,i3,8f20.2)'),' x ',ir,x(1),x(len(ir))

Tom Rosmond

Brian Barrett wrote:

>On Mon, 2006-09-04 at 11:01 -0700, Tom Rosmond wrote:
>>Attached is some error output from my tests of 1-sided message
>>passing, plus my info file. Below are two copies of a simple fortran
>>subroutine that mimics mpi_allgatherv using mpi-get calls. The top
>>version fails, the bottom runs OK. It seems clear from these
>>examples, plus the 'self_send' phrases in the error output, that there
>>is a problem internally with a processor sending data to itself. I
>>know that your 'mpi_get' implementation is simply a wrapper around
>>'send/recv' calls, so clearly this shouldn't happen. However, the
>>problem does not happen in all cases; I tried to duplicate it in a
>>simple stand-alone program with mpi_get calls and was unable to make
>>it fail. Go figure.
>That is an odd failure and at first glance it does look like there is
>something wrong with our one-sided implementation. I've filed a bug in
>our tracker about the issue and you should get updates on the ticket as
>we work on the issue.
>users mailing list