On Tuesday 06 May 2008 09:41:53 am Jeff Squyres wrote:
> I actually don't know what the RDMA CM requires for the LMC>0 case --
> does it require a unique IP address for every LID?
It requires a unique IP address for every hca/port in use by rdmacm.
I see the bug in rdmacm (since I don't believe you were trying to use rdmacm), and will have a patch out shortly.
>
> On May 6, 2008, at 5:09 AM, Lenny Verkhovsky wrote:
>
> > Hi,
> >
> > running BW benchmark with btl_openib_max_lmc >= 2 couses warning
> > ( MPI from the TRUNK ) !!!!
> >
> >
> > #mpirun --bynode -np 40 -hostfile hostfile_ompi_arbel -mca
> > btl_openib_max_lmc 2 ./mpi_p_LMC -t bw -s 400000
> > BW (40) (size min max avg) 400000 321.493757
> > 342.972837 329.493715
> >
> > #mpirun --bynode -np 40 -hostfile hostfile_ompi_arbel -mca
> > btl_openib_max_lmc 3 ./mpi_p_LMC -t bw -s 400000
> > [witch9][[7493,1],7][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch2][[7493,1],0][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch10][[7493,1],9][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch6][[7493,1],4][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch4][[7493,1],2][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch7][[7493,1],5][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch2][[7493,1],10][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch9][[7493,1],17][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch5][[7493,1],3][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch8][[7493,1],6][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch6][[7493,1],14][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch10][[7493,1],19][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch5][[7493,1],13][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch4][[7493,1],12][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch9][[7493,1],27][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch5][[7493,1],23][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch2][[7493,1],20][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch9][[7493,1],37][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch7][[7493,1],35][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch4][[7493,1],32][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch4][[7493,1],22][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch5][[7493,1],33][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch2][[7493,1],30][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch8][[7493,1],16][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch7][[7493,1],15][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch10][[7493,1],39][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch7][[7493,1],25][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch10][[7493,1],29][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch6][[7493,1],34][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch8][[7493,1],26][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch6][[7493,1],24][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > [witch8][[7493,1],36][../../../../../ompi/mca/btl/openib/connect/
> > btl_openib_connect_rdmacm.c:989:create_message] No IP address found
> > BW (40) (size min max avg) 400000 312.622582
> > 334.037277 324.014814
> >
> > using -mca btl openib,self causes warning with LMC >=10
> >
> >
> > Best regards
> > Lenny.
> > _______________________________________________
> > devel mailing list
> > devel_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
|