Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] trac 1857: SM btl hangs when msg >=4k, Performance degradation ???
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2009-04-13 10:46:46


I [unfortunately] think that our benchmark performance is
important. :-(

So I don't know if 64mb is too big, but it should probably be above
zero to avoid the performance degregation.

On Apr 13, 2009, at 10:28 AM, Eugene Loh wrote:

> Lenny Verkhovsky wrote:
>
> > Sorry, guys, I tested it on 1.3 branch, trunk version(1.4a1r20980)
> > seems to be fixed.
> >
> Great.
>
> > BUT,
> >
> > the default value of mpool_sm_min_size in 1.4a1r20980 is 67108864
> >
> > when I set it to 0, there is a performance degradation, is it OK ?
> >
> Depends on what matters for you! :^)
>
> Anyhow:
>
> 1) I think many bandwidth tests won't see this problem, but osu_bw is
> different since it pumps so many messages into the system
> concurrently.
>
> 2) For the sake of osu_bw, I think leaving the default at 64M is
> good.
>
> > $LD_LIBRARY_PATH=~/work/svn/ompi/trunk/build_x86-64/install/lib/
> > install/bin/mpirun -np 2 -mca btl sm,self -mca mpool_sm_min_size 0
> > ~/work/svn/hpc/tools/benchmarks/OMB-3.1.1/osu_bw
> > # OSU MPI Bandwidth Test v3.1.1
> > # Size Bandwidth (MB/s)
> > 1 1.20
> > 2 3.39
> > 4 6.93
> > 8 14.09
> > 16 27.80
> > 32 50.58
> > 64 101.08
> > 128 173.23
> > 256 257.81
> > 512 436.86
> > 1024 674.51
> > 2048 856.80
> > 4096 573.87
> > 8192 607.55
> > 16384 660.58
> > 32768 685.23
> > 65536 687.45
> > 131072 690.52
> > 262144 687.48
> > 524288 676.77
> > 1048576 675.74
> > 2097152 676.89
> > 4194304 677.28
> > lennyb_at_dellix7 ~/work/svn/ompi/trunk/build_x86-64
> > $LD_LIBRARY_PATH=~/work/svn/ompi/trunk/build_x86-64/install/lib/
> > install/bin/mpirun -np 2 -mca btl sm,self
> > ~/work/svn/hpc/tools/benchmarks/OMB-3.1.1/osu_bw
> > # OSU MPI Bandwidth Test v3.1.1
> > # Size Bandwidth (MB/s)
> > 1 1.72
> > 2 3.70
> > 4 7.43
> > 8 13.45
> > 16 29.83
> > 32 52.66
> > 64 105.08
> > 128 181.16
> > 256 288.16
> > 512 426.83
> > 1024 690.21
> > 2048 867.00
> > 4096 567.53
> > 8192 667.35
> > 16384 806.97
> > 32768 892.95
> > 65536 989.62
> > 131072 1009.25
> > 262144 1018.35
> > 524288 1037.32
> > 1048576 1048.75
> > 2097152 1057.51
> > 4194304 1062.16
> >
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
Cisco Systems