Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] trac 1857: SM btl hangs when msg >=4k, Performance degradation ???
From: Lenny Verkhovsky (lenny.verkhovsky_at_[hidden])
Date: 2009-04-13 11:14:39


I do care about performance more than memory as well,

but if 64M is the value that was before all the recent SM changings,

so let it stay :)

Lenny.

On Mon, Apr 13, 2009 at 5:46 PM, Jeff Squyres <jsquyres_at_[hidden]> wrote:

> I [unfortunately] think that our benchmark performance is important. :-(
>
> So I don't know if 64mb is too big, but it should probably be above zero to
> avoid the performance degregation.
>
>
> On Apr 13, 2009, at 10:28 AM, Eugene Loh wrote:
>
> Lenny Verkhovsky wrote:
>>
>> > Sorry, guys, I tested it on 1.3 branch, trunk version(1.4a1r20980)
>> > seems to be fixed.
>> >
>> Great.
>>
>> > BUT,
>> >
>> > the default value of mpool_sm_min_size in 1.4a1r20980 is 67108864
>> >
>> > when I set it to 0, there is a performance degradation, is it OK ?
>> >
>> Depends on what matters for you! :^)
>>
>> Anyhow:
>>
>> 1) I think many bandwidth tests won't see this problem, but osu_bw is
>> different since it pumps so many messages into the system concurrently.
>>
>> 2) For the sake of osu_bw, I think leaving the default at 64M is good.
>>
>> > $LD_LIBRARY_PATH=~/work/svn/ompi/trunk/build_x86-64/install/lib/
>> > install/bin/mpirun -np 2 -mca btl sm,self -mca mpool_sm_min_size 0
>> > ~/work/svn/hpc/tools/benchmarks/OMB-3.1.1/osu_bw
>> > # OSU MPI Bandwidth Test v3.1.1
>> > # Size Bandwidth (MB/s)
>> > 1 1.20
>> > 2 3.39
>> > 4 6.93
>> > 8 14.09
>> > 16 27.80
>> > 32 50.58
>> > 64 101.08
>> > 128 173.23
>> > 256 257.81
>> > 512 436.86
>> > 1024 674.51
>> > 2048 856.80
>> > 4096 573.87
>> > 8192 607.55
>> > 16384 660.58
>> > 32768 685.23
>> > 65536 687.45
>> > 131072 690.52
>> > 262144 687.48
>> > 524288 676.77
>> > 1048576 675.74
>> > 2097152 676.89
>> > 4194304 677.28
>> > lennyb_at_dellix7 ~/work/svn/ompi/trunk/build_x86-64
>> > $LD_LIBRARY_PATH=~/work/svn/ompi/trunk/build_x86-64/install/lib/
>> > install/bin/mpirun -np 2 -mca btl sm,self
>> > ~/work/svn/hpc/tools/benchmarks/OMB-3.1.1/osu_bw
>> > # OSU MPI Bandwidth Test v3.1.1
>> > # Size Bandwidth (MB/s)
>> > 1 1.72
>> > 2 3.70
>> > 4 7.43
>> > 8 13.45
>> > 16 29.83
>> > 32 52.66
>> > 64 105.08
>> > 128 181.16
>> > 256 288.16
>> > 512 426.83
>> > 1024 690.21
>> > 2048 867.00
>> > 4096 567.53
>> > 8192 667.35
>> > 16384 806.97
>> > 32768 892.95
>> > 65536 989.62
>> > 131072 1009.25
>> > 262144 1018.35
>> > 524288 1037.32
>> > 1048576 1048.75
>> > 2097152 1057.51
>> > 4194304 1062.16
>> >
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>
>
> --
> Jeff Squyres
> Cisco Systems
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>