Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] openmpi 1.4 broken -mca coll_tuned_use_dynamic_rules 1
From: Daniel Spångberg (daniels_at_[hidden])
Date: 2009-12-30 11:42:43


That works! Many thanks!

Daniel

Den 2009-12-30 16:44:52 skrev Lenny Verkhovsky
<lenny.verkhovsky_at_[hidden]>:

> it may crash if it doesnt see a file with rules.
> try providing it through the command line
> $mpirun -mca coll_tuned_use_dynamic_rules 1 -mca
> coll_tuned_dynamic_rules_filename full_path_to_file_ .....
>
> On Wed, Dec 30, 2009 at 5:35 PM, Daniel Spångberg
> <daniels_at_[hidden]>wrote:
>
>> Thanks for the help with how to set up the collectives file. I am
>> unable to
>> make it work though,
>>
>> My simple alltoall test is still crashing, although I added even added a
>> line specifically for my test commsize of 64 and 100 bytes using bruck.
>>
>> daniels_at_kalkyl1:~/.openmpi > cat mca-params.conf
>>
>> coll_tuned_use_dynamic_rules=1
>> coll_base_verbose=0
>>
>> coll_tuned_dynamic_rules_filename="/home/daniels/.openmpi/dynamic_rules_file"
>> daniels_at_kalkyl1:~/.openmpi > cat dynamic_rules_file
>>
>> 1 # num of collectives
>> 3 # ID = 3 Alltoall collective (ID in coll_tuned.h)
>> 1 # number of com sizes
>> 64 # comm size 64
>> 3 # number of msg sizes
>> 0 3 0 0 # for message size 0, bruck 1, topo 0, 0 segmentation
>> 100 3 0 0 # for message size 100, bruck 1, topo 0, 0 segmentation
>>
>> 8192 2 0 0 # 8k+, pairwise 2, no topo or segmentation
>> # end of collective rule
>>
>> Still it useful to know how to do this, when this issue gets fixed in
>> the
>> future!
>>
>> Daniel
>>
>>
>>
>> Den 2009-12-30 15:57:50 skrev Lenny Verkhovsky
>> <lenny.verkhovsky_at_[hidden]
>> >:
>>
>>
>> The only workaround that I found is a file with dynamic rules.
>>> This is an example that George sent me once. It helped for me, until it
>>> will
>>> be fixed.
>>>
>>> " Lenny,
>>>
>>> You asked for dynamic rules but it looks like you didn't provide them.
>>> Dynamic rules allow the user to specify which algorithm to be used for
>>> each
>>> collective based on a set of rules. I corrected the current behavior,
>>> so
>>> it
>>> will not crash. However, as you didn't provide dynamic rules, it will
>>> just
>>> switch back to default behavior (i.e. ignore the
>>> coll_tuned_use_dynamic_rules MCA parameter).
>>>
>>> As an example, here is a set of dynamic rules. I added some comment to
>>> clarify it, but if you have any questions please ask.
>>>
>>> 2 # num of collectives
>>> 3 # ID = 3 Alltoall collective (ID in coll_tuned.h)
>>> 1 # number of com sizes
>>> 64 # comm size 64
>>> 2 # number of msg sizes
>>> 0 3 0 0 # for message size 0, bruck 1, topo 0, 0 segmentation
>>> 8192 2 0 0 # 8k+, pairwise 2, no topo or segmentation
>>> # end of collective rule
>>> #
>>> 2 # ID = 2 Allreduce collective (ID in coll_tuned.h)
>>> 1 # number of com sizes
>>> 1 # comm size 2
>>> 2 # number of msg sizes
>>> 0 1 0 0 # for message size 0, basic linear 1, topo 0, 0 segmentation
>>> 1024 2 0 0 # for messages size > 1024, nonoverlapping 2, topo 0, 0
>>> segmentation
>>> # end of collective rule
>>> #
>>>
>>> And here is what I have in my $(HOME)/.openmpi/mca-params.conf to
>>> activate
>>> them:
>>> #
>>> # Dealing with collective
>>> #
>>> coll_base_verbose = 0
>>>
>>> coll_tuned_use_dynamic_rules = 1
>>> coll_tuned_dynamic_rules_filename = **the name of the file where you
>>> saved
>>> the rules **
>>>
>>> "
>>>
>>> On Wed, Dec 30, 2009 at 4:44 PM, Daniel Spångberg <daniels_at_[hidden]
>>> >wrote:
>>>
>>> Interesting. I found your issue before I sent my report, but I did not
>>>> realise that this was the same problem. I see now that your example is
>>>> really for openmpi 1.3.4++
>>>>
>>>> Do you know of a work around? I have not used a rule file before and
>>>> seem
>>>> to be unable to find the documentation for how to use one,
>>>> unfortunately.
>>>>
>>>> Daniel
>>>>
>>>> Den 2009-12-30 15:17:17 skrev Lenny Verkhovsky <
>>>> lenny.verkhovsky_at_[hidden]
>>>> >:
>>>>
>>>>
>>>> This is the a knowing issue,
>>>>
>>>>> https://svn.open-mpi.org/trac/ompi/ticket/2087
>>>>> Maybe it's priority should be raised up.
>>>>> Lenny.
>>>>>
>>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>>
>>
>> --
>> Daniel Spångberg
>> Materialkemi
>> Uppsala Universitet
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>

-- 
Daniel Spångberg
Materialkemi
Uppsala Universitet