On Mar 1, 2007, at 9:01 AM, Ralph Castain wrote:
> Hmmmm...only one question left unanswered - where do these "aggregate"
> parameter sets come from??? Does a user create the file? Are we
> them somewhere? Are there sets that the sys admin provides that
> everyone on
> that system can use (but aren't to be used all the time via the
> conf file)?
Yes to all. Open MPI will ship with some of them, since each
component in the system can create a file and have it included in the
distro if they choose (a la the OpenIB AMCA file). The user can
create there own to experiment with. And the sysadmin can create
versions and place them in the $SYSCONFDIR/amca-param-sets directory
or in a special location for users.
The idea was that anyone can create them just like the mca-
params.conf file we currently ship. The only difference between the
new Aggregate MCA Parameter Set file and the mca-params.conf file is
in when they are used. The mca-params.conf file is always used unless
the user overrides it. The Aggregate MCA Parameter Sets are an opt-in
feature where the user must explicitly ask for a specific AMCA set
when running a job. So the idea is that they would only need one
additional command line argument to get a specialized set of MCA
parameters for a specific application type or run. That is instead of
N mca parameters that the user must mange.
> On 3/1/07 6:52 AM, "Josh Hursey" <jjhursey_at_[hidden]> wrote:
>> I just committed back to the trunk the Aggregate MCA (AMCA) Parameter
>> Set work that Jeff Squyres and I have been working on. This will be a
>> part of the eventual v1.3 release.
>> The motivation for creating AMCA parameter sets came from the
>> realization that for certain applications a large number of MCA
>> parameters needed to be set for the job to run well and/or as the
>> user expects. So the goal of this work was to help reduce the number
>> of MCA parameters that the user has to manage, therefore leading to a
>> better end user experience with Open MPI.
>> AMCA parameter set files are formated exactly like the "~/.openmpi/
>> mca-params.conf" configuration files. In addition when AMCA parameter
>> sets are used the user may still override the parameters on the
>> command line if they like.
>> For example, let's say there is a set of MCA parameters that a user
>> would need to set to get good performance out of Netpipe when using
>> Open IB. They would typically run the application as:
>> shell$ mpirun -np 2 NPmpi
>> To use the AMCA parameter set for Open IB the user would run:
>> shell$ mpirun -np 2 -am btl-openib-benchmark NPmpi
>> This will load a series of MCA parameters for the user. If they
>> wanted to override the max_btls MCA parameter for tuning reasons they
>> would run:
>> shell$ mpirun -np 2 -am btl-openib-benchmark -mca
>> btl_open_ib_max_btls 10 NPmpi
>> AMCA parameter sets can be coupled. If we take the example above and
>> wanted to also use an AMCA parameter set for TCP, the user would run:
>> shell$ mpirun -np 2 -am btl-openib-benchmark:btl-tcp-benchmark -
>> mca btl_open_ib_max_btls 10 NPmpi
>> The AMCA parameter sets are loaded in priority order. This means that
>> the OpenIB AMCA set has priority over the TCP AMCA set. So if the TCP
>> AMCA sets the MCA parameter "mpi_leave_pinned=0" and the OpenIB AMCA
>> sets it to "mpi_leave_pinned=1" then the latter, OpenIB version, will
>> be used.
>> Additional Related MCA parameters:
>> - mca_base_param_file_prefix
>> (Default: NULL)
>> This is the fullname of the "-am" mpirun option. Used to
>> specify a ':' separated list of AMCA parameter set files.
>> - mca_base_param_file_path
>> (Default: $SYSCONFDIR/amca-param-sets/:$CWD)
>> The path to search for AMCA files with relative paths. A
>> warning will be printed if the AMCA file cannot be found.
>> If you have any problems with this new feature let me know. There
>> will be an FAQ coming shortly about this I suspect.
>> Josh Hursey
>> Josh Hursey
>> devel mailing list
> devel mailing list