Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] [OMPI svn] svn:open-mpi r32163 - in trunk: opal/mca/base orte/tools/orterun
From: Mike Dubman (miked_at_[hidden])
Date: 2014-07-15 15:03:43


these are two separate issues:

1. -x var=val (or -mca opal_base_envlist var=val) will work in the same way
opal_base_envlist does the same as "-x" and can be used in the very same
fashion as -x

2. When list of vars is passed with help of opal_base_envlist, the escaping
is possible but escaped char should differ from delimiter char.

I think -x should stay as shotrt-form alias for -mca opal_base_envlist
var=val and -mca opal_base_envlist var.
on dev meeting it was decided to deprecate it as some point.

On Tue, Jul 15, 2014 at 8:14 PM, Dave Goodell (dgoodell) <dgoodell_at_[hidden]
> wrote:

> This commit (and the subsequent amendments to the feature) doesn't appear
> to support escaping the separator. A later commit allows you to change the
> separator character, which helps, but AFAICS you still can't actually
> escape the separator itself. That seems like a real deficiency to me...
>
> Furthermore, I really like the "-x" argument and I'm sad to see that it's
> being deprecated in favor of a much clunkier syntax. Is there a good
> reason we can't keep the "-x" syntax and only complain when there is a
> conflict with the mca_base_env_list variable?
>
> -Dave
>
> On Jul 8, 2014, at 7:38 PM, svn-commit-mailer_at_[hidden] wrote:
>
> > Author: jladd (Joshua Ladd)
> > Date: 2014-07-08 20:38:25 EDT (Tue, 08 Jul 2014)
> > New Revision: 32163
> > URL: https://svn.open-mpi.org/trac/ompi/changeset/32163
> >
> > Log:
> > Opal: add a new MCA parameter that allows the user to specify a list of
> environment variables. This parameter will become the standard mechanism by
> which environment variables are set for OMPI applications replacing the -x
> option.
> >
> > mpirun ... -x env_foo1=val1 -x env_foo2 -x env_foo3=val3 should now be
> expressed as
> >
> > mpirun ... -mca mca_base_env_list env_foo1=val1+env_foo2+env_foo3=val3.
> >
> > The motivation for doing this is so that a list of environment variables
> may be set via standard MCA mechanisms such as mca parameter files, amca
> lists, etc.
> >
> > This feature was developed by Elena Shipunova and was reviewed by Josh
> Ladd.
> >
> > Text files modified:
> > trunk/opal/mca/base/help-mca-var.txt | 11 ++++++++
> > trunk/opal/mca/base/mca_base_var.c | 52
> ++++++++++++++++++++++++++++++++++++++++
> > trunk/orte/tools/orterun/help-orterun.txt | 13 +++++++++
> > trunk/orte/tools/orterun/orterun.c | 7 +++++
> > 4 files changed, 82 insertions(+), 1 deletions(-)
> >
> > Modified: trunk/opal/mca/base/help-mca-var.txt
> >
> ==============================================================================
> > --- trunk/opal/mca/base/help-mca-var.txt Tue Jul 8 20:10:04 2014
> (r32162)
> > +++ trunk/opal/mca/base/help-mca-var.txt 2014-07-08 20:38:25 EDT
> (Tue, 08 Jul 2014) (r32163)
> > @@ -121,3 +121,14 @@
> >
> > Value: %s
> > Source: %s
> > +#
> > +[incorrect-env-list-param]
> > +WARNING: The format of MCA parameter "mca_base_env_list" is a plus-sign
> (+) delimited
> > +list of VAR=VAL and/or VAR instances, e.g.: -mca mca_base_env_list
> VAR1=VAL1+VAR2+VAR3=VAL3;...
> > +If a variable, VAR, is listed but not explicitly assigned a value in
> the command line, VAR will
> > +be assigned the value set in the executing environment.
> > +
> > +The following environment variable was listed unassigned in
> "mca_base_env_list", but was
> > +not found in your environment:
> > + Variable: %s
> > + MCA variable value: %s
> >
> > Modified: trunk/opal/mca/base/mca_base_var.c
> >
> ==============================================================================
> > --- trunk/opal/mca/base/mca_base_var.c Tue Jul 8 20:10:04 2014
> (r32162)
> > +++ trunk/opal/mca/base/mca_base_var.c 2014-07-08 20:38:25 EDT
> (Tue, 08 Jul 2014) (r32163)
> > @@ -62,6 +62,7 @@
> > static char *mca_base_var_override_file = NULL;
> > static char *mca_base_var_file_prefix = NULL;
> > static char *mca_base_param_file_path = NULL;
> > +static char *mca_base_env_list = NULL;
> > static bool mca_base_var_suppress_override_warning = false;
> > static opal_list_t mca_base_var_file_values;
> > static opal_list_t mca_base_var_override_values;
> > @@ -123,6 +124,7 @@
> > static int var_set_initial (mca_base_var_t *var);
> > static int var_get (int vari, mca_base_var_t **var_out, bool original);
> > static int var_value_string (mca_base_var_t *var, char **value_string);
> > +static int mca_base_var_process_env_list(void);
> >
> > /*
> > * classes
> > @@ -255,11 +257,61 @@
> > mca_base_var_initialized = true;
> >
> > mca_base_var_cache_files(false);
> > +
> > + /* set nesessary env variables for external usage */
> > + mca_base_var_process_env_list();
> > }
> >
> > return OPAL_SUCCESS;
> > }
> >
> > +static int mca_base_var_process_env_list(void)
> > +{
> > + int i, ret;
> > + char** tokens;
> > + char* ptr;
> > + char* param, *value;
> > + ret = mca_base_var_register ("opal", "mca", "base", "env_list",
> > + "Set SHELL env variables",
> > + MCA_BASE_VAR_TYPE_STRING, NULL, 0, 0,
> OPAL_INFO_LVL_3,
> > + MCA_BASE_VAR_SCOPE_READONLY,
> &mca_base_env_list);
> > + if ((0 > ret) || (NULL == mca_base_env_list)) {
> > + return OPAL_SUCCESS;
> > + }
> > + tokens = opal_argv_split(mca_base_env_list, '+');
> > + if (NULL != tokens) {
> > + for (i = 0; NULL != tokens[i]; i++) {
> > + if (NULL == (ptr = strchr(tokens[i], '='))) {
> > + value = getenv(tokens[i]);
> > + if (NULL != value) {
> > + if (NULL != strchr(value, '=')) {
> > + param = strdup(value);
> > + value = strchr(param, '=');
> > + *value = '\0';
> > + value++;
> > + opal_setenv(param, value, true, &environ);
> > + free(param);
> > + } else {
> > + opal_setenv(tokens[i], value, true, &environ);
> > + }
> > + } else {
> > + opal_show_help("help-mca-var.txt",
> "incorrect-env-list-param",
> > + true, tokens[i], mca_base_env_list);
> > + }
> > + } else {
> > + param = strdup(tokens[i]);
> > + value = strchr(param, '=');
> > + *value = '\0';
> > + value++;
> > + opal_setenv(param, value, true, &environ);
> > + free(param);
> > + }
> > + }
> > + opal_argv_free(tokens);
> > + }
> > + return OPAL_SUCCESS;
> > +}
> > +
> > static int mca_base_var_cache_files(bool rel_path_search)
> > {
> > char *tmp;
> >
> > Modified: trunk/orte/tools/orterun/help-orterun.txt
> >
> ==============================================================================
> > --- trunk/orte/tools/orterun/help-orterun.txt Tue Jul 8 20:10:04 2014
> (r32162)
> > +++ trunk/orte/tools/orterun/help-orterun.txt 2014-07-08 20:38:25 EDT
> (Tue, 08 Jul 2014) (r32163)
> > @@ -649,4 +649,15 @@
> > The job will now be aborted. Please check your code and/or
> > adjust/remove the job execution time limit (as specified
> > by MPIEXEC_TIMEOUT in your environment).
> > -
> > +#
> > +[orterun:deprecated-env-set]
> > +WARNING: The mechanism by which environment variables are passed to
> OMPI is changing!!
> > +Specifically, beginning in the 1.9.x/2.0.x series, using "-x" to set
> environment
> > +variables is deprecated. Please use the "mca_base_env_list" MCA
> parameter. With this new
> > +mechanism, mpirun ... -x env_foo1=bar1 -x env_foo2=bar2 -x env_foo3 ...
> > +becomes: mpirun ... -mca mca_base_env_list
> env_foo1=bar1+env_foo2=bar2+env_foo3 ...
> > +#
> > +[orterun:conflict-env-set]
> > +ERROR: You have attempted to pass environment variables to OMPI with
> both the "-x" method (deprecated starting
> > +in the 1.9.x/2.0.x series) and by setting the MCA parameter
> "mca_base_env_list". OMPI does not support mixing
> > +these two methods. Please choose one method and try launching your job
> again. You job will now abort.
> >
> > Modified: trunk/orte/tools/orterun/orterun.c
> >
> ==============================================================================
> > --- trunk/orte/tools/orterun/orterun.c Tue Jul 8 20:10:04 2014
> (r32162)
> > +++ trunk/orte/tools/orterun/orterun.c 2014-07-08 20:38:25 EDT
> (Tue, 08 Jul 2014) (r32163)
> > @@ -1722,6 +1722,13 @@
> >
> > /* Did the user request to export any environment variables on the
> cmd line? */
> > if (opal_cmd_line_is_taken(&cmd_line, "x")) {
> > + char* env_set_flag = getenv("OMPI_MCA_mca_base_env_list");
> > + if (NULL != env_set_flag) {
> > + orte_show_help("help-orterun.txt",
> "orterun:conflict-env-set", false);
> > + return ORTE_ERR_FATAL;
> > + } else {
> > + orte_show_help("help-orterun.txt",
> "orterun:deprecated-env-set", false);
> > + }
> > j = opal_cmd_line_get_ninsts(&cmd_line, "x");
> > for (i = 0; i < j; ++i) {
> > param = opal_cmd_line_get_param(&cmd_line, "x", i, 0);
> > _______________________________________________
> > svn mailing list
> > svn_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/svn
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post:
> http://www.open-mpi.org/community/lists/devel/2014/07/15148.php
>