Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] [OMPI svn] svn:open-mpi r32163 - in trunk: opal/mca/base orte/tools/orterun
From: Dave Goodell (dgoodell) (dgoodell_at_[hidden])
Date: 2014-07-15 13:14:45


This commit (and the subsequent amendments to the feature) doesn't appear to support escaping the separator. A later commit allows you to change the separator character, which helps, but AFAICS you still can't actually escape the separator itself. That seems like a real deficiency to me...

Furthermore, I really like the "-x" argument and I'm sad to see that it's being deprecated in favor of a much clunkier syntax. Is there a good reason we can't keep the "-x" syntax and only complain when there is a conflict with the mca_base_env_list variable?

-Dave

On Jul 8, 2014, at 7:38 PM, svn-commit-mailer_at_[hidden] wrote:

> Author: jladd (Joshua Ladd)
> Date: 2014-07-08 20:38:25 EDT (Tue, 08 Jul 2014)
> New Revision: 32163
> URL: https://svn.open-mpi.org/trac/ompi/changeset/32163
>
> Log:
> Opal: add a new MCA parameter that allows the user to specify a list of environment variables. This parameter will become the standard mechanism by which environment variables are set for OMPI applications replacing the -x option.
>
> mpirun ... -x env_foo1=val1 -x env_foo2 -x env_foo3=val3 should now be expressed as
>
> mpirun ... -mca mca_base_env_list env_foo1=val1+env_foo2+env_foo3=val3.
>
> The motivation for doing this is so that a list of environment variables may be set via standard MCA mechanisms such as mca parameter files, amca lists, etc.
>
> This feature was developed by Elena Shipunova and was reviewed by Josh Ladd.
>
> Text files modified:
> trunk/opal/mca/base/help-mca-var.txt | 11 ++++++++
> trunk/opal/mca/base/mca_base_var.c | 52 ++++++++++++++++++++++++++++++++++++++++
> trunk/orte/tools/orterun/help-orterun.txt | 13 +++++++++
> trunk/orte/tools/orterun/orterun.c | 7 +++++
> 4 files changed, 82 insertions(+), 1 deletions(-)
>
> Modified: trunk/opal/mca/base/help-mca-var.txt
> ==============================================================================
> --- trunk/opal/mca/base/help-mca-var.txt Tue Jul 8 20:10:04 2014 (r32162)
> +++ trunk/opal/mca/base/help-mca-var.txt 2014-07-08 20:38:25 EDT (Tue, 08 Jul 2014) (r32163)
> @@ -121,3 +121,14 @@
>
> Value: %s
> Source: %s
> +#
> +[incorrect-env-list-param]
> +WARNING: The format of MCA parameter "mca_base_env_list" is a plus-sign (+) delimited
> +list of VAR=VAL and/or VAR instances, e.g.: -mca mca_base_env_list VAR1=VAL1+VAR2+VAR3=VAL3;...
> +If a variable, VAR, is listed but not explicitly assigned a value in the command line, VAR will
> +be assigned the value set in the executing environment.
> +
> +The following environment variable was listed unassigned in "mca_base_env_list", but was
> +not found in your environment:
> + Variable: %s
> + MCA variable value: %s
>
> Modified: trunk/opal/mca/base/mca_base_var.c
> ==============================================================================
> --- trunk/opal/mca/base/mca_base_var.c Tue Jul 8 20:10:04 2014 (r32162)
> +++ trunk/opal/mca/base/mca_base_var.c 2014-07-08 20:38:25 EDT (Tue, 08 Jul 2014) (r32163)
> @@ -62,6 +62,7 @@
> static char *mca_base_var_override_file = NULL;
> static char *mca_base_var_file_prefix = NULL;
> static char *mca_base_param_file_path = NULL;
> +static char *mca_base_env_list = NULL;
> static bool mca_base_var_suppress_override_warning = false;
> static opal_list_t mca_base_var_file_values;
> static opal_list_t mca_base_var_override_values;
> @@ -123,6 +124,7 @@
> static int var_set_initial (mca_base_var_t *var);
> static int var_get (int vari, mca_base_var_t **var_out, bool original);
> static int var_value_string (mca_base_var_t *var, char **value_string);
> +static int mca_base_var_process_env_list(void);
>
> /*
> * classes
> @@ -255,11 +257,61 @@
> mca_base_var_initialized = true;
>
> mca_base_var_cache_files(false);
> +
> + /* set nesessary env variables for external usage */
> + mca_base_var_process_env_list();
> }
>
> return OPAL_SUCCESS;
> }
>
> +static int mca_base_var_process_env_list(void)
> +{
> + int i, ret;
> + char** tokens;
> + char* ptr;
> + char* param, *value;
> + ret = mca_base_var_register ("opal", "mca", "base", "env_list",
> + "Set SHELL env variables",
> + MCA_BASE_VAR_TYPE_STRING, NULL, 0, 0, OPAL_INFO_LVL_3,
> + MCA_BASE_VAR_SCOPE_READONLY, &mca_base_env_list);
> + if ((0 > ret) || (NULL == mca_base_env_list)) {
> + return OPAL_SUCCESS;
> + }
> + tokens = opal_argv_split(mca_base_env_list, '+');
> + if (NULL != tokens) {
> + for (i = 0; NULL != tokens[i]; i++) {
> + if (NULL == (ptr = strchr(tokens[i], '='))) {
> + value = getenv(tokens[i]);
> + if (NULL != value) {
> + if (NULL != strchr(value, '=')) {
> + param = strdup(value);
> + value = strchr(param, '=');
> + *value = '\0';
> + value++;
> + opal_setenv(param, value, true, &environ);
> + free(param);
> + } else {
> + opal_setenv(tokens[i], value, true, &environ);
> + }
> + } else {
> + opal_show_help("help-mca-var.txt", "incorrect-env-list-param",
> + true, tokens[i], mca_base_env_list);
> + }
> + } else {
> + param = strdup(tokens[i]);
> + value = strchr(param, '=');
> + *value = '\0';
> + value++;
> + opal_setenv(param, value, true, &environ);
> + free(param);
> + }
> + }
> + opal_argv_free(tokens);
> + }
> + return OPAL_SUCCESS;
> +}
> +
> static int mca_base_var_cache_files(bool rel_path_search)
> {
> char *tmp;
>
> Modified: trunk/orte/tools/orterun/help-orterun.txt
> ==============================================================================
> --- trunk/orte/tools/orterun/help-orterun.txt Tue Jul 8 20:10:04 2014 (r32162)
> +++ trunk/orte/tools/orterun/help-orterun.txt 2014-07-08 20:38:25 EDT (Tue, 08 Jul 2014) (r32163)
> @@ -649,4 +649,15 @@
> The job will now be aborted. Please check your code and/or
> adjust/remove the job execution time limit (as specified
> by MPIEXEC_TIMEOUT in your environment).
> -
> +#
> +[orterun:deprecated-env-set]
> +WARNING: The mechanism by which environment variables are passed to OMPI is changing!!
> +Specifically, beginning in the 1.9.x/2.0.x series, using "-x" to set environment
> +variables is deprecated. Please use the "mca_base_env_list" MCA parameter. With this new
> +mechanism, mpirun ... -x env_foo1=bar1 -x env_foo2=bar2 -x env_foo3 ...
> +becomes: mpirun ... -mca mca_base_env_list env_foo1=bar1+env_foo2=bar2+env_foo3 ...
> +#
> +[orterun:conflict-env-set]
> +ERROR: You have attempted to pass environment variables to OMPI with both the "-x" method (deprecated starting
> +in the 1.9.x/2.0.x series) and by setting the MCA parameter "mca_base_env_list". OMPI does not support mixing
> +these two methods. Please choose one method and try launching your job again. You job will now abort.
>
> Modified: trunk/orte/tools/orterun/orterun.c
> ==============================================================================
> --- trunk/orte/tools/orterun/orterun.c Tue Jul 8 20:10:04 2014 (r32162)
> +++ trunk/orte/tools/orterun/orterun.c 2014-07-08 20:38:25 EDT (Tue, 08 Jul 2014) (r32163)
> @@ -1722,6 +1722,13 @@
>
> /* Did the user request to export any environment variables on the cmd line? */
> if (opal_cmd_line_is_taken(&cmd_line, "x")) {
> + char* env_set_flag = getenv("OMPI_MCA_mca_base_env_list");
> + if (NULL != env_set_flag) {
> + orte_show_help("help-orterun.txt", "orterun:conflict-env-set", false);
> + return ORTE_ERR_FATAL;
> + } else {
> + orte_show_help("help-orterun.txt", "orterun:deprecated-env-set", false);
> + }
> j = opal_cmd_line_get_ninsts(&cmd_line, "x");
> for (i = 0; i < j; ++i) {
> param = opal_cmd_line_get_param(&cmd_line, "x", i, 0);
> _______________________________________________
> svn mailing list
> svn_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/svn