Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Allan Menezes (amenezes007_at_[hidden])
Date: 2005-10-17 20:23:53


users-request_at_[hidden] wrote:

>Send users mailing list submissions to
> users_at_[hidden]
>
>Today's Topics:
>
> 1. Re: Hpl Bench mark and Openmpi rc3 (Jeff Squyres)
>
>
>----------------------------------------------------------------------
>
>Message: 1
>Date: Mon, 17 Oct 2005 10:16:39 -0400
>From: Jeff Squyres <jsquyres_at_[hidden]>
>Subject: Re: [O-MPI users] Hpl Bench mark and Openmpi rc3
>To: Open MPI Users <users_at_[hidden]>
>Message-ID: <8557a377fe1f131e23274e10e5f6e250_at_[hidden]>
>Content-Type: text/plain; charset=US-ASCII; format=flowed
>
>On Oct 13, 2005, at 1:25 AM, Allan Menezes wrote:
>
>
>
>> I have a 16 node cluster of x86 machines with FC3 running on the
>>head
>>node. I used a beta version of OSCAR 4.2 for putting together the
>>cluster. It uses /home/allan as the NFS directory.
>>
>>
>
>Greetings Allan. Sorry for the delay in replying -- we were all at an
>Open MPI working meeting last week, and the schedule got a bit hectic.
>
>Your setup sounds find.
>
>
>
>>I tried Mpich2v1.02p1 and got abench mark of 26GFlops for it approx.
>>WIth open mpi 1.0RC3 having set the LD_LIBRARY_PATH in .bashrc and the
>>/opt/openmpi/bin path in .bash_profile in the home directory
>>
>>
>
>Two quick notes here:
>
>- Open MPI's mpirun supports the "--prefix" option which obviates
>needing to set these variables in your .bashrc (although setting them
>in permanently makes things easier in the long term -- you don't need
>to always specify --prefix). See the FAQ for more details on the
>--prefix option:
>
> http://www.open-mpi.org/faq/?category=running#mpirun-prefix
>
>- OSCAR makes use of environment modules; it contains setup to
>differentiate between the multiple different MPI implementations that
>OSCAR contains. You can trivially add a modulefile for Open MPI and
>therefore use the "switcher" command to easily switch between all the
>MPI implementations on your OSCAR cluster (once we hit 1.0, we
>anticipate having an OSCAR package).
>
>
>
>>I cannnot seeem to get a performance beyond 9 GFlops approximately.
>>The block size
>>for mpich2 was 120 for best results. For open mpi for N = 22000 I have
>>to use block sizes of 10 -11 to get a performance of 9GFlops other
>>wise for larger block sizes(NB) it's worse. I used the same N=22000
>>for mpich2 and have a 16 port Gigabit Netgear ethernet switch with
>>Gigabit realtek8169 ethernet cards. Can any one tell me why the
>>performance with open mpi is so low compared to mpich2-1.02p1?
>>
>>
>
>There should clearly not be such a wide disparity in performance here;
>we don't see this kind of difference in our own internal testing.
>
>Can you send the output of "ompi_info --all"?
>
>
>

Hi Jeff,
   I installed two versions of open mpi slightly different. One on
/opt/openmpi or I would get the gfortran error and the other in
/home/allan/openmpi
However I do not think that is the problem as the path names are
specified in the bahrc and bash_profile files of the /home/allan directory.
I also log into user allan who is not a superuser.On running the open
mpi with HPL I use the following command line:
a1> mpirun -mca pls_rsh_orted /home/allan/openmpi/bin/orted -hostfile aa
-np 16 ./xhpl
from the directory where xhpl resides such as /homer/open/bench and I
use the -mca command pls_rsh_orted as it otherwise comes up with an
error that it cannot find the ORTED daemon on machines a1, a2 etc. That
is probaly aconfiguration error. However the commands above and the
setup described works fine and there are no errors in the HPL.out file,
except that it is slow.
I use an atlas BLAS library for creating xhpl from hpl.tar.gz. The make
file for hpl uses the atlas libs and the open mpi mpicc compiler for
both compilation and linking. and I have zeroed out the MPI macro paths
in Make.open(that's what I reanmed the hpl makefile) for make arch=open
in hpl directory. Please find attached the ompi_info -all file as
requested. Thank you very much:
Allan


                Open MPI: 1.0rc3r7730
   Open MPI SVN revision: r7730
                Open RTE: 1.0rc3r7730
   Open RTE SVN revision: r7730
                    OPAL: 1.0rc3r7730
       OPAL SVN revision: r7730
              MCA memory: malloc_hooks (MCA v1.0, API v1.0, Component v1.0)
           MCA paffinity: linux (MCA v1.0, API v1.0, Component v1.0)
           MCA maffinity: first_use (MCA v1.0, API v1.0, Component v1.0)
               MCA timer: linux (MCA v1.0, API v1.0, Component v1.0)
           MCA allocator: basic (MCA v1.0, API v1.0, Component v1.0)
           MCA allocator: bucket (MCA v1.0, API v1.0, Component v1.0)
                MCA coll: basic (MCA v1.0, API v1.0, Component v1.0)
                MCA coll: self (MCA v1.0, API v1.0, Component v1.0)
                MCA coll: sm (MCA v1.0, API v1.0, Component v1.0)
                  MCA io: romio (MCA v1.0, API v1.0, Component v1.0)
               MCA mpool: sm (MCA v1.0, API v1.0, Component v1.0)
                 MCA pml: ob1 (MCA v1.0, API v1.0, Component v1.0)
                 MCA pml: teg (MCA v1.0, API v1.0, Component v1.0)
                 MCA pml: uniq (MCA v1.0, API v1.0, Component v1.0)
                 MCA ptl: self (MCA v1.0, API v1.0, Component v1.0)
                 MCA ptl: sm (MCA v1.0, API v1.0, Component v1.0)
                 MCA ptl: tcp (MCA v1.0, API v1.0, Component v1.0)
                 MCA btl: self (MCA v1.0, API v1.0, Component v1.0)
                 MCA btl: sm (MCA v1.0, API v1.0, Component v1.0)
                 MCA btl: tcp (MCA v1.0, API v1.0, Component v1.0)
                MCA topo: unity (MCA v1.0, API v1.0, Component v1.0)
                 MCA gpr: null (MCA v1.0, API v1.0, Component v1.0)
                 MCA gpr: proxy (MCA v1.0, API v1.0, Component v1.0)
                 MCA gpr: replica (MCA v1.0, API v1.0, Component v1.0)
                 MCA iof: proxy (MCA v1.0, API v1.0, Component v1.0)
                 MCA iof: svc (MCA v1.0, API v1.0, Component v1.0)
                  MCA ns: proxy (MCA v1.0, API v1.0, Component v1.0)
                  MCA ns: replica (MCA v1.0, API v1.0, Component v1.0)
                 MCA oob: tcp (MCA v1.0, API v1.0, Component v1.0)
                 MCA ras: dash_host (MCA v1.0, API v1.0, Component v1.0)
                 MCA ras: hostfile (MCA v1.0, API v1.0, Component v1.0)
                 MCA ras: localhost (MCA v1.0, API v1.0, Component v1.0)
                 MCA ras: slurm (MCA v1.0, API v1.0, Component v1.0)
                 MCA rds: hostfile (MCA v1.0, API v1.0, Component v1.0)
                 MCA rds: resfile (MCA v1.0, API v1.0, Component v1.0)
               MCA rmaps: round_robin (MCA v1.0, API v1.0, Component v1.0)
                MCA rmgr: proxy (MCA v1.0, API v1.0, Component v1.0)
                MCA rmgr: urm (MCA v1.0, API v1.0, Component v1.0)
                 MCA rml: oob (MCA v1.0, API v1.0, Component v1.0)
                 MCA pls: daemon (MCA v1.0, API v1.0, Component v1.0)
                 MCA pls: fork (MCA v1.0, API v1.0, Component v1.0)
                 MCA pls: proxy (MCA v1.0, API v1.0, Component v1.0)
                 MCA pls: rsh (MCA v1.0, API v1.0, Component v1.0)
                 MCA pls: slurm (MCA v1.0, API v1.0, Component v1.0)
                 MCA sds: env (MCA v1.0, API v1.0, Component v1.0)
                 MCA sds: pipe (MCA v1.0, API v1.0, Component v1.0)
                 MCA sds: seed (MCA v1.0, API v1.0, Component v1.0)
                 MCA sds: singleton (MCA v1.0, API v1.0, Component v1.0)
                 MCA sds: slurm (MCA v1.0, API v1.0, Component v1.0)
                  Prefix: /home/allan/openmpi
                  Bindir: /home/allan/openmpi/bin
                  Libdir: /home/allan/openmpi/lib
                  Incdir: /home/allan/openmpi/include
               Pkglibdir: /home/allan/openmpi/lib/openmpi
              Sysconfdir: /home/allan/openmpi/etc
 Configured architecture: i686-pc-linux-gnu
           Configured by: allan
           Configured on: Wed Oct 12 17:56:46 EDT 2005
          Configure host: a1.lightning.net
                Built by: allan
                Built on: Wed Oct 12 18:17:41 EDT 2005
              Built host: a1.lightning.net
              C bindings: yes
            C++ bindings: yes
      Fortran77 bindings: yes (all)
      Fortran90 bindings: yes
              C compiler: gcc
     C compiler absolute: /usr/bin/gcc
             C char size: 1
             C bool size: 1
            C short size: 2
              C int size: 4
             C long size: 4
            C float size: 4
           C double size: 8
          C pointer size: 4
            C char align: 1
            C bool align: 1
             C int align: 4
           C float align: 4
          C double align: 4
            C++ compiler: g++
   C++ compiler absolute: /usr/bin/g++
      Fortran77 compiler: gfortran
  Fortran77 compiler abs: /usr/bin/gfortran
      Fortran90 compiler: gfortran
  Fortran90 compiler abs: /usr/bin/gfortran
       Fort integer size: 4
      Fort have integer1: yes
      Fort have integer2: yes
      Fort have integer4: yes
      Fort have integer8: yes
     Fort have integer16: no
         Fort have real4: yes
         Fort have real8: yes
        Fort have real16: no
      Fort have complex8: yes
     Fort have complex16: yes
     Fort have complex32: no
      Fort integer1 size: 1
      Fort integer2 size: 2
      Fort integer4 size: 4
      Fort integer8 size: 8
     Fort integer16 size: -1
          Fort real size: 4
         Fort real4 size: 4
         Fort real8 size: 8
        Fort real16 size: -1
      Fort dbl prec size: 4
          Fort cplx size: 4
      Fort dbl cplx size: 4
         Fort cplx8 size: 8
        Fort cplx16 size: 16
        Fort cplx32 size: -1
      Fort integer align: 4
     Fort integer1 align: 1
     Fort integer2 align: 2
     Fort integer4 align: 4
     Fort integer8 align: 8
    Fort integer16 align: -1
         Fort real align: 4
        Fort real4 align: 4
        Fort real8 align: 8
       Fort real16 align: -1
     Fort dbl prec align: 4
         Fort cplx align: 4
     Fort dbl cplx align: 4
        Fort cplx8 align: 4
       Fort cplx16 align: 8
       Fort cplx32 align: -1
             C profiling: yes
           C++ profiling: yes
     Fortran77 profiling: yes
     Fortran90 profiling: yes
          C++ exceptions: no
          Thread support: posix (mpi: no, progress: no)
            Build CFLAGS: -O3 -DNDEBUG -fno-strict-aliasing -pthread
          Build CXXFLAGS: -O3 -DNDEBUG -finline-functions -pthread
            Build FFLAGS:
           Build FCFLAGS:
           Build LDFLAGS: -export-dynamic
              Build LIBS: -laio -lm -lutil -lnsl
    Wrapper extra CFLAGS: -pthread
  Wrapper extra CXXFLAGS: -pthread
    Wrapper extra FFLAGS: -pthread
   Wrapper extra FCFLAGS: -pthread
   Wrapper extra LDFLAGS:
      Wrapper extra LIBS: -lutil -lnsl -ldl -Wl,--export-dynamic -laio -lm -lutil -lnsl -ldl
  Internal debug support: no
     MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
         libltdl support: 1
                 MCA mca: parameter "mca_param_files" (current value: "/home/allan/.openmpi/mca-params.conf:/home/allan/openmpi/etc/openmpi-mca-params.conf")
                          Path for MCA configuration files containing default parameter values
                 MCA mca: parameter "mca_component_path" (current value: "/home/allan/openmpi/lib/openmpi:/home/allan/.openmpi/components")
                          Path where to look for Open MPI and ORTE components
                 MCA mca: parameter "mca_verbose" (current value: <none>)
                          Top-level verbosity parameter
                 MCA mca: parameter "mca_component_show_load_errors" (current value: "1")
                          Whether to show errors for components that failed to load or not
                 MCA mca: parameter "mca_component_disable_dlopen" (current value: "0")
                          Whether to attempt to disable opening dynamic components or not
                 MCA mpi: parameter "mpi_param_check" (current value: "1")
                          Whether you want MPI API parameters checked at run-time or not. Possible values are 0 (no checking) and 1 (perform checking at run-time)
                 MCA mpi: parameter "mpi_signal" (current value: <none>)
                          If a signal is received, display the stack trace frame
                 MCA mpi: parameter "mpi_yield_when_idle" (current value: "0")
                          Yield the processor when waiting for MPI communication (for MPI processes, will default to 1 when oversubscribing nodes)
                 MCA mpi: parameter "mpi_event_tick_rate" (current value: "-1")
                          How often to progress TCP communications (0 = never, otherwise specified in microseconds)
                 MCA mpi: parameter "mpi_show_handle_leaks" (current value: "0")
                          Whether MPI_FINALIZE shows all MPI handles that were not freed or not
                 MCA mpi: parameter "mpi_no_free_handles" (current value: "0")
                          Whether to actually free MPI objects when their handles are freed
                 MCA mpi: parameter "mpi_show_mca_params" (current value: "0")
                          Whether to show all MCA parameter value during MPI_INIT or not (good for reproducability of MPI jobs)
                 MCA mpi: parameter "mpi_show_mca_params_file" (current value: <none>)
                          If mpi_show_mca_params is true, setting this string to a valid filename tells Open MPI to dump all the MCA parameter values into a file suitable for reading via the mca_param_files parameter (good for reproducability of MPI jobs)
                 MCA mpi: parameter "mpi_paffinity_alone" (current value: "0")
                          If nonzero, assume that this job is the only (set of) process(es) running on each node and bind processes to processors, starting with processor ID 0
                 MCA mpi: parameter "mpi_leave_pinned" (current value: "0")
                          leave_pinned
                MCA orte: parameter "orte_debug" (current value: "0")
                          Whether or not to enable debugging output for all ORTE components (0 or 1)
              MCA memory: parameter "memory" (current value: <none>)
                          Default selection set of components for the memory framework (<none> means "use all components that can be found")
              MCA memory: parameter "memory_base_verbose" (current value: "0")
                          Verbosity level for the memory framework (0 = no verbosity)
              MCA memory: parameter "memory_malloc_hooks_priority" (current value: "0")
           MCA paffinity: parameter "paffinity" (current value: <none>)
                          Default selection set of components for the paffinity framework (<none> means "use all components that can be found")
           MCA paffinity: parameter "paffinity_linux_priority" (current value: "10")
                          Priority of the linux paffinity component
           MCA maffinity: parameter "maffinity" (current value: <none>)
                          Default selection set of components for the maffinity framework (<none> means "use all components that can be found")
           MCA maffinity: parameter "maffinity_first_use_priority" (current value: "10")
                          Priority of the first_use maffinity component
               MCA timer: parameter "timer" (current value: <none>)
                          Default selection set of components for the timer framework (<none> means "use all components that can be found")
               MCA timer: parameter "timer_base_verbose" (current value: "0")
                          Verbosity level for the timer framework (0 = no verbosity)
               MCA timer: parameter "timer_linux_priority" (current value: "0")
           MCA allocator: parameter "allocator" (current value: <none>)
                          Default selection set of components for the allocator framework (<none> means "use all components that can be found")
           MCA allocator: parameter "allocator_base_verbose" (current value: "0")
                          Verbosity level for the allocator framework (0 = no verbosity)
           MCA allocator: parameter "allocator_basic_priority" (current value: "0")
           MCA allocator: parameter "allocator_bucket_num_buckets" (current value: "30")
           MCA allocator: parameter "allocator_bucket_priority" (current value: "0")
                MCA coll: parameter "coll" (current value: <none>)
                          Default selection set of components for the coll framework (<none> means "use all components that can be found")
                MCA coll: parameter "coll_base_verbose" (current value: "0")
                          Verbosity level for the coll framework (0 = no verbosity)
                MCA coll: parameter "coll_basic_priority" (current value: "10")
                          Priority of the basic coll component
                MCA coll: parameter "coll_basic_crossover" (current value: "4")
                          Minimum number of processes in a communicator before using the logarithmic algorithms
                MCA coll: parameter "coll_self_priority" (current value: "75")
                MCA coll: parameter "coll_sm_priority" (current value: "0")
                          Priority of the sm coll component
                MCA coll: parameter "coll_sm_control_size" (current value: "4096")
                          Length of the control data -- should usually be either the length of a cache line on most SMPs, or the size of a page on machines that support direct memory affinity page placement (in bytes)
                MCA coll: parameter "coll_sm_bootstrap_filename" (current value: "coll-sm-bootstrap")
                          Filename (in the Open MPI session directory) of the coll sm component bootstrap rendezvous mmap file
                MCA coll: parameter "coll_sm_bootstrap_num_segments" (current value: "8")
                          Number of segments in the bootstrap file
                MCA coll: parameter "coll_sm_fragment_size" (current value: "8192")
                          Fragment size (in bytes) used for passing data through shared memory (will be rounded up to the nearest control_size size)
                MCA coll: parameter "coll_sm_mpool" (current value: "sm")
                          Name of the mpool component to use
                MCA coll: parameter "coll_sm_comm_in_use_flags" (current value: "2")
                          Number of "in use" flags, used to mark a message passing area segment as currently being used or not (must be >= 2 and <= comm_num_segments)
                MCA coll: parameter "coll_sm_comm_num_segments" (current value: "8")
                          Number of segments in each communicator's shared memory message passing area (must be >= 2, and must be a multiple of comm_in_use_flags)
                MCA coll: parameter "coll_sm_tree_degree" (current value: "4")
                          Degree of the tree for tree-based operations (must be => 1 and <= min(control_size, 255))
                MCA coll: information "coll_sm_shared_mem_used_bootstrap" (value: "160")
                          Amount of shared memory used in the shared memory bootstrap area (in bytes)
                MCA coll: parameter "coll_sm_info_num_procs" (current value: "4")
                          Number of processes to use for the calculation of the shared_mem_size MCA information parameter (must be => 2)
                MCA coll: information "coll_sm_shared_mem_used_data" (value: "548864")
                          Amount of shared memory used in the shared memory data area for info_num_procs processes (in bytes)
                  MCA io: parameter "io_base_freelist_initial_size" (current value: "16")
                          Initial MPI-2 IO request freelist size
                  MCA io: parameter "io_base_freelist_max_size" (current value: "64")
                          Max size of the MPI-2 IO request freelist
                  MCA io: parameter "io_base_freelist_increment" (current value: "16")
                          Increment size of the MPI-2 IO request freelist
                  MCA io: parameter "io" (current value: <none>)
                          Default selection set of components for the io framework (<none> means "use all components that can be found")
                  MCA io: parameter "io_base_verbose" (current value: "0")
                          Verbosity level for the io framework (0 = no verbosity)
                  MCA io: parameter "io_romio_priority" (current value: "10")
                          Priority of the io romio component
                  MCA io: parameter "io_romio_delete_priority" (current value: "10")
                          Delete priority of the io romio component
               MCA mpool: parameter "mpool" (current value: <none>)
                          Default selection set of components for the mpool framework (<none> means "use all components that can be found")
               MCA mpool: parameter "mpool_base_verbose" (current value: "0")
                          Verbosity level for the mpool framework (0 = no verbosity)
               MCA mpool: parameter "mpool_sm_size" (current value: "536870912")
               MCA mpool: parameter "mpool_sm_allocator" (current value: "bucket")
               MCA mpool: parameter "mpool_sm_priority" (current value: "0")
                 MCA pml: parameter "pml" (current value: "ob1")
                          Default selection set of components for the pml framework (<none> means "use all components that can be found")
                 MCA pml: parameter "pml_base_verbose" (current value: "0")
                          Verbosity level for the pml framework (0 = no verbosity)
                 MCA pml: parameter "pml_ob1_free_list_num" (current value: "4")
                 MCA pml: parameter "pml_ob1_free_list_max" (current value: "-1")
                 MCA pml: parameter "pml_ob1_free_list_inc" (current value: "64")
                 MCA pml: parameter "pml_ob1_priority" (current value: "1")
                 MCA pml: parameter "pml_ob1_eager_limit" (current value: "131072")
                 MCA pml: parameter "pml_ob1_send_pipeline_depth" (current value: "3")
                 MCA pml: parameter "pml_ob1_recv_pipeline_depth" (current value: "4")
                 MCA pml: parameter "pml_teg_free_list_num" (current value: "256")
                 MCA pml: parameter "pml_teg_free_list_max" (current value: "-1")
                 MCA pml: parameter "pml_teg_free_list_inc" (current value: "256")
                 MCA pml: parameter "pml_teg_poll_iterations" (current value: "100000")
                 MCA pml: parameter "pml_teg_priority" (current value: "0")
                 MCA pml: parameter "pml_uniq_free_list_num" (current value: "256")
                 MCA pml: parameter "pml_uniq_free_list_max" (current value: "-1")
                 MCA pml: parameter "pml_uniq_free_list_inc" (current value: "256")
                 MCA pml: parameter "pml_uniq_poll_iterations" (current value: "100000")
                 MCA pml: parameter "pml_uniq_priority" (current value: "0")
                 MCA ptl: parameter "ptl" (current value: <none>)
                          Default selection set of components for the ptl framework (<none> means "use all components that can be found")
                 MCA ptl: parameter "ptl_base_verbose" (current value: "0")
                          Verbosity level for the ptl framework (0 = no verbosity)
                 MCA ptl: parameter "ptl_self_buffer_size" (current value: "65536")
                 MCA ptl: parameter "ptl_self_nonblocking" (current value: "1")
                 MCA ptl: parameter "ptl_self_priority" (current value: "0")
                 MCA ptl: parameter "ptl_sm_first_frag_free_list_num" (current value: "256")
                 MCA ptl: parameter "ptl_sm_first_frag_free_list_max" (current value: "-1")
                 MCA ptl: parameter "ptl_sm_first_frag_free_list_inc" (current value: "256")
                 MCA ptl: parameter "ptl_sm_second_frag_free_list_num" (current value: "256")
                 MCA ptl: parameter "ptl_sm_second_frag_free_list_max" (current value: "-1")
                 MCA ptl: parameter "ptl_sm_second_frag_free_list_inc" (current value: "256")
                 MCA ptl: parameter "ptl_sm_max_procs" (current value: "-1")
                 MCA ptl: parameter "ptl_sm_sm_extra_procs" (current value: "2")
                 MCA ptl: parameter "ptl_sm_mpool" (current value: "sm")
                 MCA ptl: parameter "ptl_sm_first_fragment_size" (current value: "1024")
                 MCA ptl: parameter "ptl_sm_max_fragment_size" (current value: "8192")
                 MCA ptl: parameter "ptl_sm_fragment_alignment" (current value: "128")
                 MCA ptl: parameter "ptl_sm_size_of_cb_queue" (current value: "128")
                 MCA ptl: parameter "ptl_sm_cb_lazy_free_freq" (current value: "120")
                 MCA ptl: parameter "ptl_sm_priority" (current value: "0")
                 MCA ptl: parameter "ptl_tcp_if_include" (current value: <none>)
                 MCA ptl: parameter "ptl_tcp_if_exclude" (current value: "lo")
                 MCA ptl: parameter "ptl_tcp_free_list_num" (current value: "256")
                 MCA ptl: parameter "ptl_tcp_free_list_max" (current value: "-1")
                 MCA ptl: parameter "ptl_tcp_free_list_inc" (current value: "256")
                 MCA ptl: parameter "ptl_tcp_sndbuf" (current value: "131072")
                 MCA ptl: parameter "ptl_tcp_rcvbuf" (current value: "131072")
                 MCA ptl: parameter "ptl_tcp_exclusivity" (current value: "0")
                 MCA ptl: parameter "ptl_tcp_first_frag_size" (current value: "65536")
                 MCA ptl: parameter "ptl_tcp_min_frag_size" (current value: "65536")
                 MCA ptl: parameter "ptl_tcp_max_frag_size" (current value: "-1")
                 MCA ptl: parameter "ptl_tcp_frag_size" (current value: "65536")
                 MCA ptl: parameter "ptl_tcp_priority" (current value: "0")
                 MCA ptl: parameter "ptl_base_include" (current value: <none>)
                 MCA ptl: parameter "ptl_base_exclude" (current value: <none>)
                 MCA btl: parameter "btl_base_debug" (current value: "0")
                          If btl_base_debug is 1 standard debug is output, if > 1 verbose debug is output
                 MCA btl: parameter "btl" (current value: <none>)
                          Default selection set of components for the btl framework (<none> means "use all components that can be found")
                 MCA btl: parameter "btl_base_verbose" (current value: "0")
                          Verbosity level for the btl framework (0 = no verbosity)
                 MCA btl: parameter "btl_self_free_list_num" (current value: "0")
                 MCA btl: parameter "btl_self_free_list_max" (current value: "-1")
                 MCA btl: parameter "btl_self_free_list_inc" (current value: "32")
                 MCA btl: parameter "btl_self_eager_limit" (current value: "131072")
                 MCA btl: parameter "btl_self_max_send_size" (current value: "262144")
                 MCA btl: parameter "btl_self_max_rdma_size" (current value: "2147483647")
                 MCA btl: parameter "btl_self_exclusivity" (current value: "65536")
                 MCA btl: parameter "btl_self_flags" (current value: "2")
                 MCA btl: parameter "btl_self_priority" (current value: "0")
                 MCA btl: parameter "btl_sm_free_list_num" (current value: "256")
                 MCA btl: parameter "btl_sm_free_list_max" (current value: "-1")
                 MCA btl: parameter "btl_sm_free_list_inc" (current value: "256")
                 MCA btl: parameter "btl_sm_max_procs" (current value: "-1")
                 MCA btl: parameter "btl_sm_sm_extra_procs" (current value: "2")
                 MCA btl: parameter "btl_sm_mpool" (current value: "sm")
                 MCA btl: parameter "btl_sm_eager_limit" (current value: "1024")
                 MCA btl: parameter "btl_sm_max_frag_size" (current value: "8192")
                 MCA btl: parameter "btl_sm_size_of_cb_queue" (current value: "128")
                 MCA btl: parameter "btl_sm_cb_lazy_free_freq" (current value: "120")
                 MCA btl: parameter "btl_sm_priority" (current value: "0")
                 MCA btl: parameter "btl_tcp_if_include" (current value: <none>)
                 MCA btl: parameter "btl_tcp_if_exclude" (current value: "lo")
                 MCA btl: parameter "btl_tcp_free_list_num" (current value: "8")
                 MCA btl: parameter "btl_tcp_free_list_max" (current value: "-1")
                 MCA btl: parameter "btl_tcp_free_list_inc" (current value: "32")
                 MCA btl: parameter "btl_tcp_sndbuf" (current value: "131072")
                 MCA btl: parameter "btl_tcp_rcvbuf" (current value: "131072")
                 MCA btl: parameter "btl_tcp_exclusivity" (current value: "0")
                 MCA btl: parameter "btl_tcp_eager_limit" (current value: "65536")
                 MCA btl: parameter "btl_tcp_min_send_size" (current value: "65536")
                 MCA btl: parameter "btl_tcp_max_send_size" (current value: "262144")
                 MCA btl: parameter "btl_tcp_min_rdma_size" (current value: "262144")
                 MCA btl: parameter "btl_tcp_max_rdma_size" (current value: "2147483647")
                 MCA btl: parameter "btl_tcp_flags" (current value: "10")
                 MCA btl: parameter "btl_tcp_priority" (current value: "0")
                 MCA btl: parameter "btl_base_include" (current value: <none>)
                 MCA btl: parameter "btl_base_exclude" (current value: <none>)
                MCA topo: parameter "topo" (current value: <none>)
                          Default selection set of components for the topo framework (<none> means "use all components that can be found")
                MCA topo: parameter "topo_base_verbose" (current value: "0")
                          Verbosity level for the topo framework (0 = no verbosity)
              MCA errmgr: parameter "errmgr" (current value: <none>)
                          Default selection set of components for the errmgr framework (<none> means "use all components that can be found")
                 MCA gpr: parameter "gpr_base_maxsize" (current value: "2147483647")
                 MCA gpr: parameter "gpr_base_blocksize" (current value: "512")
                 MCA gpr: parameter "gpr" (current value: <none>)
                          Default selection set of components for the gpr framework (<none> means "use all components that can be found")
                 MCA gpr: parameter "gpr_null_priority" (current value: "0")
                 MCA gpr: parameter "gpr_proxy_debug" (current value: "0")
                 MCA gpr: parameter "gpr_proxy_priority" (current value: "0")
                 MCA gpr: parameter "gpr_replica_debug" (current value: "0")
                 MCA gpr: parameter "gpr_replica_isolate" (current value: "0")
                 MCA gpr: parameter "gpr_replica_priority" (current value: "0")
                 MCA iof: parameter "iof_base_window_size" (current value: "4096")
                 MCA iof: parameter "iof_base_service" (current value: "0.0.0")
                 MCA iof: parameter "iof" (current value: <none>)
                          Default selection set of components for the iof framework (<none> means "use all components that can be found")
                 MCA iof: parameter "iof_proxy_debug" (current value: "1")
                 MCA iof: parameter "iof_proxy_priority" (current value: "0")
                 MCA iof: parameter "iof_svc_debug" (current value: "1")
                 MCA iof: parameter "iof_svc_priority" (current value: "0")
                  MCA ns: parameter "ns" (current value: <none>)
                          Default selection set of components for the ns framework (<none> means "use all components that can be found")
                  MCA ns: parameter "ns_proxy_debug" (current value: "0")
                  MCA ns: parameter "ns_proxy_maxsize" (current value: "2147483647")
                  MCA ns: parameter "ns_proxy_blocksize" (current value: "512")
                  MCA ns: parameter "ns_proxy_priority" (current value: "0")
                  MCA ns: parameter "ns_replica_debug" (current value: "0")
                  MCA ns: parameter "ns_replica_isolate" (current value: "0")
                  MCA ns: parameter "ns_replica_maxsize" (current value: "2147483647")
                  MCA ns: parameter "ns_replica_blocksize" (current value: "512")
                  MCA ns: parameter "ns_replica_priority" (current value: "0")
                 MCA oob: parameter "oob" (current value: <none>)
                          Default selection set of components for the oob framework (<none> means "use all components that can be found")
                 MCA oob: parameter "oob_base_verbose" (current value: "0")
                          Verbosity level for the oob framework (0 = no verbosity)
                 MCA oob: parameter "oob_tcp_peer_limit" (current value: "-1")
                 MCA oob: parameter "oob_tcp_peer_retries" (current value: "60")
                 MCA oob: parameter "oob_tcp_debug" (current value: "0")
                 MCA oob: parameter "oob_tcp_include" (current value: <none>)
                 MCA oob: parameter "oob_tcp_exclude" (current value: <none>)
                 MCA oob: parameter "oob_tcp_priority" (current value: "0")
                 MCA ras: parameter "ras" (current value: <none>)
                          Default selection set of components for the ras framework (<none> means "use all components that can be found")
                 MCA ras: parameter "ras_dash_host_priority" (current value: "5")
                          Selection priority for the dash_host RAS component
                 MCA ras: parameter "ras_hostfile_priority" (current value: "10")
                          Selection priority for the hostfile RAS component
                 MCA ras: parameter "ras_localhost_priority" (current value: "0")
                          Selection priority for the localhost RAS component
                 MCA ras: parameter "ras_slurm_priority" (current value: "75")
                          Priority of the slurm ras component
                 MCA rds: parameter "rds" (current value: <none>)
                          Default selection set of components for the rds framework (<none> means "use all components that can be found")
                 MCA rds: parameter "rds_hostfile_debug" (current value: "0")
                          Toggle debug output for hostfile RDS component
                 MCA rds: parameter "rds_hostfile_path" (current value: "/home/allan/openmpi/etc/openmpi-default-hostfile")
                          ORTE Host filename
                 MCA rds: parameter "rds_hostfile_priority" (current value: "0")
                 MCA rds: parameter "rds_resfile_debug" (current value: "0")
                          Toggle debug output for resfile RDS component
                 MCA rds: parameter "rds_resfile_name" (current value: <none>)
                          ORTE Resource filename
                 MCA rds: parameter "rds_resfile_priority" (current value: "0")
               MCA rmaps: parameter "rmaps" (current value: <none>)
                          Default selection set of components for the rmaps framework (<none> means "use all components that can be found")
               MCA rmaps: parameter "rmaps_round_robin_debug" (current value: "1")
                          Toggle debug output for Round Robin RMAPS component
               MCA rmaps: parameter "rmaps_round_robin_priority" (current value: "1")
                          Selection priority for Round Robin RMAPS component
                MCA rmgr: parameter "rmgr" (current value: <none>)
                          Default selection set of components for the rmgr framework (<none> means "use all components that can be found")
                MCA rmgr: parameter "rmgr_proxy_priority" (current value: "0")
                MCA rmgr: parameter "rmgr_urm_priority" (current value: "0")
                 MCA rml: parameter "rml" (current value: <none>)
                          Default selection set of components for the rml framework (<none> means "use all components that can be found")
                 MCA rml: parameter "rml_base_verbose" (current value: "0")
                          Verbosity level for the rml framework (0 = no verbosity)
                 MCA rml: parameter "rml_oob_priority" (current value: "0")
                 MCA pls: parameter "pls" (current value: <none>)
                          Default selection set of components for the pls framework (<none> means "use all components that can be found")
                 MCA pls: parameter "pls_daemon_priority" (current value: "10")
                 MCA pls: parameter "pls_fork_reap" (current value: "1")
                 MCA pls: parameter "pls_fork_priority" (current value: "1")
                 MCA pls: parameter "pls_fork_debug" (current value: "0")
                 MCA pls: parameter "pls_proxy_priority" (current value: "1")
                 MCA pls: parameter "pls_rsh_debug" (current value: "0")
                          Whether or not to enable debugging output for the rsh pls component (0 or 1)
                 MCA pls: parameter "pls_rsh_num_concurrent" (current value: "128")
                          How many pls_rsh_agent instances to invoke concurrently (must be > 0)
                 MCA pls: parameter "pls_rsh_orted" (current value: "orted")
                          The command name that the rsh pls component will invoke for the ORTE daemon
                 MCA pls: parameter "pls_rsh_priority" (current value: "10")
                          Priority of the rsh pls component
                 MCA pls: parameter "pls_rsh_delay" (current value: "1")
                          Delay (in seconds) between invocations of the remote agent, but only used when the "debug" MCA parameter is true, or the top-level MCA debugging is enabled (otherwise this value is ignored)
                 MCA pls: parameter "pls_rsh_reap" (current value: "1")
                          If set to 1, wait for all the processes to complete before exiting. Otherwise, quit immediately -- without waiting for confirmation that all other processes in the job have completed.
                 MCA pls: parameter "pls_rsh_assume_same_shell" (current value: "1")
                          If set to 1, assume that the shell on the remote node is the same as the shell on the local node. Otherwise, probe for what the remote shell.
                 MCA pls: parameter "pls_rsh_agent" (current value: "ssh")
                          The command used to launch executables on remote nodes (typically either "rsh" or "ssh")
                 MCA pls: parameter "pls_slurm_debug" (current value: "0")
                          Enable debugging of slurm pls
                 MCA pls: parameter "pls_slurm_priority" (current value: "75")
                          Default selection priority
                 MCA pls: parameter "pls_slurm_orted" (current value: "orted")
                          Command to use to start proxy orted
                 MCA pls: parameter "pls_slurm_args" (current value: <none>)
                          Custom arguments to srun
                 MCA sds: parameter "sds" (current value: <none>)
                          Default selection set of components for the sds framework (<none> means "use all components that can be found")
                 MCA sds: parameter "sds_base_verbose" (current value: "0")
                          Verbosity level for the sds framework (0 = no verbosity)
                 MCA sds: parameter "sds_env_priority" (current value: "0")
                 MCA sds: parameter "sds_pipe_priority" (current value: "0")
                 MCA sds: parameter "sds_seed_priority" (current value: "0")
                 MCA sds: parameter "sds_singleton_priority" (current value: "0")
                 MCA sds: parameter "sds_slurm_priority" (current value: "0")
                 MCA soh: parameter "soh" (current value: <none>)
                          Default selection set of components for the soh framework (<none> means "use all components that can be found")