Open MPI logo

MTT Users Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [MTT users] MTT Error on SLES11
From: Rafael Folco (rfolco_at_[hidden])
Date: 2009-04-08 16:05:30


Ethan, Jeff,

Sorry for the perl confusion, nevermind...

Below is attached the log and ini files, and also how I run MTT.

Thanks,

Rafael

client/mtt -d -v -p -f openmpi.ini 2>&1 |tee test.log

...

*** Run test phase starting
>> Test run [trivial]
Evaluating: trivial
Found a match! trivial [trivial
Evaluating: Simple
>> Running with [openmpi-1.2.8] / [1.2.8] / [openmpi-1.2.8]
Found MPI details: [mpi details: open mpi]
Using [mpi details: open mpi] with [MPI Install: openmpi-1.2.8]
Evaluating: # We can exit if the test passed or was skipped (i.e.,
there's
   no need
   # to cleanup).
   if test "$MTT_TEST_RUN_RESULT" = "passed" -o "$MTT_TEST_RUN_RESULT" =
   "skipped"; then
       exit 0
   fi
   
   if test "$MTT_TEST_HOSTFILE" != ""; then
       args="--hostfile $MTT_TEST_HOSTFILE"
   elif test "$MTT_TEST_HOSTLIST" != ""; then
       args="--host $MTT_TEST_HOSTLIST"
   fi
   orterun $args -np $MTT_TEST_NP --prefix $MTT_TEST_PREFIX
   mtt_ompi_cleanup.pl
Got final exec: mpirun &if(&have_hostfile(), "&join("--hostfile ",
   "&hostfile()")", "&if(&have_hostlist(), "&join("--host ",
   "&hostlist()")", "")") -np &test_np() --mca btl openib,self --debug
   --prefix &test_prefix() &test_executable() &test_argv()
chdir /tmp/ompi-core-testers/installs/dLS2/tests/trivial/test_get__trivial
Evaluating: require MTT::Test::Specify::Simple
Evaluating: $ret = &MTT::Test::Specify::Simple::Specify(@args)
Evaluating: &find_executables(".")
Got name: find_executables
Got args: "."
_do: $ret = MTT::Values::Functions::find_executables(".")
&find_executables got .
&find_exectuables returning: ./c_ring ./f77_ring ./f90_hello ./cxx_ring
   ./f77_hello ./cxx_hello ./f90_ring ./c_hello
*** ERROR: Module aborted: MTT::Test::Specify::Simple:Specify: Can't use
    string ("8") as an ARRAY ref while "strict refs" in use at
    /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.

On Wed, 2009-04-08 at 14:15 -0400, Ethan Mallove wrote:
> On Wed, Apr/08/2009 11:36:05AM, Rafael Folco wrote:
> > Well, I took a look at /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> >
> > This piece of code looks wrong to me:
> >
> > if ($#{@$ret} < 0) {
> >
> > $ret references an array
> > @$ret points to the first element of this array
> > $# returns the number of elements
> >
> > So this line is trying to count elements of the first element??! Doesn't
> > make sense. Correct me if I am wrong, what am I missing here ?
> >
> > "if ($#{$ret} < 0) {" would be correct, without @.
> >
> > I believe "strict refs" has been forced somewhere on SLES11... I also
> > tried on other distro and it works fine.
>
> How do perl -V differ between the two distros?
>
> I can not reproduce the error on a SLES 10 machine.
>
> Could you run MTT with the --debug option and send the output with the
> line 75 perl error? That might help me determine which INI param is
> responsible for the error.
>
> -Ethan
>
> >
> > Thanks,
> >
> > Rafael
> >
> > On Tue, 2009-04-07 at 15:53 -0300, Rafael Folco wrote:
> > > Hi,
> > >
> > > I'm trying to run MTT on SLES11, but I am getting an error message
> > > during the RUN phase and I can't figure out what is the problem.
> > >
> > > *** ERROR: Module aborted: MTT::Test::Specify::Simple:Specify: Can't use
> > > string ("183") as an ARRAY ref while "strict refs" in use at
> > > /tmp/ompi-core-testers/lib/MTT/Values.pm line 75.
> > >
> > > What I could see was that this error is nothing specific to any
> > > particular test, it happens at certain points during the RUN phase.
> > > Also, the BUILD phase has been completed successfully for all tests.
> > >
> > > Anybody have already seen this? Any thoughts ?
> > >
> > > Thanks in advance.
> > >
> > > Rafael
> > >
> > >
> >
> > --
> > Rafael Folco
> > OpenHPC / Test Lead
> > IBM Linux Technology Center
> > E-Mail: rfolco_at_[hidden]
> >
> > _______________________________________________
> > mtt-users mailing list
> > mtt-users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users

-- 
Rafael Folco
OpenHPC / Test Lead
IBM Linux Technology Center
E-Mail: rfolco_at_[hidden]

#======================================================================
# Overall configuration
#======================================================================

[MTT]

# OMPI Core: if you are not running in a scheduled environment and you
# have a fixed hostfile for what nodes you'll be running on, fill in
# the absolute pathname to it here. If you do not have a hostfile,
# leave it empty. Example:
# hostfile = /home/me/mtt-runs/mtt-hostfile
# This file will be parsed and will automatically set a valid value
# for &env_max_np() (it'll count the number of lines in the hostfile,
# adding slots/cpu counts if it finds them). The "hostfile" value is
# ignored if you are running in a recognized scheduled environment.
hostfile =

# OMPI Core: if you would rather list the hosts individually on the
# mpirun command line, list hosts here delimited by whitespace (if you
# have a hostfile listed above, this value will be ignored!). Hosts
# can optionally be suffixed with ":num", where "num" is an integer
# indicating how many processes may be started on that machine (if not
# specified, ":1" is assumed). The sum of all of these values is used
# for &env_max_np() at run time. Example (4 uniprocessors):
# hostlist = node1 node2 node3 node4
# Another example (4 2-way SMPs):
# hostlist = node1:2 node2:2 node3:2 node4:2
# The "hostlist" value is ignored if you are running in a scheduled
# environment or if you have specified a hostfile.
hostlist = cluster-ib-6 cluster-ib-7

# OMPI Core: if you are running in a scheduled environment and want to
# override the scheduler and set the maximum number of processes
# returned by &env_max_procs(), you can fill in an integer here.
max_np =

# OMPI Core: Output display preference; the default width at which MTT
# output will wrap.
textwrap = 76

# OMPI Core: After the timeout for a command has passed, wait this
# many additional seconds to drain all output, and then kill it with
# extreme prejiduce.
drain_timeout = 5

# OMPI Core: Whether this invocation of the client is a test of the
# client setup itself. Specifically, this value should be set to true
# (1) if you are testing your MTT client and/or INI file and do not
# want the results included in normal reporting in the MTT central
# results database. Results submitted in "trial" mode are not
# viewable (by default) on the central database, and are automatically
# deleted from the database after a short time period (e.g., a week).
# Setting this value to 1 is exactly equivalent to passing "--trial"
# on the MTT client command line. However, any value specified here
# in this INI file will override the "--trial" setting on the command
# line (i.e., if you set "trial = 0" here in the INI file, that will
# override and cancel the effect of "--trial" on the command line).
trial = 1

# OMPI Core: Set the scratch parameter here (if you do not want it to
# be automatically set to your current working directory). Setting
# this parameter accomplishes the same thing that the --scratch option
# does.
# scratch = &getenv("HOME")/mtt-scratch

# OMPI Core: Set local_username here if you would prefer to not have
# your local user ID in the MTT database
local_username = root

# OMPI Core: --force can be set here, instead of at the command line.
# Useful for a developer workspace in which it makes no sense to not
# use --force
force = 1

logfile = &getenv("HOME")/mtt.log

#======================================================================
# MPI run details
#======================================================================

[MPI Details: Open MPI]
exec = mpirun @hosts@ -np &test_np() --mca btl openib,self --debug --prefix &test_prefix() &test_executable() &test_argv()

# Yes, all these quotes are necessary. Don't mess with them!
hosts = &if(&have_hostfile(), "&join("--hostfile ", "&hostfile()")", \
            "&if(&have_hostlist(), "&join("--host ", "&hostlist()")", "")")

# Example showing conditional substitution based on the MPI get
# section name (e.g., different versions of OMPI have different
# capabilities / bugs).
mca = &enumerate( \
        "--mca btl sm,tcp,self_at_v1_1_mca_params@", \
        "--mca btl tcp,self_at_v1_1_mca_params@")

# OMPI v1.1 cannot handle heterogeneous numbers of TCP or OpenIB
# interfaces within a single job. So restrict it to a finite number
# that will be the same across all processes in the job (adjust for
# your own site, of course -- this particular example is meaningless
# if all nodes at your site have a homogeneous type and number of
# network interfaces).
v1_1_mca_params = &if(&eq("&mpi_get_name()", "ompi-nightly-v1.1"), \
        " --mca btl_tcp_if_include eth0 --mca oob_tcp_if_include eth0 --mca btl_openib_max_btls 1", "")

#----------------------------------------------------------------------
# WARNING: THIS DEFAULT after_each_exec STEP IS PRONE TO FAILURE!
# Given that part of what we are testing is ORTE itself, using orterun
# to launch something to cleanup can be problematic. We *HIGHLY*
# recommend that you replace the after_each_exec section default value
# below with something that your run-time system can performan
# natively. For example, putting "srun -N $SLURM_NNODES killall -9
# mpirun orted &test_executable()" works nicely on SLURM / Linux
# systems -- assuming that your MTT run has all nodes exclusively to
# itself (i.e., that the "killall" won't kill some legitimate jobs).
#----------------------------------------------------------------------

# A helper script is installed by the "OMPI" MPI Install module named
# "mtt_ompi_cleanup.pl". This script is orterun-able and will kill
# all rogue orteds on a node and whack any session directories.
# Invoke via orterun just to emphasize that it is not an MPI
# application. The helper script is installed in OMPI's bin dir, so
# it'll automatically be found in the path (because OMPI's bin dir is
# in the path).

after_each_exec = <<EOT
# We can exit if the test passed or was skipped (i.e., there's no need
# to cleanup).
if test "$MTT_TEST_RUN_RESULT" = "passed" -o "$MTT_TEST_RUN_RESULT" = "skipped"; then
    exit 0
fi

if test "$MTT_TEST_HOSTFILE" != ""; then
    args="--hostfile $MTT_TEST_HOSTFILE"
elif test "$MTT_TEST_HOSTLIST" != ""; then
    args="--host $MTT_TEST_HOSTLIST"
fi
orterun $args -np $MTT_TEST_NP --prefix $MTT_TEST_PREFIX mtt_ompi_cleanup.pl
EOT

[MPI get: openmpi-1.2.8]
mpi_details = openmpi-1.2.8
alreadyinstalled_dir = /usr/lib64/mpi/gcc/openmpi
module = AlreadyInstalled

[MPI install: openmpi-1.2.8]
module = Analyze::OMPI
mpi_get = openmpi-1.2.8

#======================================================================
# Test get phase
#======================================================================

[Test get: trivial]
module = Trivial
 
#----------------------------------------------------------------------

[Test get: ibm]
module = SVN
svn_username = ******
svn_password = ******
svn_url = https://svn.open-mpi.org/svn/ompi-tests/trunk/ibm
svn_post_export = <<EOT
./autogen.sh
EOT

#----------------------------------------------------------------------

[Test get: onesided]
module = SVN
svn_username = ******
svn_password = ******
svn_url = https://svn.open-mpi.org/svn/ompi-tests/trunk/onesided
svn_post_export = <<EOT
./autogen.sh
EOT

#----------------------------------------------------------------------

[Test get: mpicxx]
module = SVN
svn_username = ******
svn_password = ******
svn_url = https://svn.open-mpi.org/svn/ompi-tests/trunk/cxx-test-suite
svn_post_export = <<EOT
./autogen.sh
EOT

#----------------------------------------------------------------------

[Test get: imb]
module = SVN
svn_username = ******
svn_password = ******
svn_url = https://svn.open-mpi.org/svn/ompi-tests/trunk/IMB_2.3

#----------------------------------------------------------------------

[Test get: netpipe]
module = SVN
svn_username = ******
svn_password = ******
svn_url = https://svn.open-mpi.org/svn/ompi-tests/trunk/NetPIPE_3.6.2

#======================================================================
# Test build phase
#======================================================================

[Test build: trivial]
test_get = trivial
save_stdout_on_success = 1
merge_stdout_stderr = 1
stderr_save_lines = 100
merge_stdout_stderr = 1
stderr_save_lines = 100

module = Trivial

#----------------------------------------------------------------------

[Test build: ibm]
test_get = ibm
save_stdout_on_success = 1
merge_stdout_stderr = 1
stderr_save_lines = 100

module = Shell
shell_build_command = <<EOT
./configure CC=mpicc CXX=mpic++ F77=mpif77
make
EOT

#----------------------------------------------------------------------

[Test build: onesided]
test_get = onesided
save_stdout_on_success = 1
merge_stdout_stderr = 1
stderr_save_lines = 100
# Have the onesided tests skip the OMPI 1.1 testing; MPI-2 one-sided
# just plain doesn't work there and won't be fixed.
skip_mpi_get = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
skip_mpi_install = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
# Can also have a skip_mpi_install for the same purpose (skip specific
# installs)

module = Shell
shell_build_command = <<EOT
./configure
make
EOT

#----------------------------------------------------------------------

[Test build: mpicxx]
test_get = mpicxx
save_stdout_on_success = 1
merge_stdout_stderr = 1

skip_mpi_get = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
skip_mpi_install = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
module = Shell
shell_build_command = <<EOT
./configure CC=mpicc CXX=mpic++
make
EOT

#----------------------------------------------------------------------

[Test build: imb]
test_get = imb
save_stdout_on_success = 1
merge_stdout_stderr = 1
stderr_save_lines = 100

skip_mpi_get = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
skip_mpi_install = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
module = Shell
shell_build_command = <<EOT
cd src
make clean IMB-MPI1
EOT

#----------------------------------------------------------------------

[Test build: netpipe]
test_get = netpipe
save_stdout_on_success = 1
merge_stdout_stderr = 1
stderr_save_lines = 100

skip_mpi_get = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
skip_mpi_install = ompi-nightly-v1.1, ompi-nightly-trunk, ompi-nightly-v1.2
module = Shell
shell_build_command = <<EOT
make mpi
EOT

#======================================================================
# Test Run phase
#======================================================================

[Test run: trivial]
test_build = trivial
pass = &and(&cmd_wifexited(), &eq(&cmd_wexitstatus(), 0))
timeout = &test_np()
save_stdout_on_pass = 1
merge_stdout_stderr = 1
stdout_save_lines = 100
np = &env_max_procs()
 
specify_module = Simple
simple_only:tests = &find_executables(".")

#----------------------------------------------------------------------

[Test run: ibm]
test_build = ibm
pass = &and(&cmd_wifexited(), &eq(&cmd_wexitstatus(), 0))
skipped = &and(&cmd_wifexited(), &eq(&cmd_wexitstatus(), 77))
timeout = &max(30, &multiply(10, &test_np()))
save_stdout_on_pass = 1
merge_stdout_stderr = 1
stdout_save_lines = 100
np = &env_max_procs()
specify_module = Simple
# Similar rationale to the intel test run section
simple_first:tests = &find_executables("collective", "communicator", \
                                       "datatype", "dynamic", "environment", \
                                       "group", "info", "io", "onesided", \
                                       "pt2pt", "topology")

# Similar rationale to the intel test run section
simple_fail:tests = environment/abort environment/final
simple_fail:pass = &and(&cmd_wifexited(), &ne(&cmd_wexitstatus(), 0))
simple_fail:exclusive = 1
simple_fail:np = &env_max_procs()

#----------------------------------------------------------------------

[Test run: onesided]
test_build = onesided
pass = &and(&cmd_wifexited(), &eq(&cmd_wexitstatus(), 0))
timeout = &max(30, &multiply(10, &test_np()))
save_stdout_on_pass = 1
merge_stdout_stderr = 1
stdout_save_lines = 100
np = &if(&gt(&env_max_procs(), 0), &step(2, &max(2, &env_max_procs()), 2), 2)
specify_module = Simple
simple_pass:tests = &cat("run_list")

#----------------------------------------------------------------------

[Test run: mpicxx]
test_build = mpicxx
pass = &and(&cmd_wifexited(), &eq(&cmd_wexitstatus(), 0))
timeout = &max(30, &multiply(10, &test_np()))
save_stdout_on_pass = 1
merge_stdout_stderr = 1
argv = &if(&eq("&mpi_get_name()", "ompi-nightly-v1.1"), "-nothrow", "")
np = &env_max_procs()
specify_module = Simple
simple_pass:tests = src/mpi2c++_test

#----------------------------------------------------------------------
     
[Test run: imb correctness]
test_build = imb
pass = &and(&cmd_wifexited(), &eq(&cmd_wexitstatus(), 0))
timeout = &max(1800, &multiply(50, &test_np()))
save_stdout_on_pass = 1
merge_stdout_stderr = 1
stdout_save_lines = 100
np = &env_max_procs()
specify_module = Simple
simple_only:tests = src/IMB-MPI1

#----------------------------------------------------------------------

[Test run: imb performance]
test_build = imb
pass = &eq(&cmd_wexitstatus(), 0)
timeout = -1
save_stdout_on_pass = 1
# Ensure to leave this value as "-1", or performance results could be lost!
stdout_save_lines = -1
merge_stdout_stderr = 1
np = &env_max_procs()
argv = -npmin &test_np() &enumerate("PingPong", "PingPing", "Sendrecv", "Exchange", "Allreduce", "Reduce", "Reduce_scatter", "Allgather", "Allgatherv", "Alltoall", "Bcast", "Barrier")

specify_module = Simple
analyze_module = IMB
simple_pass:tests = src/IMB-MPI1

#----------------------------------------------------------------------

[Test run: netpipe]
test_build = netpipe
pass = &eq(&cmd_wexitstatus(), 0)
timeout = -1
save_stdout_on_pass = 1
# Ensure to leave this value as "-1", or performance results could be lost!
stdout_save_lines = -1
merge_stdout_stderr = 1
# NetPIPE is ping-pong only, so we only need 2 procs
np = 2

specify_module = Simple
analyze_module = NetPipe
simple_pass:tests = NPmpi

#======================================================================
# Reporter phase
#======================================================================

# This is a backup for while debugging MTT; it also writes results to
# a local text file

[Reporter: text file backup]
module = TextFile

textfile_filename = $phase-$section-$mpi_name-$mpi_version.txt

textfile_summary_header = <<EOT
hostname: &shell("hostname")
uname: &shell("uname -a")
who am i: &shell("who am i")
EOT

textfile_summary_footer =
textfile_detail_header =
textfile_detail_footer =

textfile_textwrap = 78