Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

From: Ethan Mallove (ethan.mallove_at_[hidden])
Date: 2007-09-28 18:04:38


Folks,

I had this wacky idea the other day.

I have a collection of scripts that I've accumulated to run
MTT with parameter combinations that MTT does not iterate on
(see #245). E.g., I have a dozen or so scripts that do
variations on something like this:

  Do a 32-bit mtt run with compiler X, on 2 nodes
  Do a 32-bit mtt run with compiler Y, on 2 nodes
  Do a 64-bit mtt run with compiler X, on 2 nodes
  Do a 64-bit mtt run with compiler Y, on 2 nodes
  Do a 32-bit mtt run with compiler X, on 1 node
  Do a 32-bit mtt run with compiler Y, on 1 node
  Do a 64-bit mtt run with compiler X, on 1 node
  Do a 64-bit mtt run with compiler Y, on 1 node
  ...

So I thought, "can I somehow harness xargs to do this
instead of creating another shell script every time I have
another testing combination I want to run?" xargs doesn't
seem powerful enough. Then I realized that the MTT run
engine *itself* does exactly what I want. Xargs on steroids.
So I created the following "MPI" details section to get MTT
to run itself:

  [MPI Details: test-matrix]
  exec = <<EOT
    client/mtt
        --debug
        --trial
        --file @INI_NAME@
        --section "mpi.get.*ompi-nightly-trunk mtt mpi.details.*open.mpi reporter.*iu.database mpi.install.*autotools ..."
        --scratch /opt/mtt/@bitness@/@compiler@/&mpi_details_name()/my-scratch-area
        prepend_path="PATH /path/to/my/compiler/@compiler@"
        bitness=@bitness@
        run_on_single_node=@run_on_single_node@
  EOT

  bitness = &enumerate(32,64)
  run_on_single_node = &enumerate(0, 1)
  compiler = &enumerate("A", "B")

The above expands to 8 MTT runs (2 compilers x 2 bitnesses x
2 hostlists). Then, I had to fake MTT out in order to *use*
the run engine. So I created some Noop.pm modules that do
nothing but scoot MTT along to the RunEngine phase:

  [MPI get: noop]
  # Fill this in from the command line
  mpi_details =
  module = Noop
  [MPI install: noop]
  mpi_get = noop
  module = Noop
  [Test get: noop]
  module = Noop
  [Test build: noop]
  test_get = noop
  module = Noop
  [Test run: noop]
  test_build = noop
  module = Noop
  timeout = -1

Finally, I can kick off a series of MTT runs like this:

  $ client/mtt --section "noop test-matrix" mpi_details=test-matrix

What do you think?

I'd like to make this use case even easier. E.g., create a
--run-engine (?) option for the specific purpose of skipping
all the phases in order to simply execute commands. Or maybe
there is another tool to do what I am trying to do? In
which case, this has been an exercise :-)

-Ethan