Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Devel mailing list

From: Ethan Mallove (ethan.mallove_at_[hidden])
Date: 2007-09-28 18:04:38


Folks,

I had this wacky idea the other day.

I have a collection of scripts that I've accumulated to run
MTT with parameter combinations that MTT does not iterate on
(see #245). E.g., I have a dozen or so scripts that do
variations on something like this:

  Do a 32-bit mtt run with compiler X, on 2 nodes
  Do a 32-bit mtt run with compiler Y, on 2 nodes
  Do a 64-bit mtt run with compiler X, on 2 nodes
  Do a 64-bit mtt run with compiler Y, on 2 nodes
  Do a 32-bit mtt run with compiler X, on 1 node
  Do a 32-bit mtt run with compiler Y, on 1 node
  Do a 64-bit mtt run with compiler X, on 1 node
  Do a 64-bit mtt run with compiler Y, on 1 node
  ...

So I thought, "can I somehow harness xargs to do this
instead of creating another shell script every time I have
another testing combination I want to run?" xargs doesn't
seem powerful enough. Then I realized that the MTT run
engine *itself* does exactly what I want. Xargs on steroids.
So I created the following "MPI" details section to get MTT
to run itself:

  [MPI Details: test-matrix]
  exec = <<EOT
    client/mtt
        --debug
        --trial
        --file @INI_NAME@
        --section "mpi.get.*ompi-nightly-trunk mtt mpi.details.*open.mpi reporter.*iu.database mpi.install.*autotools ..."
        --scratch /opt/mtt/@bitness@/@compiler@/&mpi_details_name()/my-scratch-area
        prepend_path="PATH /path/to/my/compiler/@compiler@"
        bitness=@bitness@
        run_on_single_node=@run_on_single_node@
  EOT

  bitness = &enumerate(32,64)
  run_on_single_node = &enumerate(0, 1)
  compiler = &enumerate("A", "B")

The above expands to 8 MTT runs (2 compilers x 2 bitnesses x
2 hostlists). Then, I had to fake MTT out in order to *use*
the run engine. So I created some Noop.pm modules that do
nothing but scoot MTT along to the RunEngine phase:

  [MPI get: noop]
  # Fill this in from the command line
  mpi_details =
  module = Noop
  [MPI install: noop]
  mpi_get = noop
  module = Noop
  [Test get: noop]
  module = Noop
  [Test build: noop]
  test_get = noop
  module = Noop
  [Test run: noop]
  test_build = noop
  module = Noop
  timeout = -1

Finally, I can kick off a series of MTT runs like this:

  $ client/mtt --section "noop test-matrix" mpi_details=test-matrix

What do you think?

I'd like to make this use case even easier. E.g., create a
--run-engine (?) option for the specific purpose of skipping
all the phases in order to simply execute commands. Or maybe
there is another tool to do what I am trying to do? In
which case, this has been an exercise :-)

-Ethan