Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Devel mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-10-06 15:02:15


Ethan and I talked about this in Paris.

I think it's Whacky. :-) But hey, whacky can be good. :-)

I'm not a big fan of the --run-engine idea; my counter-proposal was
to make a --get-value option where you could query the value of any
field in any section (subject to full expansion rules). Hence, you
could have a shell script that does something like:

mtt --get-value "mpi_details: test-matrix" exec > stuff_to_run

Then iterate over the lines in stuff_to_run and execute each of them.

On Sep 29, 2007, at 12:04 AM, Ethan Mallove wrote:

> Folks,
>
> I had this wacky idea the other day.
>
> I have a collection of scripts that I've accumulated to run
> MTT with parameter combinations that MTT does not iterate on
> (see #245). E.g., I have a dozen or so scripts that do
> variations on something like this:
>
> Do a 32-bit mtt run with compiler X, on 2 nodes
> Do a 32-bit mtt run with compiler Y, on 2 nodes
> Do a 64-bit mtt run with compiler X, on 2 nodes
> Do a 64-bit mtt run with compiler Y, on 2 nodes
> Do a 32-bit mtt run with compiler X, on 1 node
> Do a 32-bit mtt run with compiler Y, on 1 node
> Do a 64-bit mtt run with compiler X, on 1 node
> Do a 64-bit mtt run with compiler Y, on 1 node
> ...
>
> So I thought, "can I somehow harness xargs to do this
> instead of creating another shell script every time I have
> another testing combination I want to run?" xargs doesn't
> seem powerful enough. Then I realized that the MTT run
> engine *itself* does exactly what I want. Xargs on steroids.
> So I created the following "MPI" details section to get MTT
> to run itself:
>
> [MPI Details: test-matrix]
> exec = <<EOT
> client/mtt
> --debug
> --trial
> --file @INI_NAME@
> --section "mpi.get.*ompi-nightly-trunk mtt
> mpi.details.*open.mpi reporter.*iu.database
> mpi.install.*autotools ..."
> --scratch /opt/mtt/@bitness@/@compiler@/&mpi_details_name()/
> my-scratch-area
> prepend_path="PATH /path/to/my/compiler/@compiler@"
> bitness=@bitness@
> run_on_single_node=@run_on_single_node@
> EOT
>
> bitness = &enumerate(32,64)
> run_on_single_node = &enumerate(0, 1)
> compiler = &enumerate("A", "B")
>
> The above expands to 8 MTT runs (2 compilers x 2 bitnesses x
> 2 hostlists). Then, I had to fake MTT out in order to *use*
> the run engine. So I created some Noop.pm modules that do
> nothing but scoot MTT along to the RunEngine phase:
>
> [MPI get: noop]
> # Fill this in from the command line
> mpi_details =
> module = Noop
> [MPI install: noop]
> mpi_get = noop
> module = Noop
> [Test get: noop]
> module = Noop
> [Test build: noop]
> test_get = noop
> module = Noop
> [Test run: noop]
> test_build = noop
> module = Noop
> timeout = -1
>
> Finally, I can kick off a series of MTT runs like this:
>
> $ client/mtt --section "noop test-matrix" mpi_details=test-matrix
>
> What do you think?
>
> I'd like to make this use case even easier. E.g., create a
> --run-engine (?) option for the specific purpose of skipping
> all the phases in order to simply execute commands. Or maybe
> there is another tool to do what I am trying to do? In
> which case, this has been an exercise :-)
>
> -Ethan
> _______________________________________________
> mtt-devel mailing list
> mtt-devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-devel

-- 
Jeff Squyres
Cisco Systems