Subject: Re: [MTT users] test suites and np question
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2016-05-18 09:20:56
On May 18, 2016, at 7:11 AM, Kawashima, Takahiro <t-kawashima_at_[hidden]> wrote:
>
> Fujitsu started to try MTT + ompi-tests on our machines.
> With the sample .ini file, we wrote our .ini file and some
> test suites are run.
>
> I have two questions.
>
> (a) There are many test suites (directories) in ompi-tests.
> ibm, onesided, sun, ...
> Which test suites should I use to participate in
> OMPI MTT daily/weekly run?
The general guidance is: run as many tests as you have resources for. Meaning: we'll take any testing you can give. :-)
Have a look in ompi-tests:cisco/mtt/community/*.ini and cisco/mtt/usnic/*.ini. Those are the ini files I use every night for Cisco usNIC-specific testing and community-wide testing. You can see the results of them in the MTT community reporter:
I generally aim for about 20-24 hours of testing. It's a little fuzzy, because Cisco's MTT will only fire for a given version (I'm currently testing the master, v1.10, and v2.x branches) if there were new commits that day (i.e., if there's a new nightly snapshot tarball since the last run).
If you run too many tests such that your testing is more than 24 hours, then your resources quickly get behind and you're testing tarballs from days ago -- and that's not too useful.
> (b) What is the recommended `np` value (number of processes)?
> Should I use the largest `np` I can run?
Yes, subject to what I mentioned above: you want to aim for a total of ~24 hours of testing so that you can start the next cycle with the next night's snapshot tarball.
You can pack this in however you want -- do lots of small-np-value tests and a few large-np-value tests (just to sanity test large np values, etc.), etc.
You can also take into account that little development is done on the weekends. For example, you might want to aim for ~24 hours of testing on Monday-Thursday evenings, and then aim for a 3-day run on Friday evening (because there might not be new tarballs generated over the weekend).
> Does it depend on test suites?
Yes. Some test suites have upper-bounds on the number of processes they can run. IIRC, the Intel test suite, for example, can only run up to 64 processes (because of some hard-coded array sizes) unless you use a specific -D to compile it (that increases the size of these arrays).
-- Jeff Squyres jsquyres_at_[hidden] For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/