Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Devel mailing list

Subject: [MTT devel] MTToGDS
From: Mike Dubman (mike.ompi_at_[hidden])
Date: 2009-09-29 15:21:30


Hello guys and gals,

We have completed development and testing of Google DataStore support in MTT
and are glad to submit it for community tests.

New Files:

========

The following new files were added to support GDS inside MTT:

1. client/bquery.pl

   Perl-based GDS client, provides basic DB querying/fetching capabilities.
It creates resultset (files in YAML format) from user-provided sql-like
query

2. client/breport.pl

   Perl-based report tool, creates excel reports from yaml files, generated
by bquery.pl tool.

3. client/custom_launchers/

   For brave only: custom launchers for non-standard HPC, mpi-based
applications

4. lib/MTT/Reporter/MTTGDS.pm

   GDS Reporter, saves mtt results to GDS (see samples/gds-demo.ini for
configuration examples)

5. lib/MTT/Utils/ClusterInfo.pm

   Helper library to gather node hw/sw configuration information which is
saved in GDS together with tests results.

6. New TestResults analyzers for HPC applications:

   lib/MTT/test/Analyze/Performance/Fluent.pm

   lib/MTT/test/Analyze/Performance/HPCC.pm

   lib/MTT/test/Analyze/Performance/HPLGDS.pm

   lib/MTT/test/Analyze/Performance/OpenFoam.pm

   lib/MTT/test/Analyze/Performance/PamCrash.pm

7. samples/gds-demo.ini

   Example of howto configure GDS in MTT and run bquery/breport tools at the
end of MTT session

8. server/gds/

   GDS backend part, which is running at Google and providing Object to
YAML, YAML to Object translation service as well as helper code for
bquery.pl DB client.

9. docs/gds/

   Various documentation

Known Issues and Limitations:

======================

* lib/MTT/Utils/ClusterInfo.pm uses "sudo" command to gather node`s hardware
information.

* When using client/custom_launchers/ to run tests, it is impossible to kill
the test application when timeout reached.

How to start using MTToGDS:

======================

* Contact Jeff to provide you with GDS login/password which is needed for
querying/saving to DB
(*http://open-mpi-mtt.appspot.com*>
)

* See samples/gds-demo.ini for configuration examples as well as for DB
querying and reports generation.

* Read Google GQL syntax documentation and use it with bquery.pl in order to
query&fetch objects from GDB.
* The following perl modules are required for all MTToGDS components:
 libYAML
YAML::Syck
YAML::XS

for breport:
GD::Graph
Spreadsheet::WriteExcel

You can install it on linux systems with yum as following:
yum install perl-libyaml perl-YAML-Syck perl-YAML-XS perl-GD-Graph
perl-Spreadsheet-WriteExcel

Special Thanks to:

==============

Igor Ivanov, Andrew Senin, Alexander Alekhin from Argus-Cv.com for they
contribution in developing and testing of this feature!

Regards

Mike