Hello Guys,

I have played a bit with google datastore and here is a proposal for mtt DB infra and some accompanying tools for submission and querying:


1. Scope and requirements
====================

a. provide storage services for storing test results generated by mtt. Storage services will be implemented over datastore.
b. provide storage services for storing benchmarking results generated by various mpi based applications  (not mtt based, for example: fluent, openfoam)
c. test or benchmarking results stored in the datastore can be grouped and referred as a group (for example: mtt execution can generate many mtt results consisting of different phases. This mtt execution will be referred as a session)
d. Benchmarking and test results which are generated by mtt or any other mpi based applications, can be stored in the datastore and grouped by some logical criteria.
e. The mtt should not depend or call directly any datastore`s provided APIs. The mtt client (or framework/scripts executing mpi based applications) should generate test/benchmarking results in some internal format, which will be processed later by external tools. These external tools will be responsible for saving test results in the datastore. Same rules should be applied for non mtt based executions of mpi-based applications (line fluent, openfoam,...). The scripts which are wrapping such executions will dump benchmarking results in some internal form for later processing by external tools.

f. The internal form for representation of test/benchmarking results can be XML. The external tool will receive (as cmd line params) XML files, process them and save to the datastore.

d. The external tools will be familiar with datastore object model and will provide bridge between test results (XML) and actual datastore.



2. Flow and use-cases
=================

a. The mtt client will dump all test related information into XML file. The file will be created for every phase executed by mtt. (today there are many summary txt and html files generated for every test phase, it is pretty easy to add xml generation of the same information)

b. mtt_save_to_db.py - script which will go over mtt scratch dir, find all xml files generated for every mtt phase, parse it and save to datastore, preserving test results relations,i.e. all test results will be grouped by mtt general info: mpi version, name, date, ....

c. same script can scan, parse and save from xml files generated by wrapper scripts for non mtt based executions (fluent, ..)

d. mtt_query_db.py script will be provided with basic query capabilities over proposed datastore object model. Most users will prefer writing custom sql-like select queries for fetching results.

3. Important notes:
==============

a. The single mtt client execution generates many result files, every generated file represents test phase. This file contains test results and can be characterized as a set of attributes with its values. Every test phase has its own attributes which are differ for different phases. For example: attributes for TestBuild phase has keys "compiler_name, compiler_version", the MPIInstall phase has attributes: prefix_dir, arch, ....
Hence, most of the datastore objects representing phases of MTT  are derived from "db.Expando" model, which allows having dynamic attributes for its derived sub-classes.


The attached is archive with a simple test for using datastore for mtt. Please see models.py file with proposed object model and comment.

You can run the attached example in the google datastore dev environment. (http://code.google.com/appengine/downloads.html)

Please comment.


Thanks

Mike