Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] RFC: revamp OPAL database framework
From: Ralph Castain (rhc_at_[hidden])
Date: 2014-04-11 16:07:18


WHAT: Revamp the opal database framework, including renaming it to "dstore" to reflect that it isn't a "database"

WHY: The original modex datastore operation became entangled with true database APIs
                for things like Postgres. In addition, George has proposed a cleaner separation of modex
                data, organized by code layer and perhaps other attributes

WHEN: April 29th, barring issues

WHERE: https://bitbucket.org/rhc/opal-db

***** Details *****

The "db" framework was originally located in ORTE, and was used for database-related operations in support of the sensor framework. It was then extended to hold and support the modex data, and subsequently moved to the OPAL level in anticipation of the BTLs moving down there. At the current time, it has several APIs relating to modex support, and one API for general database operations. In order to separate out modex info between BTL data that must be shared with peers, data which only needs to be swapped during comm-spawn, and data used only for internal support, we created an "opal_scope_t" and require that you tag your data with one those three scopes when it is "stored". You then fetch data based on the scope as well as the key.

This change does the following:

* moves the pure database operations to a new ORTE "db" framework - basically, restoring the database operations to where they used to be. This includes moving the postgres, print, and sqlite components

* renames the modex datastore framework in OPAL to "dstore", leaving the hash and pmi components there.

We have removed the opal_scope_t designation. The modex datastore framework instead now includes the concept of a "handle" that is somewhat equivalent to a POSIX file descriptor. You "open" a datastore and return a handle for that storage area - all subsequent calls to store and fetch data must provide the handle indicating the storage area. Each handle actually translates into a unique module, so the data remains completely isolated from any other handle.

Only one datastore component can be selected. Thus, the pmi component now includes its own hash table storage, and we've put the common hash table support code in the datastore base.

The database framework operates the same way as the datastore, except that it allows multiple components to be active, and for the caller to specify the desired database backend when "opening" the database handle.

HTH
Ralph