Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Devel mailing list

Subject: Re: [MTT devel] MTT GDS -- one more...
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-03-03 14:29:54

On Feb 21, 2010, at 5:52 AM, Mike Dubman wrote:

> > > 2. In reading through the Google Appengine docs, the GDS stuff looks like
> > we mainly can access the data through GQL. I don't see any mention of doing
> > map/reduce kinds of computations (Ethan and I were talking on the phone
> > today about MTT Appengine possibilities). I'm new to all this stuff, so
> > it's quite possible that a) I missed it, or b) I just don't understand what
> > I'm seeing/reading yet. Or does GQL do map/reduce on the back end to do its
> > magic? Is GQL the main/only way we have to access GDS?
> >
> > As far as I and Igor know there are no way of doing Map/Reduce with GDS. And
> > GQL (or filters which is practically synonym) is the main and only way to
> > access GDS data.
> btw, afaik - the gds is using mapreduce explicitly on the backend - one needs do nothing in order to ENABLE it: just use their API and the framework will do the rest.

Hm; ok. So they only give us limited access to map reduce -- via GQL? (i.e., we can only do what GQL allows us to do)

For example, I'd love to be able to specify my own reductions in order to do some real data mining, parsing through the data, etc. Specifically, I'd like to be able to exploit the whole "bring the computation to the data" philosophy of map reduce, because MTT data can be *huge*. But I don't see how to do this with GQL...?

Am I missing something?

Jeff Squyres
For corporate legal information go to: