Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Devel mailing list

Subject: Re: [MTT devel] MTT GDS -- one more...
From: Andrew Senin (as_at_[hidden])
Date: 2010-02-12 11:35:38


Hello Jeff,

I worked with Igor on the GDS framework (although Igor knows more tech
details than me). Let me put my two cents to the discussion.

> 1. It looks like the main benefits of using the Google App Engine --
specifically for MTT -- is that we can use the GDS and/or we can host an
application on their web servers. Is that correct?

I think yes. Also GDS should work faster than a relational DB on large
amounts of data.

> 2. In reading through the Google Appengine docs, the GDS stuff looks like
we mainly can access the data through GQL. I don't see any mention of doing
map/reduce kinds of computations (Ethan and I were talking on the phone
today about MTT Appengine possibilities). I'm new to all this stuff, so
it's quite possible that a) I missed it, or b) I just don't understand what
I'm seeing/reading yet. Or does GQL do map/reduce on the back end to do its
magic? Is GQL the main/only way we have to access GDS?

As far as I and Igor know there are no way of doing Map/Reduce with GDS. And
GQL (or filters which is practically synonym) is the main and only way to
access GDS data.

> 3. Is there a reason that MTTGDS.pm doesn't use the python API to directly
talk to GDS? I.e., what is the rationale for using a web app on appengine?
Is the web app doing stuff that we can't do at the client? Ditto for
bquery.pl and breport.pl. (these questions are partially fueled by my
curiosity and concern about why we're using so much CPU at Google)

There are a few reasons of doing it. The first is speed. When we post new
data we firstly try to find if there is a copy of corresponding MpiInfo,
ClustreInfo and other *Info classes. If we did it directly from client
scripts the delays would be higher (depending on Internet connection speed).
Price of it is additional CPU cycles on google servers. The second and more
important is that when we have such logic on server we (instead of GDS
clients) are responsible for maintaining correct structure of links between
objects. If such logic was implemented on client side user could (by mistake
or on purpose) break links between objects.

Regards,
Andrew Senin

-----Original Message-----
Date: Thu, 11 Feb 2010 21:43:21 -0500
From: Jeff Squyres <jsquyres_at_[hidden]>
Subject: [MTT devel] MTT GDS -- one more...
To: Development list for the MPI Testing Tool <mtt-devel_at_[hidden]>
Message-ID: <8A556B10-1618-47EA-96A9-33F22AECD3F1_at_[hidden]>
Content-Type: text/plain; charset=us-ascii

Heh... even more questions...

(BTW, Ethan and I have asked soooo many questions that if it helps, I can
setup a webex and we can all discuss this in person rather than via
1,000,000 annoying emails from us. :-) Webex can call you; no one will
need to pay for an international call)

1. It looks like the main benefits of using the Google App Engine --
specifically for MTT -- is that we can use the GDS and/or we can host an
application on their web servers. Is that correct?

2. In reading through the Google Appengine docs, the GDS stuff looks like we
mainly can access the data through GQL. I don't see any mention of doing
map/reduce kinds of computations (Ethan and I were talking on the phone
today about MTT Appengine possibilities). I'm new to all this stuff, so
it's quite possible that a) I missed it, or b) I just don't understand what
I'm seeing/reading yet. Or does GQL do map/reduce on the back end to do its
magic? Is GQL the main/only way we have to access GDS?

3. Is there a reason that MTTGDS.pm doesn't use the python API to directly
talk to GDS? I.e., what is the rationale for using a web app on appengine?
Is the web app doing stuff that we can't do at the client? Ditto for
bquery.pl and breport.pl. (these questions are partially fueled by my
curiosity and concern about why we're using so much CPU at Google)

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/