Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Devel mailing list

Subject: Re: [MTT devel] More GDS questions
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-02-12 11:51:19


On Feb 12, 2010, at 11:36 AM, Andrew Senin wrote:

> I would also like to add to Igor’s comment that CPU time shown by Google is a sum of all CPUs of distributed system involved in update operation (and who knows how many servers are involved?). Also they “define ‘CPU hour’ in terms of a hypothetical 1.4 GHz processor, whereas the actual processors we use in production vary but are generally faster than this” (see comment of DonSchwarz: http://groups.google.com/group/google-appengine-java/browse_thread/thread/aa9f18638b7bbea9?pli=1). According to the same topic 6.5 CPU hours is about 2.3 minutes real time. I think you may try to remove some of indexes which need to be updated on each new file upload (see Datastore Indexes on Web admin console).

Excellent information. Google seemed to agree that 2.3 mins of real time should be nowhere near 6.5 CPU hours quota and they claimed that they fixed at least one issue regarding bulk uploads.

However, this thread does imply that trickling in data over time instead of doing bulk uploads is a good idea.

Was the rationale of caching all MTTGDS info during the Submit phase and actually uploading it during Finalize just a measure to reduce submission latency?

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/