From: Jeff Squyres \(jsquyres\) (jsquyres_at_[hidden])
Date: 2006-06-29 12:45:08
> -----Original Message-----
> From: mtt-users-bounces_at_[hidden]
> [mailto:mtt-users-bounces_at_[hidden]] On Behalf Of Andrew Friedley
> Sent: Thursday, June 29, 2006 11:29 AM
> To: General user list for the MPI Testing Tool
> Subject: Re: [MTT users] Test output to perfbase
>
> > I'll change my question to: can you send an example of the
> format that
> > perfbase is expecting for submitting the data running
> multiple tests in
> > a single http post to perfbase.php? E.g., say I have the results of
> > running all the intel tests -- what is the format that you are
> > expecting?
>
> I'll try to make some sample data tonight. I need to review how line
> separators work with perfbase first.
Thanks.
> > Another question -- how exactly are you categorizing these
> results on
> > the server? You made mention of "runs" below -- from your
> context I'm
> > assuming that "run" has some specific meaning to perfbase,
> especially
> > with the categorization of output data.
>
> For efficiency, one run is one run of a test suite. What
Which efficiency? Uploads? Database storage? Querying?
In a conversation with Sun, it turns out that we both want to have the
ability to see partial results (e.g., running the entire Intel suite may
take many hours -- it would be good to be able to see results
more-or-less as they occur). Is there a technical issue that would
prevent submitting 1 (or small batches of) result(s) at a time?
> field is the
> BTL selection reported in? If it 'occurs once' in the XML,
> it is stored
> per test suite (run), if it 'occurs many', that is a per-test
> field and
> can vary in a single run (i.e. tcp and openib would go together).
Ok, good. So this would help with the database storage required for
test results -- many of the fields will be the same for each of the
individual tests in the intel test suite (for example).
> > 1. send all results from the above intel run in a single
> submit (i.e.,
> > all tcp and all openib results). Since we submit the MCA
> params as part
> > of the data, our queries later can distinguish tcp vs. openib data.
>
> This is what I want. We can easily construct queries to only
So I guess I'm still not clear on *why* you want this. :-) Can you
specify the reasons?
> get what
> we want i.e. just tcp, just threaded, just odin, etc.
>
> > 2. send all the tcp results in one submit and then send the
> openib data
> > in a separate submit.
>
> Bleh - so this works if you only consider the 'btl'
> parameter. But what
> ito consider other parameters in this way?
Yes, that information (tcp vs. openib) is in one of the fields that we
send back (it has to be, otherwise the results are somewhat
meaningless). It's not a standalone "btl" field, though -- it's more of
a "here's the MCA parameters that were specified" field. So queries for
tcp results will probably need to search for "tcp" in the MCA parameters
field.
But this is the same issue regardless of whether we submit 1 result at a
time or all at once, right? I guess I don't see the difference for
selecting "tcp" vs. "openib" results based on whether we submit 1 result
at a time or all at once -- can you clarify? I think I must be missing
something...
If this is all possible, then -- at least in my mind -- I don't see a
reason why multiple submits vs. a single submit is *required*.
Obviously, multiple submits will take more bandwidth than a single
submit -- but I see that as an optimization that we can [and should]
work out later. Specifically: reducing the bandwidth of submits doesn't
need to be in the initial version since our primary, immediate goal is
to get this functional, running nightly tests, and sending out test
results in the morning as long as the current, unoptimized bandwidth
requirements are not too onerous on milliways.
-- Jeff Squyres Server Virtualization Business Unit Cisco Systems