Open MPI logo

MTT Devel Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all MTT Users mailing list

Subject: Re: [MTT users] Patch to add --local-scratch option
From: Ethan Mallove (ethan.mallove_at_[hidden])
Date: 2008-09-16 12:07:31


Nice! A few thoughts ...

What happens if one uses --local-scratch, but leaves out the
--scratch option? In this case, I think MTT should assume
--scratch is the same as --local-scratch.

Could the "local" in --local-scratch ever be misleading?
Couldn't a user ever use a mounted filesystem that's faster
than all their local filesystem? Should it be
--fast-scratch?

For future work, how about --scratch taking a (CSV?) list of
scratch directories. MTT then does a quick check for which
is the fastest filesystem (if such a check is
possible/feasible), and proceeds accordingly. That is, doing
everything it possible can in a fast scratch (builds,
writing out metadata, etc.), and installing the MPI(s) into
the slow mounted scratch. Would this be possible?

On Mon, Sep/15/2008 06:15:43PM, Tim Mattox wrote:
> Hello,
> Attached is a patchfile for the mtt trunk that adds a
> --local-scratch <dir_name>
> option to client/mtt. You can also specify something like
> this in your [MTT] ini section:
> local_scratch = &shell("echo /tmp/`whoami`_mtt")
>
> This local-scratch directory is then used for part of the --mpi-install
> phase to speed up your run. Specifically, the source-code of the
> MPI is untarred there, configure is run, make all, and make check.
> Then, when make install is invoked the MPI is installed into the
> usual place as if you hadn't used --local-scratch. If you don't
> use --local-scratch, then the builds occur in the usual place that
> they have before.
>
> For the clusters at IU that seem to have slow NSF home directories,
> this cuts the --mpi-install phase time in half.
>
> The downside is that if the MPI build fails, your build directory is out
> on some compile-node's /tmp and is harder to go debug. But, since
> mpi build failures are now rare, this should make for quicker turnaround
> for the general case.
>
> I think I adjusted the code properly for the vpath build case, but I've
> never used that so haven't tested it. Also, I adjusted the free disk space
> check code. Right now it only checks the free space on --scratch,
> and won't detect if --local-scratch is full. If people really care, I
> could make it check both later. But for now, if your /tmp is full
> you probably have other problems to worry about.
>
> Comments? Can you try it out, and if I get no objections, I'd like
> to put this into the MTT trunk this week.

I'll test your patch soon.

-Ethan

> --
> Tim Mattox, Ph.D. - http://homepage.mac.com/tmattox/
> tmattox_at_[hidden] || timattox_at_[hidden]
> I'm a bright... http://www.the-brights.net/
> _______________________________________________
> mtt-users mailing list
> mtt-users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/mtt-users