Subject: Re: [MTT users] [MTT bugs] [MTT] #212: Generic network locking server
From: Ethan Mallove (ethan.mallove_at_[hidden])
Date: 2009-11-06 17:18:56
On Mon, Mar/05/2007 09:42:53PM, MTT wrote:
> #212: Generic network locking server
> ------------------------------------------+---------------------------------
> Reporter: jsquyres | Owner: jsquyres
> Type: defect | Status: closed
> Priority: major | Milestone: v2.0.2
> Component: Both client and server side | Version: trunk
> Resolution: fixed | Keywords:
> ------------------------------------------+---------------------------------
> Changes (by jsquyres):
>
> * status: new => closed
> * resolution: => fixed
>
> Comment:
>
> (In [642]) Fixes #211, #212
>
> * Implement new fields in the INI file [MTT] section for specifying a
> comma-delimited list of proxies to use for HTTP/FTP downloads, the
> SVN module, and
> * '''http_proxy''': URL for HTTP proxying (to include proper method
> prefix, such as "!http://")
> * '''https_proxy''': Ditto for HTTPS proxying
> * '''ftp_proxy''': Ditto for FTP proxying
> * This ended up ''requiring'' a proper locking solution (see below).
> * Implemented MTT::Lock framework to allow arbitrary
> locking/unlocking by string name (typically a filename). A new
> optional "[lock]" section now exists in the INI file. If
> unspecified, all locks/unlocks are no-ops.
> * Just like other sections, you must specify a "module=<foo>" line
> in the [lock] section to specify which module is used to implement
> the locks.
> * One module currently exists in this framework: MTTLockServer.
> It takes 2 parameters: mttlockserver_host and
> mttlockserver_port. Example:
> {{{
> [Lock]
> module = MTTLockServer
> mttlockserver_host = my-lock-server.example.com
> mttlockserver_port = 12345
> }}}
> * Fixups from r640 to bring the mtt-lock-server online
> * The mtt-lock-server is currently in client/, but may move someday.
> It takes a few parameters when launched:
> * '''-f''': To run it in the foreground (default is to daemonize).
> * '''-p''': port number to listen on
> * '''-s''': log to syslog
>
> Typically, you run a single mtt-lock-server for your cluster and then
> go run as many MTT's as you want. The lock server will enforce
> serialization where necessary.
I'm running multiple MTT clients out of the same scratch directory
using SGE. I'm running into race conditions between the multiple
clients, where one client is overwriting another's data in the .dump
files - which is a Very Bad Thing(tm). I'm running the
client/mtt-lock-server, and I've added the corresponding [Lock]
section in my INI file. Will my MTT clients now not interfere with
each other's .dump files? I'm skeptical of this because I don't see,
e.g., Lock() calls in SaveRuns(). How do I make my .dump files safe?
-Ethan
>
> '''NOTE:''' The SVN module ''must'' have a locking module if multiple
> MTT's are going to be running simultaneously with a proxy, due to the
> fact that SVN will only use the file $HOME/.subversion/servers to find
> where its proxy server is located. Hence, MTT will lock that file,
> set the proxy, do the SVN checkout/export, restore the file, and then
> unlock it. If locking is not used, mutliple MTT clients could stop on
> the "servers" file simultaneously and Bad Things(tm) could happen.
>
> --
> Ticket URL: <https://svn.open-mpi.org/trac/mtt/ticket/212#comment:7>
> MTT <http://www.open-mpi.org/>
> Issue tracking for the MPI Testing Tool.
>