Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] OMPI/ORTE and tools
From: Ralph Castain (rhc_at_[hidden])
Date: 2008-01-16 08:47:02


Hello all

Summary: this note provides a brief overview of how various tools can
interface to OMPI applications once the next version of ORTE is integrated
into the trunk. It includes a request for input regarding any needs (e.g.,
additional commands to be supported in the interface) that have not been
adequately addressed.

As many of you know, I have been working on a tmp branch to complete the
revamp of ORTE that has been in progress for quite some time. Among other
things, this revamp is intended to simplify the system, provide enhanced
scalability, and improved reliability.

As part of that effort, I have extensively revised the support for external
tools. In the past, tools such as the Eclipse PTP could only interact with
Open MPI-based applications via ORTE API's, thus exposing the tool to any
changes in those APIs. Most tools, however, do not require the level of
control provided by the APIs and can benefit from a simplified interface.

Accordingly, the revamped ORTE now offers alternative methods of
interaction. The primary change has been the creation of a communications
library with a simple serial protocol for interacting with OMPI jobs. Thus,
tools now have three choices for interacting with OMPI jobs:

1. I have created a new communications library that tools can link against.
It does not include all of the ORTE or OMPI libraries, so it is a very small
memory footprint. Besides the usual calls to initialize and finalize, the
library contains utilities for finding all of the OMPI jobs running on that
HNP (i.e., all OMPI jobs whose mpirun was executed from that host), querying
the status of a job (provides the job map plus all proc states); querying
the status of nodes (provides node names, status, and list of procs on each
node including their state); querying the status of a specific process;
spawning a new job; and terminating a job. In addition, you can attach to
output streams of any process, specifying stdout, stderr, or both - this
"tees" the specified streams, so it won't interfere with the job's normal
output flow.

I could also create a utility to allow attachment to the input stream of a
process. However, I'm a little concerned about possible conflicts with
whatever is already flowing across that stream. I would appreciate any
suggestions as to whether or not to provide that capability.

Note: we removed the concept of the ORTE "universe", so a tool can now talk
to any mpirun without complications. Thus, tools can simultaneously
"connect" to and monitor multiple mpiruns, if desired.

2. link against all of OMPI or ORTE, and execute a standalone program. In
this mode, your tool would act as a surrogate for mpirun by directly
spawning the user's application. This provides some flexibility, but it does
mean that both the tool and the job -must- end together, and that the tool
may need to be revised whenever OMPI/ORTE APIs are updated.

3. link against all of OMPI or ORTE, executing as a distributed set of
processes. In this mode, you would execute your tool via "mpirun -pernode
./my_tool" (or whatever command is appropriate - this example would launch
one tool process on every node in the allocation). If the tool processes
need to communicate with each other, they can call MPI_Init or orte_init,
depending upon the level of desired communication. Note that the tool job
will be completely standalone from the application job and must be
terminated separately.

In all of these cases, it is possible for tool processes to connect (via MPI
and/or ORTE-RML) to a job's processes provided that the application supports
it.

I can provide more details, of course, to anyone wishing them. What I would
appreciate, though, is any feedback about desired commands, mode of
operation, etc. that I might have missed or people would prefer be changed.
This code is all in a private repository for my tmp branch, but I expect
that to merge with the trunk fairly soon. I have provided a couple of
example tools to illustrate the above modes of operation in that code.

Thanks
Ralph