Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Handling output of processes
From: Ralph Castain (rhc_at_[hidden])
Date: 2009-01-26 14:14:28


Yes. The --tag-output option will prepend [job,rank]<stdout> (or
stderr, whichever is appropriate) to each line. I don't insert a
colon, though I suppose that would easily be done for grep purposes.

I just finished implementing the --output-filename option that will
split the output from each rank into its own file. Completed the
timestamp option last night.

Almost have the --screen option done - it's a little trickier. :-)

Ralph

On Jan 26, 2009, at 11:56 AM, Douglas Guptill wrote:

> Hello Ralph:
>
> Please forgive if this has already been covered...
>
> Have you considered prefixing each line of output from each process
> with something like "process_number" and a colon?
>
> That is what IBM's poe does. Separating the output is then easy:
> cat file | grep 0: > file.0
> cat file | grep 1: > file.1
> etc.
>
> In the meantime, I got around the problem by having each process open
> its own file (instead of using stdout), and embedding the process
> number in the file name.
>
> Hope that helps,
> Douglas.
>
>
> On Sun, Jan 25, 2009 at 05:20:47AM -0700, Ralph Castain wrote:
>> For those of you following this thread:
>>
>> I have been impressed by the various methods used to grab the output
>> from processes. Since this is clearly something of interest to a
>> broad
>> audience, I would like to try and make this easier to do by adding
>> some options to mpirun. Coming in 1.3.1 will be --tag-output, which
>> will automatically tag each line of output with the rank of the
>> process - this was already in the works, but obviously doesn't meet
>> the needs expressed here.
>>
>> I have done some prelim work on a couple of options based on this
>> thread:
>>
>> 1. spawn a screen and redirect process output to it, with the ability
>> to request separate screens for each specified rank. Obviously,
>> specifying all ranks would be the equivalent of replacing "my_app" on
>> the mpirun cmd line with "xterm my_app". However, there are cases
>> where you only need to see the output from a subset of the ranks, and
>> that is the intent of this option.
>>
>> 2. redirect output of specified processes to files using the provided
>> filename appended with ".rank". You can do this for all ranks, or a
>> specified subset of them.
>>
>> 3. timestamp output
>>
>> Is there anything else people would like to see?
>>
>> It is also possible to write a dedicated app such as Jody described,
>> but that is outside my purview for now due to priorities. However, I
>> can provide technical advice to such an effort, so feel free to ask.
>>
>> Ralph
>>
>>
>> On Jan 23, 2009, at 12:19 PM, Gijsbert Wiesenekker wrote:
>>
>>> jody wrote:
>>>> Hi
>>>> I have a small cluster consisting of 9 computers (8x2 CPUs, 1x4
>>>> CPUs).
>>>> I would like to be able to observe the output of the processes
>>>> separately during an mpirun.
>>>>
>>>> What i currently do is to apply the mpirun to a shell script which
>>>> opens a xterm for each process,
>>>> which then starts the actual application.
>>>>
>>>> This works, but is a bit complicated, e.g. finding the window
>>>> you're
>>>> interested in among 19 others.
>>>>
>>>> So i was wondering is there a possibility to capture the processes'
>>>> outputs separately, so
>>>> i can make an application in which i can switch between the
>>>> different
>>>> processor outputs?
>>>> I could imagine that could be done by wrapper applications which
>>>> redirect the output over a TCP
>>>> socket to a server application.
>>>>
>>>> But perhaps there is an easier way, or something like this alread
>>>> does exist?
>>>>
>>>> Thank You
>>>> Jody
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>>
>>> For C I use a printf wrapper function that writes the output to a
>>> logfile. I derive the name of the logfile from the mpi_id. It
>>> prefixes the lines with a time-stamp, so you also get some basic
>>> profile information. I can send you the source code if you like.
>>>
>>> Regards,
>>> Gijsbert
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> FORTRAN?
> The syntactically incorrect statement "DO 10 I = 1.10" will parse and
> generate code creating a variable, DO10I, as follows: "DO10I =
> 1.10" If that
> doesn't terrify you, it should.
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users