Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Configure and Build ok, but mpi module not recognized?
From: Terry Frankcombe (terry_at_[hidden])
Date: 2008-09-23 20:34:04


Just to jump in on the side of Fortran here: The statement ordering
rules are indeed sensible. You need to have your implicit typing set
before you start declaring stuff (so include must come after implicit).
You need to have all your modules used before setting your implicit
typing (as modules often define new types, which must be known before
you start declaring implicit rules).

On Tue, 2008-09-23 at 13:34 -0400, Gus Correa wrote:
> Hi Brian and list
>
> Terry Frankcombe is right on the spot on that recommendation to you.
>
> Just to support Terry's suggestion, here is what "Fortran 95/2003
> Explained", by Michael Metcalf,
> John Reid, and Malcolm Cohen, Oxford Univ. Press, 2004, pp. 144, Section
> 7.10, says about it:
>
> "Any 'use' statements must precede other specification statements in a
> scoping unit."
>
> Fortran 90/95/2003 scoping rules are a pain,
> written on a legalistic-forensic style, but this one at least is easy to
> remember! :)
>
> BTW, "scope" corresponds to the location and accessibility of a variable
> or another Fortran entity
> on a programming unit.
> I.e., to be accessed by a programming unit, the entity must be "visible"
> there,
> even if it is declared somewhere else.
> "Scope" is not related to the program unit source form (i.e. not to free
> vs. fixed form),
> which may have been part of the confusion on our conversation.
>
> Cheers,
> Gus Correa
>
> --
> ---------------------------------------------------------------------
> Gustavo J. Ponce Correa, PhD - Email: gus_at_[hidden]
> Lamont-Doherty Earth Observatory - Columbia University
> P.O. Box 1000 [61 Route 9W] - Palisades, NY, 10964-8000 - USA
> ---------------------------------------------------------------------
>
>
> Brian Harker wrote:
>
> >Ahhhhhhh, now that makes sense. Never included, always used. Thanks!
> >
> >On Mon, Sep 22, 2008 at 8:55 PM, Terry Frankcombe <terry_at_[hidden]> wrote:
> >
> >
> >>Remember what include does: it essentially dumps mpif.h into the
> >>source. So to be proper F90 you need:
> >>
> >>PROGRAM main
> >>USE local_module
> >>IMPLICT NONE
> >>INCLUDE 'mpif.h'
> >>...
> >>
> >>
> >>On Mon, 2008-09-22 at 20:17 -0600, Brian Harker wrote:
> >>
> >>
> >>>Well, I'm stumped then...my top-level program is the only one that
> >>>uses MPI interfaces. I USE other f90 module files, but none of them
> >>>are dependent on MPI functions. For example here's the first few
> >>>lines of code where things act up:
> >>>
> >>>PROGRAM main
> >>>INCLUDE 'mpif.h' (this line used to be "USE mpi"...this is
> >>>correct replacement, right?)
> >>>USE local_module
> >>>IMPLICIT NONE
> >>>...
> >>>STOP
> >>>END PROGRAM main
> >>>
> >>>with local_module.f90:
> >>>
> >>>MODULE local_module
> >>>CONTAINS
> >>>
> >>>SUBROUTINE local_subroutine
> >>>IMPLICIT NONE
> >>>...
> >>>RETURN
> >>>END SUBROUTINE local_subroutine
> >>>
> >>>END MODULE local_module
> >>>
> >>>and mpif90 gives me grief about local_module being out of scope within
> >>>main. To the best of my knowledge, I have never used fixed-form or
> >>>had free-vs-fixed form compiler issues with ifort. Thanks!
> >>>
> >>>On Mon, Sep 22, 2008 at 7:56 PM, Gus Correa <gus_at_[hidden]> wrote:
> >>>
> >>>
> >>>>Hi Brian and list
> >>>>
> >>>>On my code I have
> >>>>
> >>>> include 'mpif.h'
> >>>>
> >>>>with single quotes around the file name.
> >>>>I use single quotes, but double quotes are also possible according to the
> >>>>F90 standard.
> >>>>If you start at column 7 and end at column 72,
> >>>>you avoid any problems with free vs. fixed Fortran form (which may happen if
> >>>>one decides
> >>>>to mess the compiler options on Makefiles, for instance).
> >>>>This is PDP, paranoid defensive programming.
> >>>>I type the Fortran commands in lowercase (include) but this may not really
> >>>>make any difference
> >>>>(although there are compiler options to transliterate upper to lower, be/not
> >>>>be case sensitive, etc)
> >>>>
> >>>>However, since you asked, I changed the "include 'mpif.h'" location to
> >>>>column 1, column 6,
> >>>>etc, and I could compile the code with OpenMPI mpif90 without any problems.
> >>>>Hence, I presume mpif90 uses the free form as default.
> >>>>So, I wonder if you still have some residual mix of gfortran and ifort
> >>>>there,
> >>>>or if there is some funny configuration on your ifort.cfg file choosing the
> >>>>fixed form as a default.
> >>>>
> >>>>In any case, going out of scope probably has nothing to do with fixed vs
> >>>>free form.
> >>>>I only have one "use mpi" statement, no other modules "used" in the little
> >>>>hello_f90.
> >>>>It may well be that if you try other "use" statements, particularly if the
> >>>>respective modules
> >>>>involve other MPI modules or MPI function interfaces, things may break and
> >>>>be out of scope.
> >>>>I presume you are not talking of hello_f90 anymore, but of a larger code.
> >>>>The large code examples I have here use other F90 modules (not mpi.mod),
> >>>>but they don't rely on MPI interfaces.
> >>>>
> >>>>Glad to know that the main problem is gone (I read the more recent
> >>>>messages).
> >>>>I was reluctant to tell you to do a "make distclean", and start fresh,
> >>>>configure again, make again,
> >>>>because you said you had built the OpenMPI library more than once.
> >>>>Now I think that was exactly what I should have told you to do.
> >>>>Everybody builds these things many times to get them right.
> >>>>Then a few more times to make it efficient, to optimize, etc.
> >>>>Can you imagine how many times Rutherford set up that gold foil experiment
> >>>>until he got it right? :)
> >>>>After it is done, the past errors become trivial, and all the rest is
> >>>>reduced just to stamp collecting. :)
> >>>>Configure is always complex, and dumping its output to a log file is
> >>>>worth the effort, to check out for sticky problems like this, which often
> >>>>happen.
> >>>>(Likewise for make and make install.)
> >>>>
> >>>>I hope this helps,
> >>>>Gus Correa
> >>>>
> >>>>--
> >>>>---------------------------------------------------------------------
> >>>>Gustavo J. Ponce Correa, PhD - Email: gus_at_[hidden]
> >>>>Lamont-Doherty Earth Observatory - Columbia University
> >>>>P.O. Box 1000 [61 Route 9W] - Palisades, NY, 10964-8000 - USA
> >>>>---------------------------------------------------------------------
> >>>>
> >>>>
> >>>>Brian Harker wrote:
> >>>>
> >>>>
> >>>>
> >>>>>Hi Gus-
> >>>>>
> >>>>>Thanks for the idea. One question: how do you position INCLUDE
> >>>>>statements in a fortran program, because if I just straight substitute
> >>>>>' INCLUDE "mpif.h" ' for ' USE mpi ', I get a lot of crap telling me
> >>>>>my other USE statements are not positioned correctly within the scope
> >>>>>and nothing compiles. I have tried various positions, but I seem to
> >>>>>be suffering from a lot of BS. Am I overlooking something very
> >>>>>obvious?
> >>>>>
> >>>>>On Mon, Sep 22, 2008 at 5:42 PM, Gus Correa <gus_at_[hidden]> wrote:
> >>>>>
> >>>>>
> >>>>>
> >>>>>>Hi Brian and list
> >>>>>>
> >>>>>>I seldom used the "use mpi" syntax before.
> >>>>>>I have a lot of code here written in Fortran 90,
> >>>>>>but and mpif.h is included instead "use mpi".
> >>>>>>The MPI function calls are the same in Fortran 77 and Fortran 90 syntax,
> >>>>>>hence there is just one line of code to change, if one wants to go fully
> >>>>>>F90.
> >>>>>>All works well, though.
> >>>>>>This style is a legacy from the times when the Fortran90 interface of MPI
> >>>>>>was not fully developed,
> >>>>>>and the mpi.mod was broken most of the time.
> >>>>>>That is no longer the case, though.
> >>>>>>
> >>>>>>This mixed mode may be one way around your problem, although not ideal.
> >>>>>>The main drawback of using the "include mpif.h" syntax
> >>>>>>is that there is no parameter checking of the MPI function calls,
> >>>>>>whereas the "use mpi" syntax provide some level of parameter checking
> >>>>>>through the mpi.mod module.
> >>>>>>If you are careful when you write your MPI calls,
> >>>>>>and check if all parameters match the syntax requirements strictly,
> >>>>>>this shouldn't be a problem, though.
> >>>>>>(Nevertheless, a few days ago somebody was suffering here on the list
> >>>>>>with a
> >>>>>>problem that was just
> >>>>>>a missing "ierr" parameter in an MPI_Send call. Jeff found it out,
> >>>>>>after
> >>>>>>several emails back and forth.)
> >>>>>>
> >>>>>>If not for anything else, just for the fun of it, to see how far the
> >>>>>>compilation goes,
> >>>>>>I would comment out "use mpi" and replace it by "include mpif.h".
> >>>>>>
> >>>>>>As for architecture and compiler, I have used with no problem OpenMPI
> >>>>>> (and
> >>>>>>MPICH2 and MPICH-1) on various
> >>>>>>x86 and x86_64 machines, both AMD and Intel, under different Linux
> >>>>>>flavors
> >>>>>>(Fedora, CentOS, Red Hat),
> >>>>>>in standalone machines (using SMP and shared memory),
> >>>>>>and on a small cluster using Ethernet 100T , then using Gigabit
> >>>>>>Ethernet,
> >>>>>>compiling it with gcc and ifort and with gcc and pgf90.
> >>>>>>
> >>>>>>Note that I don't have icc, I use gcc and g++.
> >>>>>>Are the icc and ifort versions that you have the same?
> >>>>>>I don't know if different versions may not mix well, but it is a
> >>>>>>possibility.
> >>>>>>
> >>>>>>I tried to reproduce your problem by writing a simple hello_f90.f90
> >>>>>>program,
> >>>>>>with the "use mpi" syntax,
> >>>>>>and compiling it with the OpenMPI mpif90.
> >>>>>>However, the program compiled without any problem,
> >>>>>>and I didn't need to point to the mpi.mod directory using the "-module"
> >>>>>>flag
> >>>>>>either,
> >>>>>>despite my having a "gfortran-openmpi" version of mpi.mod in
> >>>>>>/usr/lib/openmpi/.
> >>>>>>The program runs fine too, with 2 processes, 4 processes, etc.
> >>>>>>
> >>>>>>On your first message on this thread, your configure command declares
> >>>>>>environment
> >>>>>>variables for CC, CXX, F77, and FC without full paths.
> >>>>>>I wonder if there is any chance that there are multiple versions of Intel
> >>>>>>compilers on your
> >>>>>>system, that somehow may be the cause for the confusion (perhaps as early
> >>>>>>as
> >>>>>>the build time).
> >>>>>>
> >>>>>>Also, note that mpi.mod is not an include file.
> >>>>>>It is located in the lib directory of openmpi.
> >>>>>>There was some confusion about this before, with a discussion about "-I"
> >>>>>>directories, etc,
> >>>>>>but hopefully this was clarified already.
> >>>>>>
> >>>>>>Another thing to check is if there is any funny compiler configuration
> >>>>>>(say
> >>>>>>in /opt/intel/fc/bla/bla/ifort.cfg).
> >>>>>>
> >>>>>>Yet another thing I would look at are the logs for OpenMPI configure,
> >>>>>>make,
> >>>>>>and make install,
> >>>>>>if you saved them, to see if the Fortran90 interface was built to
> >>>>>>completion.
> >>>>>>
> >>>>>>Also, a suggestion is to compile with the verbose "-v" option, to see if
> >>>>>>it
> >>>>>>spits out some
> >>>>>>clue to what is going on.
> >>>>>>I hope this helps.
> >>>>>>
> >>>>>>Gus Correa
> >>>>>>(from the stamp-collecting side of science :) )
> >>>>>>
> >>>>>>--
> >>>>>>---------------------------------------------------------------------
> >>>>>>Gustavo J. Ponce Correa, PhD - Email: gus_at_[hidden]
> >>>>>>Lamont-Doherty Earth Observatory - Columbia University
> >>>>>>P.O. Box 1000 [61 Route 9W] - Palisades, NY, 10964-8000 - USA
> >>>>>>---------------------------------------------------------------------
> >>>>>>
> >>>>>>
> >>>>>>Brian Harker wrote:
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>>Hi guys-
> >>>>>>>
> >>>>>>>Still no dice. The only mpi.mod files I have are the ones generated
> >>>>>>>from my compile and build from source (and they are where they should
> >>>>>>>be), so there's definitely no confusion amongst the modules. And
> >>>>>>>specifying the fulls path to the correct mpi.mod module (like Gus
> >>>>>>>suggested with the -module option) gives no change. I am running out
> >>>>>>>of ideas and patience, as I'm sure you all are too! Perhaps openMPI
> >>>>>>>just doesn't play nice with my compiler suite and hardware
> >>>>>>>architecture...? Thanks for all the input!
> >>>>>>>
> >>>>>>>On Mon, Sep 22, 2008 at 11:27 AM, Gus Correa <gus_at_[hidden]>
> >>>>>>>wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>>Hi Brian and list
> >>>>>>>>
> >>>>>>>>I read your original posting and Jeff's answers.
> >>>>>>>>
> >>>>>>>>Here on CentOS from Rocks Cluster I have a "native" OpenMPI, with a
> >>>>>>>>mpi.mod,
> >>>>>>>>compiled with gfortran.
> >>>>>>>>Note that I don't even have gfortran installed!
> >>>>>>>>This is besides the MPI versions (MPICH2 and OpenMPI)
> >>>>>>>>I installed from scratch using combinations of ifort and pgi with gcc.
> >>>>>>>>It may be that mpif90 is not picking the right mpi.mod, as Jeff
> >>>>>>>>suggested.
> >>>>>>>>Something like this may be part of your problem.
> >>>>>>>>A "locate mpi.mod" should show what your system has.
> >>>>>>>>
> >>>>>>>>Have you tried to force the directory where mpi.mod is searched for?
> >>>>>>>>Something like this:
> >>>>>>>>
> >>>>>>>>/full/path/to/openmpi/bin/mpif90 -module
> >>>>>>>>/full/path/to/openmpi_mpi.mod_directory/ hello_f90.f90
> >>>>>>>>
> >>>>>>>>The ifort man pages has the "-module" syntax details.
> >>>>>>>>
> >>>>>>>>I hope this helps.
> >>>>>>>>
> >>>>>>>>Gus Correa
> >>>>>>>>
> >>>>>>>>--
> >>>>>>>>---------------------------------------------------------------------
> >>>>>>>>Gustavo J. Ponce Correa, PhD - Email: gus_at_[hidden]
> >>>>>>>>Lamont-Doherty Earth Observatory - Columbia University
> >>>>>>>>P.O. Box 1000 [61 Route 9W] - Palisades, NY, 10964-8000 - USA
> >>>>>>>>---------------------------------------------------------------------
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>Brian Harker wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>Hi Gus-
> >>>>>>>>>
> >>>>>>>>>Thanks for the input. I have been using full path names to both the
> >>>>>>>>>wrapper compilers and mpiexec from the first day I had two MPI
> >>>>>>>>>implementations on my machine, depending on if I want to use MPICH or
> >>>>>>>>>openMPI, but still the problem remains. ARGGGGGG!
> >>>>>>>>>
> >>>>>>>>>On Mon, Sep 22, 2008 at 9:40 AM, Gus Correa <gus_at_[hidden]>
> >>>>>>>>>wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>>Hello Brian and list
> >>>>>>>>>>
> >>>>>>>>>>My confusing experiences with multiple MPI implementations
> >>>>>>>>>>were fixed the day I decided to use full path names to the MPI
> >>>>>>>>>>compiler
> >>>>>>>>>>wrappers (mpicc, mpif77, etc) at compile time,
> >>>>>>>>>>and to the MPI job launcher (mpirun, mpiexec, and so on) at run time,
> >>>>>>>>>>and to do this in a consistent fashion (using the tools from the same
> >>>>>>>>>>install to compile and to run the programs).
> >>>>>>>>>>
> >>>>>>>>>>Most Linux distributions come with built in MPI implementations
> >>>>>>>>>>(often
> >>>>>>>>>>times
> >>>>>>>>>>more than one),
> >>>>>>>>>>and so do commercial compilers and other tools.
> >>>>>>>>>>You end up with a mess of different MPI versions on your "native"
> >>>>>>>>>>PATH,
> >>>>>>>>>>as well as variety of bin, lib, and include directories containing
> >>>>>>>>>>different
> >>>>>>>>>>MPI stuff.
> >>>>>>>>>>The easy way around is to use full path names, particularly if you
> >>>>>>>>>>install
> >>>>>>>>>>yet another MPI implementation
> >>>>>>>>>>from scratch.
> >>>>>>>>>>Another way is to fix your PATH on your initialization files (.cshrc,
> >>>>>>>>>>etc)
> >>>>>>>>>>to point to your preferred implementation (put the appropriate bin
> >>>>>>>>>>directory
> >>>>>>>>>>ahead of everything else).
> >>>>>>>>>>Yet another is to install the "environment modules" package on your
> >>>>>>>>>>system
> >>>>>>>>>>and use it consistently.
> >>>>>>>>>>
> >>>>>>>>>>My two cents.
> >>>>>>>>>>
> >>>>>>>>>>Gus Correa
> >>>>>>>>>>
> >>>>>>>>>>--
> >>>>>>>>>>---------------------------------------------------------------------
> >>>>>>>>>>Gustavo J. Ponce Correa, PhD - Email: gus_at_[hidden]
> >>>>>>>>>>Lamont-Doherty Earth Observatory - Columbia University
> >>>>>>>>>>P.O. Box 1000 [61 Route 9W] - Palisades, NY, 10964-8000 - USA
> >>>>>>>>>>---------------------------------------------------------------------
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>Brian Harker wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>>I built and installed both MPICH2 and openMPI from source, so no
> >>>>>>>>>>>distribution packages or anything. MPICH2 has the modules located
> >>>>>>>>>>>in
> >>>>>>>>>>>/usr/local/include, which I assume would be found (since its in my
> >>>>>>>>>>>path), were it not for specifying -I$OPENMPI_HOME/lib at compile
> >>>>>>>>>>>time,
> >>>>>>>>>>>right? I can't imagine that if you tell it where to look for the
> >>>>>>>>>>>correct modules, it would search through your path first before
> >>>>>>>>>>>going
> >>>>>>>>>>>to where you tell it to go. Or am I too optimistic? Thanks again
> >>>>>>>>>>>for
> >>>>>>>>>>>the input!
> >>>>>>>>>>>
> >>>>>>>>>>>On Mon, Sep 22, 2008 at 8:58 AM, Jeff Squyres <jsquyres_at_[hidden]>
> >>>>>>>>>>>wrote:
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>>On Sep 22, 2008, at 10:10 AM, Brian Harker wrote:
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>Thanks for the reply...crap, $HOME/openmpi/lib does contains all
> >>>>>>>>>>>>>the
> >>>>>>>>>>>>>various lilbmpi* files as well as mpi.mod,
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>That should be correct.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>but still get the same
> >>>>>>>>>>>>>error at compile-time. Yes, I made sure to specifically build
> >>>>>>>>>>>>>openMPI
> >>>>>>>>>>>>>with ifort 10.1.012, and did run the --showme command right after
> >>>>>>>>>>>>>installation to make sure the wrapper compiler was using ifort as
> >>>>>>>>>>>>>well.
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>Ok, good.
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>>Before posting to this mailing list, I did uninstall and
> >>>>>>>>>>>>>re-install
> >>>>>>>>>>>>>openMPI several times to make sure I had a clean install. Still
> >>>>>>>>>>>>>no
> >>>>>>>>>>>>>luck. :(
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>>
> >>>>>>>>>>>>Ok. Have you checked around your machine to ensure that there is
> >>>>>>>>>>>>no
> >>>>>>>>>>>>other
> >>>>>>>>>>>>mpi.mod that the compiler is finding first? E.g., in your MPICH2
> >>>>>>>>>>>>installation? Or is Open MPI installed by your distro, perchance?
> >>>>>>>>>>>>You
> >>>>>>>>>>>>might want to try a "rpm -qa | grep openmpi" (or whatever your
> >>>>>>>>>>>>distro's
> >>>>>>>>>>>>equivalent is to check already-installed packages).
> >>>>>>>>>>>>
> >>>>>>>>>>>>--
> >>>>>>>>>>>>Jeff Squyres
> >>>>>>>>>>>>Cisco Systems
> >>>>>>>>>>>>
> >>>>>>>>>>>>_______________________________________________
> >>>>>>>>>>>>users mailing list
> >>>>>>>>>>>>users_at_[hidden]
> >>>>>>>>>>>>http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>>
> >>>>>>>>>>_______________________________________________
> >>>>>>>>>>users mailing list
> >>>>>>>>>>users_at_[hidden]
> >>>>>>>>>>http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>_______________________________________________
> >>>>>>>>users mailing list
> >>>>>>>>users_at_[hidden]
> >>>>>>>>http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>>
> >>>>>>_______________________________________________
> >>>>>>users mailing list
> >>>>>>users_at_[hidden]
> >>>>>>http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>_______________________________________________
> >>>>users mailing list
> >>>>users_at_[hidden]
> >>>>http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>>>
> >>>>
> >>>>
> >>>
> >>>
> >>>
> >>_______________________________________________
> >>users mailing list
> >>users_at_[hidden]
> >>http://www.open-mpi.org/mailman/listinfo.cgi/users
> >>
> >>
> >>
> >
> >
> >
> >
> >
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users