On Jun 2, 2010, at 1:31 PM, guillaume ranquet wrote:
> > ~/openmpi-1.4.2-bin/bin/mpirun --mca btl openib,sm,self --mca pml ^cm ~/bwlat/mpi_helloworld
> the first command seems to be wrong, I had an error message:
> MCA framework parameters can only take a single negation operator
Correct. Scott's 2nd one should be tried.
Open MPI has 2 flavors of MX support: a BTL and an MTL. You need to disable both of them. His 2nd command effectively does that:
1. cm is the PML is uses MTLs; by disabling cm, you're disabling all MTLs -- including the MX MTL
2. By specifically listing openib,sm,self, you're only allowing those BTLs to be used (not the MX BTL).
> granquet_at_bordeplage-9 ~/openmpi-1.4.2 $ ~/openmpi-1.4.2-bin/bin/mpirun
> - --mca btl openib,sm,self --mca pml ^cm ~/bwlat/mpi_helloworld
> Hello world from process 0 of 1
> granquet_at_bordeplage-9 ~/openmpi-1.4.2 $
> > Ok, there is no segfault when it can't find IB.
I'm not sure I follow this comment.
>From your prior mails:
- there's no segv when ptmalloc is disabled at run-time via the env var
- there's no segv when MX is completed disabled (both BTL and MTL)
What happens if you run with only MX? I *assume* that works with no segv...?
It might be interesting to see what happens if you run with:
mpirun --mca btl mx,openib,sm,self --mca pml ^cm --mca mpi_leave_pinned 0 ...yourapp...
This should run with both verbs and MX, and the memory manager is in place at run-time, but it isn't being used to track memory. That's slightly different than having the memory manager in place at run-time *and* using it to track memory.
> the goal is to run the same version everywhere on every nodes (for the
> sake of simplicity).
> the current plans were targeting 1.4.1.
> I don't think our users would mind upgrading to 1.4.2.
FWIW, it *is* the same version on all nodes -- you're just running with different MCA parameter values. Also FWIW, the sysadmin can hide these MCA params in a system-level file so that users don't have to deal with them, if that works for you. See:
For corporate legal information go to: