Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Michael Edwards (miedward_at_[hidden])
Date: 2007-06-06 17:44:18


I am runing open-mpi 1.1.1-1 compiled from OFED1.1 which I downloaded
from their website.

I am using SGE installed via OSCAR 5.0 and when running under SGE I
get the "mca_mpool_openib_register: ibv_reg_mr(0x590000,528384) failed
with error: Cannot allocate memory" error discussed at length in your
FAQ.

When I run from the command line using mpirun, I don't get the errors.
 Of course, I don't know how to tell if the code is actually using the
IB interface instead of the GigE network...

I tried the suggestions in the FAQ regarding setting the memlock
parameter in /etc/security/limits.conf: and all the nodes return
"unlimited" in response to "ulimit -l" after rebooting the nodes. The
problem persists under SGE and still does not appear when simply using
mpirun.

I assumed it would work since openmpi 1.1.1 was included as working
with SGE in OSCAR 5.0, but I don't know how different that version and
the one included with OFED is.

Any suggestions would be appreciated.