Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] Infinipath context limit
From: Daniël Mantione (daniel.mantione_at_[hidden])
Date: 2008-02-06 05:57:06


Hello,

I am trying to use OpenMPI on a cluster with Infinipath and 8 core nodes.
I get these errors when using more than 4 processes:

node017.13311ipath_userinit: assign_port command failed: Device or
resource busy
[node017:13311] Open MPI failed to open a PSM endpoint: No free InfiniPath
contexts available on /dev/ipath
[node017:13311] Error in psm_ep_open (error No free ports could be
obtained)
node017.13315ipath_userinit: assign_port command failed: Device or
resource busy
[node017:13315] Open MPI failed to open a PSM endpoint: No free InfiniPath
contexts available on /dev/ipath
[node017:13315] Error in psm_ep_open (error No free ports could be
obtained)
node017.13314ipath_userinit: assign_port command failed: Device or
resource busy
node017.13313ipath_userinit: assign_port command failed: Device or
resource busy
[node017:13313] Open MPI failed to open a PSM endpoint: No free InfiniPath
contexts available on /dev/ipath
[node017:13313] Error in psm_ep_open (error No free ports could be
obtained)
[node017:13314] Open MPI failed to open a PSM endpoint: No free InfiniPath
contexts available on /dev/ipath
[node017:13314] Error in psm_ep_open (error No free ports could be
obtained)

The Infinipath User Guide writes this:

"Context Sharing Enabled: The MPI library provides PSM the local process layout
so that InfiniPath contexts available on each node can be shared if necessary; for
example, when running more node programs than contexts. By default, the
QLE7140 and QHT7140 have a maximum of four and eight sharable InfiniPath
contexts, respectively. Up to 4 node programs (from the same MPI job) can share
an InfiniPath context, for a total of 16 node programs per node for each QLE7140
and 32 node programs per node for each QHT7140.
The error message when this limit is exceeded is:

No free InfiniPath contexts available on /dev/ipath
"

It looks like OpenMPI is running into the context limit, apparently 4
inthis case. Can I do the context sharing mentioned with OpenMPI?

Best regards,

Daniël Mantione