Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] slowdown with infiniband and latest CentOS kernel
From: Dave Love (d.love_at_[hidden])
Date: 2014-02-27 10:47:39


[I don't know what thread this is without References: or citation.]

Bernd Dammann <bd_at_[hidden]> writes:

> Hi,
>
> I found this thread from before Christmas, and I wondered what the
> status of this problem is. We experience the same problems since our
> upgrade to Scientific Linux 6.4, kernel 2.6.32-431.1.2.el6.x86_64, and
> OpenMPI 1.6.5.
>
> Users have reported severe slowdowns in all kinds of applications,
> like VASP, OpenFOAM, etc.

I'm surprised a kernel change should be related to core binding, if
that's the issue, or caused your slowdown. We were running that kernel
OK until recently with those sort of applications and that OMPI version.
(The change to the default alltoallv collective algorithm in the OMPI
1.6 series, discussed in the archives, might affect you if you upgraded
through it.)

> Using the workaround '--bind-to-core' does only make sense for those
> jobs, that allocate full nodes, but the majority of our jobs don't do
> that.

I don't consider it a workaround. Just use a resource manager that
sorts it out for you. For what it's worth, a recipe for SGE/OMPI is at
<http://arc.liv.ac.uk/SGE/howto/sge-configs.html#_core_binding>. We're
happy with that (and seem to be at least on a par with Intel using
OMPI+GCC+OpenBLAS) now users automatically get binding.