Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] slowdown with infiniband and latest CentOS kernel
From: Dave Love (d.love_at_[hidden])
Date: 2014-02-27 10:47:39


[I don't know what thread this is without References: or citation.]

Bernd Dammann <bd_at_[hidden]> writes:

> Hi,
>
> I found this thread from before Christmas, and I wondered what the
> status of this problem is. We experience the same problems since our
> upgrade to Scientific Linux 6.4, kernel 2.6.32-431.1.2.el6.x86_64, and
> OpenMPI 1.6.5.
>
> Users have reported severe slowdowns in all kinds of applications,
> like VASP, OpenFOAM, etc.

I'm surprised a kernel change should be related to core binding, if
that's the issue, or caused your slowdown. We were running that kernel
OK until recently with those sort of applications and that OMPI version.
(The change to the default alltoallv collective algorithm in the OMPI
1.6 series, discussed in the archives, might affect you if you upgraded
through it.)

> Using the workaround '--bind-to-core' does only make sense for those
> jobs, that allocate full nodes, but the majority of our jobs don't do
> that.

I don't consider it a workaround. Just use a resource manager that
sorts it out for you. For what it's worth, a recipe for SGE/OMPI is at
<http://arc.liv.ac.uk/SGE/howto/sge-configs.html#_core_binding>. We're
happy with that (and seem to be at least on a par with Intel using
OMPI+GCC+OpenBLAS) now users automatically get binding.