Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] Patch to disable GCC __builtin_ operations
From: Josh Hursey (jjhursey_at_[hidden])
Date: 2011-06-09 08:51:36


The odd thing about this environment is that the head node seems to
have a slightly different setup than the compute nodes (not sure why
exactly, but that's what it is). So hwloc is configured and runs
correctly on the head node, but when it is asked to run on the compute
nodes it segvs at the call site of the __builtin_ functions.

For Open MPI, hwloc can only be built one way and needs to run on both
the compute and head nodes - so configuring hwloc two ways is not an
option. Further, to have two different installs I need to be able to
build on the compute nodes, which I cannot.

I suspect that the ABI compatibility of the libc interface is what is
enabling the remainder of the code to work in both environments, and
that the __builtin_ functions bypass that ABI to put in system
specific code that (for whatever reason) does not match on the compute
nodes.

I agree it is a somewhat exotic environment, and I am open to other
possible solutions. This patch does fix the problem, and so does
compiling with a non-gcc compiler - though essentially the same work
around. If there was a way to tell gcc directly to not allow
__builtin_ then I would do that, but no such option seems to exist.

Per Jeff's comment, we could turn it off by default in Open MPI, but
it would come at a slight performance penalty as Dave mentioned. Since
the number of environments in which this would be a problem are small,
I would suggest leaving it enabled in Open MPI and documenting the
configure option. For this system we already pass a large number of
configure options to get things to work correctly, this would just be
one more.

-- Josh

On Wed, Jun 8, 2011 at 5:58 PM, Samuel Thibault
<samuel.thibault_at_[hidden]> wrote:
> Josh Hursey, le Wed 08 Jun 2011 22:28:53 +0200, a écrit :
>> I hit a problem when installing hwloc statically on a machine with a
>> slightly different gcc support libraries and OSs on the head/compile
>> node versus the compute nodes. The builtin functions would cause hwloc
>> to segfault when run on the compute nodes. By disabling the builtin
>> operations, and using the more portable techniques seemed to do the
>> trick.
>
> This is very odd.  How does it segfault exactly?  Unless you are (or
> your distribution is) using an -march=something option, gcc is not
> supposed to produce code which can not be run on another machine.
>
> Samuel
> _______________________________________________
> hwloc-users mailing list
> hwloc-users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>
>

-- 
Joshua Hursey
Postdoctoral Research Associate
Oak Ridge National Laboratory
http://users.nccs.gov/~jjhursey