Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] [hwloc-announce] Hardware locality (hwloc) v1.1rc1 released
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2010-11-11 05:11:31


Le 11/11/2010 02:31, Samuel Thibault a écrit :
>> get_mempolicy: Invalid argument
>> hwloc_get_membind failed (errno 22 Invalid argument)
>>
>
> Could you try to increase the value of max_os_index?
>
> I can see in the kernel source code the following in sys_get_mempolicy:
>
> if (nmask != NULL && maxnode < MAX_NUMNODES)
> return -EINVAL;
>
> and MAX_NUMNODES depends on .config ...
>

And indeed MAX_NUMNODES is (1<<CONFIG_NODES_SHIFT) and
CONFIG_NODES_SHIFT=9 on rhel6 kernels. We pass a single ulong to the
kernel, so it's not large enough to store 1<<9 bits. We couldn't
reproduce on Debian and RHEL5 since NODE_SHIFT=6 there.

We had to loop until we found the kernel NR_CPUS for sched_getaffinity,
we can do the same to find the kernel MAX_NUMNODES for get_mempolicy.
The attached patch may help. Only slightly tested obviously since I
don't have any kernel causing the problem.

Brice