Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] Thread binding problem
From: Samuel Thibault (samuel.thibault_at_[hidden])
Date: 2012-09-06 04:48:26


Samuel Thibault, le Thu 06 Sep 2012 10:45:45 +0200, a écrit :
> Gabriele Fatigati, le Thu 06 Sep 2012 10:12:38 +0200, a écrit :
> > mbind hwloc_linux_set_area_membind()  fails:
> >
> > Error from HWLOC mbind: Cannot allocate memory 
>
> Ok. mbind is not really supposed to allocate much memory, but it still
> does allocate some, to record the policy
>
> > // hwloc_obj_t obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_NODE, tid);
> > hwloc_obj_t obj = hwloc_get_obj_by_type(topology, HWLOC_OBJ_PU, tid);
> > hwloc_cpuset_t cpuset = hwloc_bitmap_dup(obj->cpuset);
> > hwloc_bitmap_singlify(cpuset);
> > hwloc_set_cpubind(topology, cpuset, HWLOC_CPUBIND_THREAD);
> >
> > for( i = chunk*tid; i < len; i+=PAGE_SIZE) {
> > // res = hwloc_set_area_membind_nodeset(topology, &array[i], PAGE_SIZE, obj->nodeset, HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_THREAD);
> > res = hwloc_set_area_membind(topology, &array[i], PAGE_SIZE, cpuset, HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_THREAD);
>
> and I'm afraid that calling set_area_membind for each page might be too
> dense: the kernel is probably allocating a memory policy record for each
> page, not being able to merge adjacent equal policies.

I forgot to mention: the amount of memory allocated by each mbind call
can be controlled through the configured maximum number of nodes in the
kernel (CONFIG_NODES_SHIFT).

Samuel