Open MPI logo

Portable Hardware Locality (hwloc) Documentation: v1.1

  |   Home   |   Support   |   FAQ   |  

Memory binding

Enumerations

enum  hwloc_membind_policy_t {
  HWLOC_MEMBIND_DEFAULT, HWLOC_MEMBIND_FIRSTTOUCH, HWLOC_MEMBIND_BIND, HWLOC_MEMBIND_INTERLEAVE,
  HWLOC_MEMBIND_REPLICATE, HWLOC_MEMBIND_NEXTTOUCH
}
 

Memory binding policy.

More...
enum  hwloc_membind_flags_t {
  HWLOC_MEMBIND_PROCESS, HWLOC_MEMBIND_THREAD, HWLOC_MEMBIND_STRICT, HWLOC_MEMBIND_MIGRATE,
  HWLOC_MEMBIND_NOCPUBIND
}
 

Memory binding flags.

More...

Functions

int hwloc_set_membind_nodeset (hwloc_topology_t topology, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int flags)
 Bind current process memory on the given nodeset nodeset.
int hwloc_set_membind (hwloc_topology_t topology, hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int flags)
 Bind current process memory on memory nodes near the given cpuset cpuset.
int hwloc_get_membind_nodeset (hwloc_topology_t topology, hwloc_nodeset_t nodeset, hwloc_membind_policy_t *policy, int flags)
 Get current process memory binding in nodeset nodeset.
int hwloc_get_membind (hwloc_topology_t topology, hwloc_cpuset_t cpuset, hwloc_membind_policy_t *policy, int flags)
 Get current process memory binding in cpuset cpuset.
int hwloc_set_proc_membind_nodeset (hwloc_topology_t topology, hwloc_pid_t pid, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int flags)
 Bind given process memory on the given nodeset nodeset.
int hwloc_set_proc_membind (hwloc_topology_t topology, hwloc_pid_t pid, hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int flags)
 Bind given process memory on memory nodes near the given cpuset cpuset.
int hwloc_get_proc_membind_nodeset (hwloc_topology_t topology, hwloc_pid_t pid, hwloc_nodeset_t nodeset, hwloc_membind_policy_t *policy, int flags)
 Get current process memory binding in nodeset nodeset.
int hwloc_get_proc_membind (hwloc_topology_t topology, hwloc_pid_t pid, hwloc_cpuset_t cpuset, hwloc_membind_policy_t *policy, int flags)
 Get current process memory binding in cpuset cpuset.
int hwloc_set_area_membind_nodeset (hwloc_topology_t topology, const void *addr, size_t len, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int flags)
 Bind some memory range on the given nodeset nodeset.
int hwloc_set_area_membind (hwloc_topology_t topology, const void *addr, size_t len, hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int flags)
 Bind some memory range on memory nodes near the given cpuset cpuset.
int hwloc_get_area_membind_nodeset (hwloc_topology_t topology, const void *addr, size_t len, hwloc_nodeset_t nodeset, hwloc_membind_policy_t *policy, int flags)
 Get some memory range memory binding in nodeset nodeset.
int hwloc_get_area_membind (hwloc_topology_t topology, const void *addr, size_t len, hwloc_cpuset_t cpuset, hwloc_membind_policy_t *policy, int flags)
 Get some memory range memory binding in cpuset cpuset.
void * hwloc_alloc (hwloc_topology_t topology, size_t len)
 Allocate some memory.
void * hwloc_alloc_membind_nodeset (hwloc_topology_t topology, size_t len, hwloc_const_nodeset_t nodeset, hwloc_membind_policy_t policy, int flags)
 Allocate some memory on the given nodeset nodeset.
void * hwloc_alloc_membind (hwloc_topology_t topology, size_t len, hwloc_const_cpuset_t cpuset, hwloc_membind_policy_t policy, int flags)
 Allocate some memory on memory nodes near the given cpuset cpuset.
int hwloc_free (hwloc_topology_t topology, void *addr, size_t len)
 Free some memory allocated by hwloc_alloc() or hwloc_alloc_membind().

Detailed Description

Note:
Not all OSes support all ways to bind existing allocated memory (migration), future memory allocation, explicit memory allocation, etc. and the corresponding binding functions may fail. -1 is returned and errno is set to ENOSYS when it is not possible to bind the requested kind of object processes/threads). errno is set to EXDEV when the requested cpuset can not be enforced (e.g. some systems only allow one NUMA node).

The most portable version that should be preferred over the others, whenever possible, is

which will try to allocate new data bound to the given set, possibly by changing the current memory binding policy, or at worse allocate memory without binding it at all. Since HWLOC_MEMBIND_STRICT is not given, this will even not fail unless a mere malloc() itself would fail, i.e. ENOMEM.

Each binding is available with a CPU set argument or a NUMA memory node set argument. The name of the latter ends with _nodeset. It is also possible to convert between CPU set and node set using hwloc_cpuset_to_nodeset or hwloc_cpuset_from_nodeset.

Note:
On some OSes, memory binding may have effects on CPU binding, see HWLOC_MEMBIND_NOCPUBIND

Enumeration Type Documentation

Memory binding flags.

These flags can be used to refine the binding policy.

Note:
Not all systems support all kinds of binding.
Enumerator:
HWLOC_MEMBIND_PROCESS 

Set policy for all threads of the current (possibly multithreaded) process.

HWLOC_MEMBIND_THREAD 

Set policy for the current thread of the current process.

HWLOC_MEMBIND_STRICT 

Request strict binding from the OS. The function will fail if the binding can not be completely enforced.

HWLOC_MEMBIND_MIGRATE 

Migrate existing allocated memory. If memory can not be migrated and the STRICT flag is passed, an error will be returned.

HWLOC_MEMBIND_NOCPUBIND 

Avoid any effect on CPU binding.

On some OSes, some memory binding function would also bind the application on the corresponding CPUs. It is often not a problem for the application, but if it is, setting this flag will make hwloc avoid using OS functions that would also bind on CPUs. This will however reduce the support of memory bindings, i.e. potentially return ENOSYS in some cases.

Memory binding policy.

These can be used to choose the binding policy.

Note that not all systems support all kinds of binding.

Enumerator:
HWLOC_MEMBIND_DEFAULT 

Reset the memory allocation policy to the system default.

HWLOC_MEMBIND_FIRSTTOUCH 

Allocate memory on the given nodes, but preferably on the node where the first accessor is running.

HWLOC_MEMBIND_BIND 

Allocate memory on the given nodes.

HWLOC_MEMBIND_INTERLEAVE 

Allocate memory on the given nodes in a round-robin manner.

HWLOC_MEMBIND_REPLICATE 

Replicate memory on the given nodes.

HWLOC_MEMBIND_NEXTTOUCH 

On next touch of existing allocated memory, migrate it to the node where the memory reference happened.


Function Documentation

void* hwloc_alloc ( hwloc_topology_t  topology,
size_t  len 
)

Allocate some memory.

This is equivalent to malloc(), except it tries to allocated page-aligned memory from the OS.

Note:
The allocated memory should be freed with hwloc_free().
void* hwloc_alloc_membind ( hwloc_topology_t  topology,
size_t  len,
hwloc_const_cpuset_t  cpuset,
hwloc_membind_policy_t  policy,
int  flags 
)

Allocate some memory on memory nodes near the given cpuset cpuset.

Returns:
-1 with errno set to ENOSYS if the action is not supported and HWLOC_MEMBIND_STRICT is given
-1 with errno set to EXDEV if the binding cannot be enforced and HWLOC_MEMBIND_STRICT is given
Note:
The allocated memory should be freed with hwloc_free().
void* hwloc_alloc_membind_nodeset ( hwloc_topology_t  topology,
size_t  len,
hwloc_const_nodeset_t  nodeset,
hwloc_membind_policy_t  policy,
int  flags 
)

Allocate some memory on the given nodeset nodeset.

Returns:
-1 with errno set to ENOSYS if the action is not supported and HWLOC_MEMBIND_STRICT is given
-1 with errno set to EXDEV if the binding cannot be enforced and HWLOC_MEMBIND_STRICT is given
Note:
The allocated memory should be freed with hwloc_free().
int hwloc_free ( hwloc_topology_t  topology,
void *  addr,
size_t  len 
)

Free some memory allocated by hwloc_alloc() or hwloc_alloc_membind().

int hwloc_get_area_membind ( hwloc_topology_t  topology,
const void *  addr,
size_t  len,
hwloc_cpuset_t  cpuset,
hwloc_membind_policy_t policy,
int  flags 
)

Get some memory range memory binding in cpuset cpuset.

int hwloc_get_area_membind_nodeset ( hwloc_topology_t  topology,
const void *  addr,
size_t  len,
hwloc_nodeset_t  nodeset,
hwloc_membind_policy_t policy,
int  flags 
)

Get some memory range memory binding in nodeset nodeset.

int hwloc_get_membind ( hwloc_topology_t  topology,
hwloc_cpuset_t  cpuset,
hwloc_membind_policy_t policy,
int  flags 
)

Get current process memory binding in cpuset cpuset.

int hwloc_get_membind_nodeset ( hwloc_topology_t  topology,
hwloc_nodeset_t  nodeset,
hwloc_membind_policy_t policy,
int  flags 
)

Get current process memory binding in nodeset nodeset.

int hwloc_get_proc_membind ( hwloc_topology_t  topology,
hwloc_pid_t  pid,
hwloc_cpuset_t  cpuset,
hwloc_membind_policy_t policy,
int  flags 
)

Get current process memory binding in cpuset cpuset.

int hwloc_get_proc_membind_nodeset ( hwloc_topology_t  topology,
hwloc_pid_t  pid,
hwloc_nodeset_t  nodeset,
hwloc_membind_policy_t policy,
int  flags 
)

Get current process memory binding in nodeset nodeset.

int hwloc_set_area_membind ( hwloc_topology_t  topology,
const void *  addr,
size_t  len,
hwloc_const_cpuset_t  cpuset,
hwloc_membind_policy_t  policy,
int  flags 
)

Bind some memory range on memory nodes near the given cpuset cpuset.

Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_area_membind_nodeset ( hwloc_topology_t  topology,
const void *  addr,
size_t  len,
hwloc_const_nodeset_t  nodeset,
hwloc_membind_policy_t  policy,
int  flags 
)

Bind some memory range on the given nodeset nodeset.

Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_membind ( hwloc_topology_t  topology,
hwloc_const_cpuset_t  cpuset,
hwloc_membind_policy_t  policy,
int  flags 
)

Bind current process memory on memory nodes near the given cpuset cpuset.

Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_membind_nodeset ( hwloc_topology_t  topology,
hwloc_const_nodeset_t  nodeset,
hwloc_membind_policy_t  policy,
int  flags 
)

Bind current process memory on the given nodeset nodeset.

Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_proc_membind ( hwloc_topology_t  topology,
hwloc_pid_t  pid,
hwloc_const_cpuset_t  cpuset,
hwloc_membind_policy_t  policy,
int  flags 
)

Bind given process memory on memory nodes near the given cpuset cpuset.

Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced
int hwloc_set_proc_membind_nodeset ( hwloc_topology_t  topology,
hwloc_pid_t  pid,
hwloc_const_nodeset_t  nodeset,
hwloc_membind_policy_t  policy,
int  flags 
)

Bind given process memory on the given nodeset nodeset.

Returns:
-1 with errno set to ENOSYS if the action is not supported
-1 with errno set to EXDEV if the binding cannot be enforced