Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: [OMPI devel] Fwd: SM BTL NUMA awareness patches
From: Lenny Verkhovsky (lenny.verkhovsky_at_[hidden])
Date: 2008-06-10 11:52:18

I didn't want to bring it on the teleconference
but I want to commit Gleb's NUMA awareness patch before you branching trunk.
Since I didn't get any objection / response about it I guess it's OK.

Best Regards

---------- Forwarded message ----------
From: Lenny Verkhovsky <lenny.verkhovsky_at_[hidden]>
Date: Tue, Jun 3, 2008 at 2:38 PM
Subject: [OMPI devel] SM BTL NUMA awareness patches
To: Open MPI Developers <devel_at_[hidden]>

Hi, all,
If there are no comments for this patch
I can commit it.


---------- Forwarded message ----------
From: Gleb Natapov <glebn_at_[hidden]>
Date: 2008/5/28
Subject: [OMPI devel] SM BTL NUMA awareness patches
To: devel_at_[hidden]


Attached two patches implement NUMA awareness in SM BTL. The first one
adds two new functions to maffinity framework required by the second
patch. The functions are:

 opal_maffinity_base_node_name_to_id() - gets a string that represents a
                                        memory node name and translates
                                        it to memory node id.
 opal_maffinity_base_bind() - binds an address range to specific
                                        memory node.

The bind() function cannot be implemented by all maffinity components.
(There is no way first_use maffinity component can implement such
functionality). In this case this function can be set to NULL.

The second one adds NUMA awareness support to SM BTL and SM MPOOL. Each
process determines what CPU it is running on and exchange this info with
other local processes. Each process creates separate MPOOL for every
memory node available and use them to allocate memory on specific memory
nodes if needed. For instance circular buffer memory is always allocated
on memory node local to receiver process.

To use this on a Linux machine carto file with HW topology description
be provided. Processes should be bound to specific CPU (by specifying
rank file for instance) and session directory should be created on tmpfs
file system (otherwise Linux ignores memory binding commands) by
setting orte_tmpdir_base parameter to point to tmpfs mount point.

Questions and suggestion are alway welcome.

devel mailing list