Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] SM BTL NUMA awareness patches
From: Lenny Verkhovsky (lenny.verkhovsky_at_[hidden])
Date: 2008-06-12 10:35:50


OK,
I will commit it next week.
I did saw performance improvement in the worst scenario. I believe that with
increasing numbers of CPUs the improvement will more noticable.

On Thu, Jun 12, 2008 at 1:00 AM, Brad Benton <bradford.benton_at_[hidden]>
wrote:

> Lenny,
>
> I've looked over the code more and did some initial tests with it. It
> didn't seem to hurt anything in the default case. I also consulted with
> George and he would like to see these patches get in for 1.3. Since it
> seems to do no harm in the default case, I am okay with that as well. So,
> unless anyone else has objections, please go ahead and apply this to the
> trunk.
>
> BTW, in your testing, were you able to measure any noticeable performance
> improvements?
>
> Thanks & Regards,
> --brad
>
>
>
> On Tue, Jun 10, 2008 at 2:32 PM, Brad Benton <bradford.benton_at_[hidden]>
> wrote:
>
>> Hi Lenny,
>>
>> My apologies for not replying sooner. I would like to look at these
>> patches a bit more. Since this adds a feature (NUMA awareness in the SM
>> BTL) as well as introduces interface changes to the maffinity framework, I
>> would also like to get George's opinion before deciding whether or not go
>> bring this into the trunk before branching for 1.3.
>>
>> Regards,
>> --Brad
>>
>>
>>
>> On Tue, Jun 10, 2008 at 10:52 AM, Lenny Verkhovsky <
>> lenny.verkhovsky_at_[hidden]> wrote:
>>
>>> Hi,
>>> I didn't want to bring it on the teleconference
>>> but I want to commit Gleb's NUMA awareness patch before you branching
>>> trunk.
>>> Since I didn't get any objection / response about it I guess it's OK.
>>>
>>> Best Regards
>>> Lenny,
>>>
>>> ---------- Forwarded message ----------
>>> From: Lenny Verkhovsky <lenny.verkhovsky_at_[hidden]>
>>> Date: Tue, Jun 3, 2008 at 2:38 PM
>>> Subject: [OMPI devel] SM BTL NUMA awareness patches
>>> To: Open MPI Developers <devel_at_[hidden]>
>>>
>>>
>>> Hi, all,
>>> If there are no comments for this patch
>>> I can commit it.
>>>
>>> Lenny.
>>>
>>> ---------- Forwarded message ----------
>>> From: Gleb Natapov <glebn_at_[hidden]>
>>> Date: 2008/5/28
>>> Subject: [OMPI devel] SM BTL NUMA awareness patches
>>> To: devel_at_[hidden]
>>>
>>>
>>> Hi,
>>>
>>> Attached two patches implement NUMA awareness in SM BTL. The first one
>>> adds two new functions to maffinity framework required by the second
>>> patch. The functions are:
>>>
>>> opal_maffinity_base_node_name_to_id() - gets a string that represents a
>>> memory node name and translates
>>> it to memory node id.
>>> opal_maffinity_base_bind() - binds an address range to
>>> specific
>>> memory node.
>>>
>>> The bind() function cannot be implemented by all maffinity components.
>>> (There is no way first_use maffinity component can implement such
>>> functionality). In this case this function can be set to NULL.
>>>
>>> The second one adds NUMA awareness support to SM BTL and SM MPOOL. Each
>>> process determines what CPU it is running on and exchange this info with
>>> other local processes. Each process creates separate MPOOL for every
>>> memory node available and use them to allocate memory on specific memory
>>> nodes if needed. For instance circular buffer memory is always allocated
>>> on memory node local to receiver process.
>>>
>>> To use this on a Linux machine carto file with HW topology description
>>> should
>>> be provided. Processes should be bound to specific CPU (by specifying
>>> rank file for instance) and session directory should be created on tmpfs
>>> file system (otherwise Linux ignores memory binding commands) by
>>> setting orte_tmpdir_base parameter to point to tmpfs mount point.
>>>
>>> Questions and suggestion are alway welcome.
>>>
>>> --
>>> Gleb.
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>>
>>>
>>
>