Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] Shared memory optimizations in OMPI
From: Shamik Ganguly (shamikg_at_[hidden])
Date: 2011-11-22 10:12:28


Thanks a lot Jeff.

PIN is a dynamic binary instrumentation tool from Intel. It runs on top of
the Binary in the MPI node. When its given function calls to instrument, it
will insert trappings before/after that funtion call in the binary of the
program you are instrumenting and you can insert your own functions.

I am doing some memory address profiling on benchmarks running on MPI and I
was using PIN to get the Load/Store addresses. Furthermore I needed to know
which LD/ST were coming from actual socket communication and which are
coming from shared memory optimizations. So i needed to know which
functions/where exactly were they taking that decision so that I can
instrument the appropriate MPI library function call (the actual low level
function, not the API like MPI_Sends/Recvs) in PIN. Hence I guess I am
actually zooming down to a 1000 ft view :)

Any suggestion is welcome. I will go into the ob1 directory and try to hunt
around to see how exactly its being done.

Regards,
Shamik

On Tue, Nov 22, 2011 at 10:08 AM, Shamik Ganguly
<shamik.ganguly_at_[hidden]>wrote:

> Thanks a lot Jeff.
>
> PIN is a dynamic binary instrumentation tool from Intel. It runs on top of
> the Binary in the MPI node. When its given function calls to instrument, it
> will insert trappings before/after that funtion call in the binary of the
> program you are instrumenting and you can insert your own functions.
>
> I am doing some memory address profiling on benchmarks running on MPI and
> I was using PIN to get the Load/Store addresses. Furthermore I needed to
> know which LD/ST were coming from actual socket communication and which are
> coming from shared memory optimizations. So i needed to know which
> functions/where exactly were they taking that decision so that I can
> instrument the appropriate MPI library function call (the actual low level
> function, not the API like MPI_Sends/Recvs) in PIN. Hence I guess I am
> actually zooming down to a 1000 ft view :)
>
> I will go into the ob1 directory and try to hunt around to see how exactly
> its being done.
>
> Regards,
> Shamik
>
>
> On Tue, Nov 22, 2011 at 9:04 AM, Jeff Squyres <jsquyres_at_[hidden]> wrote:
>
>> On Nov 22, 2011, at 1:09 AM, Shamik Ganguly wrote:
>>
>> > I want to trace when the MPI library prevents an MPI_Send from going
>> to the socket and makes it access shared memory because the target node is
>> on the same chip (CMP). I want to use PIN to trace this. Can you please
>> give me some pointers about which functions are taking this decision so
>> that I can instrument the appropriate library calls in PIN?
>>
>> What's PIN?
>>
>> The decision is made in the ob1 PML plugin. Way back during MPI_INIT,
>> each MPI process creates lists of BTLs to use to contact each MPI process
>> peer.
>>
>> When a process is on the same *node* (e.g., a single server) -- not just
>> the same processor socket/chip -- the shared memory BTL is given preference
>> to all other BTLs by use of a priority mechanism. Hence, the "sm" BTL is
>> put at the front of the BML lists (BML = BTL multiplexing layer -- it's
>> essentially just list management for BTLs).
>>
>> Later, when MPI_SEND comes through, it uses the already-setup BML lists
>> to determine which BTL(s) to use to send a message.
>>
>> That's the 50,000 foot view.
>>
>> --
>> Jeff Squyres
>> jsquyres_at_[hidden]
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>
>
>
> --
> Shamik Ganguly
>
>

-- 
Shamik Ganguly
2nd year, MS (CSE-Hardware), University of Michigan, Ann Arbor
B.Tech.(E&ECE), IITKGP (2008)