On Nov 22, 2011, at 1:09 AM, Shamik Ganguly wrote:
> I want to trace when the MPI library prevents an MPI_Send from going to the socket and makes it access shared memory because the target node is on the same chip (CMP). I want to use PIN to trace this. Can you please give me some pointers about which functions are taking this decision so that I can instrument the appropriate library calls in PIN?
The decision is made in the ob1 PML plugin. Way back during MPI_INIT, each MPI process creates lists of BTLs to use to contact each MPI process peer.
When a process is on the same *node* (e.g., a single server) -- not just the same processor socket/chip -- the shared memory BTL is given preference to all other BTLs by use of a priority mechanism. Hence, the "sm" BTL is put at the front of the BML lists (BML = BTL multiplexing layer -- it's essentially just list management for BTLs).
Later, when MPI_SEND comes through, it uses the already-setup BML lists to determine which BTL(s) to use to send a message.
That's the 50,000 foot view.
For corporate legal information go to: