FYI, recently, I was tracking down the source of page faults in our application that has real-time requirements. I found that disabling the sm component (--mca btl ^sm) eliminated many page faults I was seeing. I now have much better deterministic performance in that I no longer see outlier measurements (jobs that usually take 3 ms would sometimes take 15 ms). I did not notice a performance penalty using a network stack.
>Because communicating through shared memory when sending messages between processes on the same server is far faster than going through a network stack.
I see... But that is not good for diskless clusters. Am I right? assume processes are on a node (which has no disk). In this case, their communication go though network (from computing node to server) then IO and then network again (from server to computing node).
From: Jeff Squyres <firstname.lastname@example.org>
To: Mahmood Naderan <email@example.com>; Open MPI Users <firstname.lastname@example.org>
Sent: Saturday, October 27, 2012 6:19 PM
Subject: Re: [OMPI users] openmpi shared memory feature
On Oct 27, 2012, at 10:49 AM, Mahmood Naderan wrote:
> Why openmpi uses shared memory model?
Because communicating through shared memory when sending messages between processes on the same server is far faster than going through a network stack.
> this can be disabled though by setting "--mca ^sm".
> It seems that by default openmpi uses such feature (shared memory backing files) which is strange.
> users mailing list
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/