Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] EXTERNAL: Re: openmpi shared memory feature
From: Hodge, Gary C (gary.c.hodge_at_[hidden])
Date: 2012-10-30 09:51:45


FYI, recently, I was tracking down the source of page faults in our application that has real-time requirements. I found that disabling the sm component (--mca btl ^sm) eliminated many page faults I was seeing. I now have much better deterministic performance in that I no longer see outlier measurements (jobs that usually take 3 ms would sometimes take 15 ms). I did not notice a performance penalty using a network stack.

From: users-bounces_at_[hidden] [mailto:users-bounces_at_[hidden]] On Behalf Of Mahmood Naderan
Sent: Saturday, October 27, 2012 12:47 PM
To: Jeff Squyres
Cc: users_at_[hidden]
Subject: EXTERNAL: Re: [OMPI users] openmpi shared memory feature

>Because communicating through shared memory when sending messages between processes on the same server is far faster than going through a network stack.

I see... But that is not good for diskless clusters. Am I right? assume processes are on a node (which has no disk). In this case, their communication go though network (from computing node to server) then IO and then network again (from server to computing node).
Regards,
Mahmood

________________________________
From: Jeff Squyres <jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>>
To: Mahmood Naderan <nt_mahmood_at_[hidden]<mailto:nt_mahmood_at_[hidden]>>; Open MPI Users <users_at_[hidden]<mailto:users_at_[hidden]>>
Sent: Saturday, October 27, 2012 6:19 PM
Subject: Re: [OMPI users] openmpi shared memory feature

On Oct 27, 2012, at 10:49 AM, Mahmood Naderan wrote:

> Why openmpi uses shared memory model?

Because communicating through shared memory when sending messages between processes on the same server is far faster than going through a network stack.

> this can be disabled though by setting "--mca ^sm".
> It seems that by default openmpi uses such feature (shared memory backing files) which is strange.
>
> Regards,
> Mahmood
> _______________________________________________
> users mailing list
> users_at_[hidden]<mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users

--
Jeff Squyres
jsquyres_at_[hidden]<mailto:jsquyres_at_[hidden]>
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/