Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] some NUMA / affinity questions
From: Mark Kosmowski (mark.kosmowski_at_[hidden])
Date: 2008-03-24 06:29:32


I have a successful ompi installation and my software runs across my
humble cluster of three dual-Opteron (single core) nodes on OpenSUSE
10.2. I'm planning to upgrade some RAM soon and have been thinking of
playing with affinity, since each cpu will have it's own DIMMs after
the upgrade. I have read the FAQ and know to use "--mca
mpi_paffinity_alone 1" to enable affinity.

It looks like I am running ompi 1.1.4 (see below).

mark_at_LT:~> ompi_info | grep affinity
           MCA paffinity: linux (MCA v1.0, API v1.0, Component v1.1.4)
           MCA maffinity: first_use (MCA v1.0, API v1.0, Component v1.1.4)
           MCA maffinity: libnuma (MCA v1.0, API v1.0, Component v1.1.4)

Does this old version of ompi do a good job of implementing affinity
or would it behoove me to use the current version if I am interested
in trying affinity?

What sorts of time gains do people typically see with affinity? (I'm
a chemistry student running planewave solid state calculation software
if this helps with the question).

Lastly, two of the three machines will have all of their DIMM slots
populated by equal sized DIMMs. However, one of my machines has two
processors, each of which having four DIMM slots. This machine will
be getting 4 @ 1 Gb DIMMs and 2 @ 2 Gb DIMMs. i am assuming that the
best thing for affinity would be to put all of the 1 Gb DIMMs to one
processor and the 2 Gb DIMMs to the other and to put the 2 Gb DIMMs in
slots 0 and 1. Does it matter which processor gets which set of
DIMMs?

Thank you,

Mark