Le 01/12/2012 12:45, Leta Melkamu a écrit :
some doubts on the use of knem with openmpi, everything works
it is a bit not clear on the usage of knem flags while running
my open-mpi program.
like --mca btl_sm_knem_dma_min 4860 is enough or I have to add
more flag like --mca btl_sm_eager_limit 4276 in the same run?
or can you please suggest me a good documentation link about
knem flag usage, I have tried to look around but no good info
regarding those staff. otherwise I will end up to test each
command with different flag value for each run.
What are you trying to do?
* Use knem for direct copy through kernel for medium/large messages:
(common use case)
"--mca btl_sm_use_knem 1" is enough. You can "cat /dev/knem" before
and after a run to see that knem counters have increased, which
means OMPI successfully passed some copy requests to knem.
* Use knem for short messages:
"--mca btl_sm_eager_limit 4276" may help. But I am not sure that's a
good idea since knem was designed for large messages.
* Offload knem copies to I/OAT hardware on Intel servers. That's
what "btl_sm_knem_dma_min" is for. Not sure you really want to do
that either, it's not much interesting on current Intel servers.