Note that "eager" RDMA is only used for short messages -- it's not
really relevant to whether the same user buffers are re-used or not
(the mpi_leave_pinned parameter for long messages is only useful if
long buffers are re-used). See this FAQ item:
For benchmarks (like SKAMPI) that re-use long buffers, you might want
to use the mpi_leave_pinned MCA parameter:
On Jun 5, 2008, at 9:47 AM, Gabriele Fatigati wrote:
> i'm testing SKaMPI Benchmark on IBM Blade System over Infiniband.
> Current version of OpenMPI is 1.2.6
> I have tried to disable RDMA setting btl_openib_use_eager_rdma = 0.
> But, i have noted that, in MPI collectives execution time, there are
> few difference beetween RDMA active and none. Before tests, I
> expected that with RDMA off, excecution time was more long.
> So, i suppose that SKaMPI benchmark does continues reallocation of
> buffers that forbid benefits of RDMA protocol. Indeed, if initial
> buffer address change every time, we have to do very much
> registration of memory pages afterwards decay of perfomance.
> I used RDMA pipeline protocol. This protocol should makes no
> assumption about the application reuse of source and target buffers.
> But, is it every true?
> Parameters net are explained below.
> MCA btl: parameter "btl_openib_mpool" (current value: "rdma")
> MCA btl: parameter "btl_openib_ib_max_rdma_dst
> _ops" (current value: "4")
> MCA btl: parameter "btl_openib_use_eager_rdma" (current value: "1")
> MCA btl: parameter "btl_openib_eager_rdma_threshold" (current value:
> MCA btl: parameter "btl_openib_max_eager_rdma" (current value: "16")
> MCA btl: parameter "btl_openib_eager_rdma_num" (current value: "16")
> MCA btl: parameter "btl_openib_min_rdma_size" (current value:
> MCA btl: parameter "btl_openib_max_rdma_size" (current value:
> Gabriele Fatigati
> CINECA Systems & Tecnologies Department
> Supercomputing Group
> Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy
> www.cineca.it Tel: +39 051 6171722
> g.fatigati_at_[hidden] _______________________________________________
> users mailing list