> Thanks for the fast answer. So is this latency normal for TCP
> communications over MPI!? Could RDMA maybe reduce the latency? It
> should work with those cards but there are still problems with OFED.
> iWARP is also one of the features they offer but if it works...
Yes, ~40us TCP latency is normal (it can be worse too).
If you need lower MPI latency you need to look elsewhere (but it's not
going to be TCP). Check SCore, OpenMX and Gamma. SCore is more mature of
the three but OpenMX looks promising too. We get less than 15 us using
SCore MPI and Intel NICs (IMB PingPong). Of course commercial MPI
libraries offer low latency too e.g. Scali MPI.
> Dresden University of Technology
> Center for Information Services
> and High Performance Computing (ZIH)
> D-01062 Dresden
> e-mail: Andy.Georgi_at_[hidden]
> WWW: http://www.tu-dresden.de/zih
> users mailing list
I. Kozin (i.kozin at dl.ac.uk)
Computational Science and Engineering Dept.
STFC Daresbury Laboratory
Daresbury Science and Innovation Centre
Daresbury, Warrington, WA4 4AD
tel: +44 (0) 1925 603308
fax: +44 (0) 1925 603634