Yes, the buffer was being re-used. No we didnt try to benchmark it with netpipe and other stuffs. But the program was pretty simple. Do you think, I need to test it with bigger chunks (&gt;8MB) for communication.?<br />We also tried manipulating eager_limit and min_rdma_sze, but no success.<br /><br />Neeraj<br /><br />On Fri, 12 Oct 2007 13:00:10 +0200 Open MPI Users  wrote<br /><br />  Hello,<br /><br />  &gt;        The code was pretty simple. I was trying to send 8MB data from one<br /><br />  &gt;    rank to other in a loop(say 1000 iterations). And then i was taking the<br /><br />  &gt;    average of time taken and was calculating the bandwidth.<br /><br />  &gt; <br /><br />  &gt;    The above logic i tried with both mpirun-with-mca-parameters and without<br /><br />  &gt;    any parameters. And to my surprise, the performance was degrading when i<br /><br />  &gt;    was trying to manipulate.<br /><br />  That sounds strange. So did you re-use the communication buffers? Did<br /><br />  you try to run some existing benchmarks like Netpipe [1], IMB or<br /><br />  Netgauge [2]?<br /><br />  <br /><br />  &gt;    Now I have another question in mind. Is it possible to have IB Hardware<br /><br />  &gt;    Multicast implementation in OpenMPI? I have gone through the<br /><br />  &gt;    issues/challenges for the same, but also read couple of people who have<br /><br />  &gt;    successfully done it for Ethernet/Giga-bit Ethernet and IPoIB ofcourse in<br /><br />  &gt;    experimental stage. Actually i want to contribute for it in OpenMPI and<br /><br />  &gt;    need the help for the same.<br /><br />  As far as I know, there are two groups/people working on this. Andy<br /><br />  Friedley implements a "traditional" ACK based approach (like the one<br /><br />  that the OSU folks published about some time ago) and I implemented a<br /><br />  new idea for extreme scale (see "A practically constant-time MPI<br /><br />  Broadcast Algorithm for large-scale InfiniBand Clusters with<br /><br />  Multicast" [3]). I know that my version is still unstable and has some<br /><br />  problems. But I'm working on this.<br /><br />  <br /><br />  Best,<br /><br />    Torsten<br /><br />  <br /><br />  [1]: http://www.scl.ameslab.gov/netpipe/<br /><br />  [2]: http://www.unixer.de/research/netgauge/<br /><br />  [3]: https://www.unixer.de/publications/#hoefler-cac07<br /><br />  <br /><br />  -- <br /><br />   bash$ :(){ :|:&amp;};: --------------------- http://www.unixer.de/ -----<br /><br />  Computer scientists are the historians of computing.  -- Gordon Bell<br /><br />  _______________________________________________<br /><br />  users mailing list<br /><br />  users@open-mpi.org<br /><br />  http://www.open-mpi.org/mailman/listinfo.cgi/users<br /><br />  
<br>