Sorry, i forgot to introduce the system.. Ours is the customized OFED stack implemented to work on the specific hardware.. We tested the stack with the q-perf and Intel Benchmarks(IMB-3.2.2).. they went fine.. We want to execute the osu_benchamark3.1.1 suite on our OFED..<br>
<br><div class="gmail_quote">On Wed, Feb 29, 2012 at 9:57 PM, Venkateswara Rao Dokku <span dir="ltr">&lt;<a href="mailto:dvrao.584@gmail.com">dvrao.584@gmail.com</a>&gt;</span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
Hiii,<br>I tried executing osu_benchamarks-3.1.1 suite with the openmpi-1.4.3... I could run 10 bench-mark tests (except osu_put_bibw,osu_put_bw,osu_<div>get_bw,osu_latency_mt)
 out of 14 tests in the bench-mark suite... and the remaining tests are 
hanging at some message size.. the output is shown below<div>
<br></div><div><div>[root@test2 ~]# mpirun --prefix /usr/local/ -np 2 
--mca btl openib,self,sm -H 192.168.0.175,192.168.0.174 --mca 
orte_base_help_aggregate 0 /root/ramu/ofed_pkgs/osu_benchmarks-3.1.1/osu_put_bibw</div><div>
failed to create doorbell file /dev/plx2_char_dev </div><div>--------------------------------------------------------------------------</div><div>WARNING: No preset parameters were found for the device that Open MPI</div>



<div>detected:</div><div><br></div><div>  Local host:            test1</div><div>  Device name:           plx2_0</div><div>  Device vendor ID:      0x10b5</div><div>  Device vendor part ID: 4277</div><div><br></div><div>


Default device parameters will be used, which may result in lower</div>
<div>performance.  You can edit any of the files specified by the</div><div>btl_openib_device_param_files MCA parameter to set values for your</div><div>device.</div><div><br></div><div>NOTE: You can turn off this warning by setting the MCA parameter</div>



<div>      btl_openib_warn_no_device_params_found to 0.</div><div>--------------------------------------------------------------------------</div><div>failed to create doorbell file /dev/plx2_char_dev </div><div>--------------------------------------------------------------------------</div>



<div>WARNING: No preset parameters were found for the device that Open MPI</div><div>detected:</div><div><br></div><div>  Local host:            test2</div><div>  Device name:           plx2_0</div><div>  Device vendor ID:      0x10b5</div>



<div>  Device vendor part ID: 4277</div><div><br></div><div>Default device parameters will be used, which may result in lower</div><div>performance.  You can edit any of the files specified by the</div><div>btl_openib_device_param_files MCA parameter to set values for your</div>



<div>device.</div><div><br></div><div>NOTE: You can turn off this warning by setting the MCA parameter</div><div>      btl_openib_warn_no_device_params_found to 0.</div><div>--------------------------------------------------------------------------</div>



<div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div>



<div># OSU One Sided MPI_Put Bi-directional Bandwidth Test v3.1.1</div><div># Size     Bi-Bandwidth (MB/s)</div><div>plx2_create_qp line: 415 </div><div>plx2_create_qp line: 415 </div><div>plx2_create_qp line: 415 </div>


<div>
plx2_create_qp line: 415 </div><div>1                         0.00</div><div>2                         0.00</div><div>4                         0.01</div><div>8                         0.03</div><div>16                        0.07</div>



<div>32                        0.15</div><div>64                        0.11</div><div>128                       0.21</div><div>256                       0.43</div><div>512                       0.88</div><div>1024                      2.10</div>



<div>2048                      4.21</div><div>4096                      8.10</div><div>8192                     16.19</div><div>16384                     8.46</div><div>32768                    20.34</div><div>65536                    39.85</div>



<div>131072                   84.22</div><div>262144                  142.23</div><div>524288                  234.83</div><div>mpirun: killing job...</div><div><br></div><div>--------------------------------------------------------------------------</div>



<div>mpirun noticed that process rank 0 with PID 7305 on node test2 exited on signal 0 (Unknown signal 0).</div><div>--------------------------------------------------------------------------</div><div>2 total processes killed (some possibly by mpirun during cleanup)</div>



<div>mpirun: clean termination accomplished</div><div><br></div><div><div>[root@test2
 ~]# mpirun --prefix /usr/local/ -np 2 --mca btl openib,self,sm -H 
192.168.0.175,192.168.0.174 --mca orte_base_help_aggregate 0 
/root/ramu/ofed_pkgs/osu_benchmarks-3.1.1/osu_put_bw</div>
<div>failed to create doorbell file /dev/plx2_char_dev </div><div>--------------------------------------------------------------------------</div><div>WARNING: No preset parameters were found for the device that Open MPI</div>



<div>detected:</div><div><br></div><div>  Local host:            test1</div><div>  Device name:           plx2_0</div><div>  Device vendor ID:      0x10b5</div><div>  Device vendor part ID: 4277</div><div><br></div><div>


Default device parameters will be used, which may result in lower</div>
<div>performance.  You can edit any of the files specified by the</div><div>btl_openib_device_param_files MCA parameter to set values for your</div><div>device.</div><div><br></div><div>NOTE: You can turn off this warning by setting the MCA parameter</div>



<div>      btl_openib_warn_no_device_params_found to 0.</div><div>--------------------------------------------------------------------------</div><div>failed to create doorbell file /dev/plx2_char_dev </div><div>--------------------------------------------------------------------------</div>



<div>WARNING: No preset parameters were found for the device that Open MPI</div><div>detected:</div><div><br></div><div>  Local host:            test2</div><div>  Device name:           plx2_0</div><div>  Device vendor ID:      0x10b5</div>



<div>  Device vendor part ID: 4277</div><div><br></div><div>Default device parameters will be used, which may result in lower</div><div>performance.  You can edit any of the files specified by the</div><div>btl_openib_device_param_files MCA parameter to set values for your</div>



<div>device.</div><div><br></div><div>NOTE: You can turn off this warning by setting the MCA parameter</div><div>      btl_openib_warn_no_device_params_found to 0.</div><div>--------------------------------------------------------------------------</div>



<div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div><div>alloc_srq max: 512 wqe_shift: 5</div>



<div># OSU One Sided MPI_Put Bandwidth Test v3.1.1</div><div># Size        Bandwidth (MB/s)</div><div>plx2_create_qp line: 415 </div><div>plx2_create_qp line: 415 </div><div>plx2_create_qp line: 415 </div><div>plx2_create_qp line: 415 </div>



<div>1                         0.02</div><div>2                         0.05</div><div>4                         0.10</div><div>8                         0.19</div><div>16                        0.39</div><div>32                        0.77</div>



<div>64                        1.53</div><div>128                       2.57</div><div>256                       4.16</div><div>512                       8.30</div><div>1024                     16.62</div><div>2048                     33.22</div>



<div>4096                     66.51</div><div>8192                     42.45</div><div>16384                    11.99</div><div>32768                    18.20</div><div>65536                    76.04</div><div>131072                   98.64</div>



<div>262144                  407.66</div><div>524288                  489.84</div><div>mpirun: killing job...</div><div><br></div><div>--------------------------------------------------------------------------</div><div>


mpirun noticed that process rank 0 with PID 7314 on node test2 exited on signal 0 (Unknown signal 0).</div>
<div>--------------------------------------------------------------------------</div><div>2 total processes killed (some possibly by mpirun during cleanup)</div><div>mpirun: clean termination accomplished</div><div><br></div>



</div><div>I even checked the logs but i couldn&#39;t see any errors... </div><div>Could you suggest a way to overcome/debug this issue..</div><div><br></div><div>Thanks for the kind reply..</div></div></div><br clear="all">
<font color="#888888">

<br>-- <br><span style="color:rgb(51,51,255)">Thanks &amp; Regards,<br>D.Venkateswara Rao,</span><br style="color:rgb(51,51,255)"><span style="color:rgb(51,51,255)">Software Engineer,One Convergence Devices Pvt Ltd.,<br>

Jubille Hills,Hyderabad.<br>
</span><br>
</font></blockquote></div><br><br clear="all"><br>-- <br><span style="color:rgb(51,51,255)">Thanks &amp; Regards,<br>D.Venkateswara Rao,</span><br style="color:rgb(51,51,255)"><span style="color:rgb(51,51,255)">Software Engineer,One Convergence Devices Pvt Ltd.,<br>
Jubille Hills,Hyderabad.<br></span><br>