Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] users Digest, Vol 1217, Issue 2, Message3
From: jan (jan_at_[hidden])
Date: 2009-05-21 22:33:11


Thanks Jeff. We solve this problem finally. Download the newest
OFED-1.4.1-rc6.tgz, and reinstall all node's infiniband drivers and
utilities. Everythings looks good, and I have my own coffee time now. Thanks
again.

Best Regards,

Gloria Jan
Wavelink Technology Inc

>I don't think the speed of the down port matters; port_down means that
>there's no cable connected, so the values are probably fairly random.
>
>
> On May 7, 2009, at 10:38 PM, jan wrote:
>
>> Anyone can help me to find out problem or bug in my cluster? The output
>> of "ibv_devinfo -v" from Dell blade with infiniband module look very
>> strange. The phys_port_cnt is 2, one active, and another down. The
>> active port is 20x speed, the down port is 10x speed. We are using Dell
>> PowerEdge M600 Blade Serverwith Mellanox ConnectX DDR infiniband
>> Mezzanine card and Cisco M2401G infiniband switch. OS is centOS 5.3,
>> kernel 2.6.18-128.1.6el5 with PGI V7.2-5 compiler, and OFED-1.4.1-rc4
>> with openmpi-1.3.2:
>>
>> # ibv_devinfo -v
>> hca_id: mlx4_0
>> fw_ver: 2.5.000
>> node_guid: 0018:8b90:97fe:73cd
>> sys_image_guid: 0018:8b90:97fe:73d0
>> vendor_id: 0x02c9
>> vendor_part_id: 25418
>> hw_ver: 0xA0
>> board_id: DEL08C0000002
>> phys_port_cnt: 2
>> max_mr_size: 0xffffffffffffffff
>> page_size_cap: 0xfffff000
>> max_qp: 131008
>> max_qp_wr: 16351
>> device_cap_flags: 0x000c1c66
>> max_sge: 32
>> max_sge_rd: 0
>> max_cq: 65408
>> max_cqe: 4194303
>> max_mr: 131056
>> max_pd: 32764
>> max_qp_rd_atom: 16
>> max_ee_rd_atom: 0
>> max_res_rd_atom: 2096128
>> max_qp_init_rd_atom: 128
>> max_ee_init_rd_atom: 0
>> atomic_cap: ATOMIC_HCA (1)
>> max_ee: 0
>> max_rdd: 0
>> max_mw: 0
>> max_raw_ipv6_qp: 0
>> max_raw_ethy_qp: 0
>> max_mcast_grp: 8192
>> max_mcast_qp_attach: 56
>> max_total_mcast_qp_attach: 458752
>> max_ah: 0
>> max_fmr: 0
>> max_srq: 65472
>> max_srq_wr: 16383
>> max_srq_sge: 31
>> max_pkeys: 128
>> local_ca_ack_delay: 15
>> port: 1
>> state: PORT_ACTIVE (4)
>> max_mtu: 2048 (4)
>> active_mtu: 2048 (4)
>> sm_lid: 4
>> port_lid: 16
>> port_lmc: 0x00
>> max_msg_sz: 0x40000000
>> port_cap_flags: 0x02510868
>> max_vl_num: 8 (4)
>> bad_pkey_cntr: 0x0
>> qkey_viol_cntr: 0x0
>> sm_sl: 0
>> pkey_tbl_len: 128
>> gid_tbl_len: 128
>> subnet_timeout: 18
>> init_type_reply: 0
>> active_width: 4X (2)
>> active_speed: 5.0 Gbps (2)
>> phys_state: LINK_UP (5)
>> GID[ 0]:
>> fe80:0000:0000:0000:0018:8b90:97fe:73ce
>>
>> Best Regards,
>>
>> Gloria Jan
>> Wavelink Technology Inc.
>
>
> --
> Jeff Squyres
> Cisco Systems
>