Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Troy Benjegerdes (hozer_at_[hidden])
Date: 2005-08-24 21:21:37


I have some Netpipe graphs of OpenMPI and Mviapich on OpenIB gen2 on
Opteron systems, one with PCI-X IB cards, and the other with PCI-Express
DDR IB cards.

I'd like to chat with someone who fill me in a bit on what's going on
with performance, and how the BTL for IB is implemented. One thing I'd
like to try is modifying the kernel to allow 95% of physical memory to
be registered for IB, and see how much of a difference this makes.

I am also toying with the idea of writing an OpenIB BTL module for
Netpipe, so I can evaluate the BTL layer performance directly without
any MPI overhead.

I also get the following messages on startup:

libibverbs: Warning: no userspace device-specific driver found for
uverbs0 driver search path: /usr/lib/infiniband
        [0,1,1][btl_openib_component.c:267:mca_btl_openib_component_init]
        No hca's found on this host!
        
Something strange is going on, because it is finding at least 1 IB
port, given the performance numbers.

-- 
--------------------------------------------------------------------------
Troy Benjegerdes                'da hozer'                hozer_at_[hidden]  
Somone asked me why I work on this free (http://www.fsf.org/philosophy/)
software stuff and not get a real job. Charles Shultz had the best answer:
"Why do musicians compose symphonies and poets write poems? They do it
because life wouldn't have any meaning for them if they didn't. That's why
I draw cartoons. It's my life." -- Charles Shultz