Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] 32-bit openib btl fails on 64-bit OS
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-04-11 23:39:15


On Apr 6, 2008, at 5:58 PM, Teng Lin wrote:
> In order to run a 32-bit program on a 64-bit cluster, one has to build
> 32-bit OpenMPI. Following some instructions on this mailing list, I
> successfully built OpenMPI 1.2.4 on 64-bit OS.

It's not clear from your text -- did you build Open MPI as 32 or 64 bit?

> However, I run into
> openib problem when I try to run hello_c program. I also built 64-bit
> OpenMPI from same source. The interesting fact is 64-bit OpenMPI works
> just fine. Below is the output from orterun,
>
> ############################################################################
> iceland:/home/tlin/test_pbs>/home/tin/openmpi-1.2.4/bin/orterun -np 2
> --hostfile mach.lst /home/tlin/test_pbs/hello_c.32
> --------------------------------------------------------------------------
> The OpenIB BTL failed to initialize while trying to create an internal
> queue. This typically indicates a failed OpenFabrics installation or
> faulty hardware. The failure occured here:
>
> Host: cl1n004
> OMPI source: btl_openib.c:828
> Function: ibv_create_cq()
> Error: Invalid argument (errno=22)
> Device: mthca0
>
> You may need to consult with your system administrator to get this
> problem fixed.
> --------------------------------------------------------------------------
[snip]
> ######################################################################
> I saw this error on before on other cluster. Following the instruction
> on (http://www.open-mpi.org/faq/?category=openfabrics#ib-locked-pages)
> does fix the problem. However, I doubt that is the reason why 32 bit
> OpenMPI does not work on this cluster. Output from limit looks fine to
> me. And if that is the case, 64 bit OpenMPI will not work. Any ideas?

I'm unclear exactly what you're saying is happening -- can you clarify?

- 32 bit Open MPI does not build at all
- 64 bit Open MPI builds but yields the error message shown above

Is that right?

Can you send all the information listed here:

  http://www.open-mpi.org/community/help/

-- 
Jeff Squyres
Cisco Systems