Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] Compiling OpenMPI 1.5.4 on Debian 6 qemu arm6l
From: Jeffrey Squyres (jsquyres_at_[hidden])
Date: 2012-02-28 11:11:55


Ok. We'll rely on your patches. :-)

On Feb 28, 2012, at 11:00 AM, Leif Lindholm wrote:

> We'd need a few ifdefs, effectively.
>
> One on the dmb/mcr and one on the 64-bit, depending on v6k or higher.
>
> This would provide ARMv6 support only though - ARMv5 or earlier (like debian "armel") will still miss out.
>
>> -----Original Message-----
>> From: devel-bounces_at_[hidden] [mailto:devel-bounces_at_[hidden]] On
>> Behalf Of Jeffrey Squyres
>> Sent: 28 February 2012 14:10
>> To: Leif Lindholm
>> Cc: Open MPI Developers; Ron Broberg
>> Subject: Re: [OMPI devel] Compiling OpenMPI 1.5.4 on Debian 6 qemu
>> arm6l
>>
>> Are there any changes we need to make to OMPI?
>>
>>
>> On Feb 28, 2012, at 7:50 AM, Leif Lindholm wrote:
>>
>>> Hi Ron,
>>>
>>> Excellent work! Indeed - simply dropping the DMBs can lead to memory
>> consistency issues even on ARMv6.
>>>
>>> The architectural semantics for memory barriers exist in ARMv6 though
>> - they just weren't given dedicated mnemonics.
>>> What you could do is to simply replace the inline "dmb" sequences
>> with inline cp15 operations:
>>> - "MCR p15, 0, r0, c7, c10, 5"
>>> (the 'r0' is an encoding artefact and doesn't affect the register
>>> contents)
>>>
>>> LDREXD/STREXD weren't part of the ARMv6 base architecture, although
>> they are supported by the 1176 which is used in the Raspberry PI. If
>> your tools support detecting/building for extension subarchitecture
>> ARMv6k (supported by 1176), you can actually keep the 64-bit atomics
>> in.
>>>
>>> Best Regards,
>>>
>>> Leif
>>>
>>> References:
>>>
>> http://infocenter.arm.com/help/topic/com.arm.doc.ddi0301h/Babfdddg.html
>>>
>> http://infocenter.arm.com/help/topic/com.arm.doc.ddi0301h/Babhejba.html
>>>
>>>> -----Original Message-----
>>>> From: Jeffrey Squyres [mailto:jsquyres_at_[hidden]]
>>>> Sent: 28 February 2012 12:30
>>>> To: Ron Broberg; Open MPI Developers
>>>> Cc: Leif Lindholm
>>>> Subject: Re: [OMPI devel] Compiling OpenMPI 1.5.4 on Debian 6 qemu
>>>> arm6l
>>>>
>>>> Ron -- Many thanks!
>>>>
>>>> Leif -- can you comment on this? (yes, I'm passing the buck to our
>> ARM
>>>> Open MPI representative :-) )
>>>>
>>>>
>>>> On Feb 26, 2012, at 1:22 PM, Ron Broberg wrote:
>>>>
>>>>> I would like to report the following information regarding
>> compiling
>>>> OpenMPI on Debian ARMv6. I won't submit this as a patch because I
>> don't
>>>> believe that "delete all 'dmb' instructions" can be considered a
>> well
>>>> developed patch. But this information may be of use to someone down
>> the
>>>> line.
>>>>>
>>>>> I was able to compile the upstream openmpi-1.5.4 distribution on a
>>>> Debian6 armv6l qemu emulation.
>>>>> http://www.open-mpi.org/software/ompi/v1.4/downloads/openmpi-
>>>> 1.4.5.tar.bz2
>>>>>
>>>>>
>>>>> You have to make 3 changes to the package
>>>>> 1) Delete all references to the RISC instruction 'dmb'
>>>>> 2) Modify the 'configure' file to include an 'armv6' option
>>>>> 3) Compile with CFLAGS=-march=armv6
>>>>>
>>>>> ## 1) make the following edits to these three files
>>>>> ./opal/asm/generated/atomic-local.s
>>>>> delete all dmb instructions
>>>>> ./opal/asm/base/ARM.asm
>>>>> delete all dmb instructions
>>>>> ./opal/include/opal/sys/arm/atomic.h
>>>>> change the lines:
>>>>> #if OPAL_WANT_SMP_LOCKS
>>>>> #define MB() __asm__ __volatile__ ("dmb" : : : "memory")
>>>>> #define RMB() __asm__ __volatile__ ("dmb" : : : "memory")
>>>>> #define WMB() __asm__ __volatile__ ("dmb" : : : "memory")
>>>>> #else
>>>>> #define MB()
>>>>> #define RMB()
>>>>> #define WMB()
>>>>> #endif
>>>>>
>>>>> to read:
>>>>> #define MB()
>>>>> #define RMB()
>>>>> #define WMB()
>>>>>
>>>>> ## 2) add the following to the 'configure' file at line 26946 of
>>>> 171183
>>>>> goto line 26946, there should be an 'alpha-' section above and an
>>>> 'armv7' below
>>>>> insert the following
>>>>> armv6*)
>>>>> ompi_cv_asm_arch="ARM"
>>>>> OPAL_ASM_SUPPORT_64BIT=0
>>>>> OMPI_GCC_INLINE_ASSIGN='"mov %0, #0" : "=&r"(ret)'
>>>>> ;;
>>>>>
>>>>> ## 3) compile and install with the following CFLAGS
>>>>> CFLAGS=-march=armv6
>>>>> ./configure CFLAGS=-march=armv6
>>>>> make
>>>>> sudo make install
>>>>>
>>>>> more information about my build at
>>>>> http://rhinohide.wordpress.com/2012/02/26/openmpi-on-raspberry-pi/
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> devel mailing list
>>>>> devel_at_[hidden]
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>>
>>>>
>>>> --
>>>> Jeff Squyres
>>>> jsquyres_at_[hidden]
>>>> For corporate legal information go to:
>>>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>>>
>>>
>>>
>>> -- IMPORTANT NOTICE: The contents of this email and any attachments
>> are confidential and may also be privileged. If you are not the
>> intended recipient, please notify the sender immediately and do not
>> disclose the contents to any other person, use it for any purpose, or
>> store or copy the information in any medium. Thank you.
>>>
>>
>>
>> --
>> Jeff Squyres
>> jsquyres_at_[hidden]
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>
>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
> -- IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.
>

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/