Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2006-10-14 09:35:23


It also depends on what exactly you mean by "static binaries" (see
below).

To fill in the background for those on the list, we had some prior
off-list e-mail about static builds where I said:

-----
Keep in mind that "--disable-shared --enable-static" on OMPI's
configure line refers to how *Open MPI's libraries are built*; it
does not refer to how Open MPI's libraries or executables are linked
to external libraries (we didn't make this up; it's a GNU standard).
For example, I just did a "--disable-shared --enable-static" build on
my machine. Running ldd on the generated "mpirun", I see that it is
fully static with respect to all OMPI components/libraries, but it
still links to a variety of other shared libraries:

shell$ ldd mpirun
         libnuma.so.1 => /usr/lib64/libnuma.so.1 (0x0000003af3a00000)
         libdl.so.2 => /lib64/libdl.so.2 (0x0000002a95585000)
         libnsl.so.1 => /lib64/libnsl.so.1 (0x0000003af9300000)
         libutil.so.1 => /lib64/libutil.so.1 (0x0000003af9500000)
         libm.so.6 => /lib64/tls/libm.so.6 (0x0000003af3f00000)
         libpthread.so.0 => /lib64/tls/libpthread.so.0
(0x0000003af4700000)
         libc.so.6 => /lib64/tls/libc.so.6 (0x0000003af3c00000)
         /lib64/ld-linux-x86-64.so.2 (0x0000003af3800000)

[...snipped...]

Typically, one would use the "-static" (or your compiler's
equivalent) flag to produce fully static executables. I was unable
to make this work on my development systems, however; it seems that
some Linux system calls *require* shared libraries (I'm not sure
why). For example:

[...failed link line omitted, similar to what Josh showed...]

Let me know if this is a priority for you and I can poke around
further. I'm guessing that your main goal is with regards to
libtorque, and the other system libraries aren't that important.
-----

With that background, a few questions on this mail:

- You mentioned that you used "--enable-static" but you did not use
"--disable-shared". Did you intend to build both the static and
dynamic OMPI libraries?

- What exactly is your intent: a) do you want 100% static executables
(to include the normal Linux system libraries), b) do you just want
to avoid having to set LD_LIBRARY_PATH for some of the esoteric
libraries such as libmpi, libtorque, libibverbs, etc., or c)
something else?

Given the messages from the linker and my prior mail, I'm not sure
that a) is possible -- it will require more poking around to know for
sure. b) is definitely possible simply by using --enable-static and
--disable-shared and then using "mpicc" (and friends) as normal.
OMPI will be built statically (to include torque and libibverbs,
assuming you have .a's versions of both of those), but as Scott
mentioned, system libraries such as dl and sysfs will be linked
dynamically.

On Oct 13, 2006, at 10:22 PM, Scott Weitzenkamp ((sweitzen)) wrote:

> Shouldn't mpicc always link system libraries (like -ldl and -lsysfs)
> dynamically?
>
> Scott Weitzenkamp
> SQA and Release Manager
> Server Virtualization Business Unit
> Cisco Systems
>
>
>> -----Original Message-----
>> From: users-bounces_at_[hidden]
>> [mailto:users-bounces_at_[hidden]] On Behalf Of Josh England
>> Sent: Friday, October 13, 2006 3:53 PM
>> To: Open MPI Users
>> Subject: [OMPI users] problem building static w/ 1.1.2_rc4
>>
>> Hi,
>>
>> I'm trying to statically compile a simple program, but have run into
>> some difficulties so far. Trying with my own link line I get many
>> complaints about symbols being redefined:
>>
>> [root_at_dlogin2 examples]# gcc -static -o cpi cpi.c
>> -I/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/include
>> -L/apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib
>> -L/apps/torque/lib -L/usr/ofed/lib64 -lmpi -lopal -ldl -lpthread -lrt
>> -lorte -lopal -ltorque -lutil -libverbs -lsysfs -lm
>>
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(ltdl.o)(.text+0x2ef): In function `sys_dl_open':
>> : warning: Using 'dlopen' in statically linked applications
>> requires at
>> runtime the shared libraries from the glibc version used for linking
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> liborte.a(sys_info.o)(.text+0xa2): In function `orte_sys_info':
>> : warning: Using 'getpwuid' in statically linked applications
>> requires
>> at runtime the shared libraries from the glibc version used
>> for linking
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(if.o)(.text+0x5d9): In function `opal_ifaddrtoname':
>> : warning: Using 'gethostbyname' in statically linked applications
>> requires at runtime the shared libraries from the glibc
>> version used for
>> linking
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x1880): In function `free':
>> : multiple definition of `free'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x3a8a): first defined here
>> /usr/bin/ld: Warning: size of symbol `free' changed from 231
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 255 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x38d0): In function `malloc':
>> : multiple definition of `malloc'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x2fd5): first defined here
>> /usr/bin/ld: Warning: size of symbol `malloc' changed from 319
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 461 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x3fb0): In function `realloc':
>> : multiple definition of `realloc'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x3b71): first defined here
>> /usr/bin/ld: Warning: size of symbol `realloc' changed from 448
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 783 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x4c0): In function `_int_new_arena':
>> : multiple definition of `_int_new_arena'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x352): first defined here
>> /usr/bin/ld: Warning: size of symbol `_int_new_arena' changed from
>> 242
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 224 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0xff0): In function `_int_free':
>> : multiple definition of `_int_free'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x113a): first defined here
>> /usr/bin/ld: Warning: size of symbol `_int_free' changed from 856
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 1897 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x1980): In function `_int_malloc':
>> : multiple definition of `_int_malloc'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x1b0a): first defined here
>> /usr/bin/ld: Warning: size of symbol `_int_malloc' changed from 3193
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 3548 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x2840): In function `_int_memalign':
>> : multiple definition of `_int_memalign'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x28b9): first defined here
>> /usr/bin/ld: Warning: size of symbol `_int_memalign' changed from 464
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 524 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x2a50): In function `_int_valloc':
>> : multiple definition of `_int_valloc'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x3d71): first defined here
>> /usr/bin/ld: Warning: size of symbol `_int_valloc' changed from 69
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 71 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x2c10): In function `_int_realloc':
>> : multiple definition of `_int_realloc'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x2cd8): first defined here
>> /usr/bin/ld: Warning: size of symbol `_int_realloc' changed from 765
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 1273 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)(.text+0x42c0): In function `__malloc_check_init':
>> : multiple definition of `__malloc_check_init'
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o)(.text+0x631): first defined here
>> /usr/bin/ld: Warning: size of symbol `__malloc_check_init'
>> changed from
>> 141
>> in
>> /apps/x86_64/mpi/openmpi/gcc-3.4.6/openmpi-1.1.2_rc4-ofed/lib/
>> libopal.a(lt1-malloc.o) to 164 in
>> /usr/lib/gcc/x86_64-redhat-linux/3.4.6/../../../../lib64/libc.
>> a(malloc.o)
>> collect2: ld returned 1 exit status
>>
>>
>> When I try mpicc, its seemingly unable to find some symbols,
>> even when I
>> add the right library:
>>
>> [root_at_dlogin2 examples]# mpicc -static -o cpi cpi.c
>> -L/usr/lib64 -lsysfs
>> ...
>> ... Same 'multiple definition' stuff as above, followed by:
>> ...
>> /usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
>> x23b): In
>> function `ibverbs_init':
>> : undefined reference to `sysfs_open_class'
>> /usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
>> x2d0): In
>> function `ibverbs_init':
>> : undefined reference to `sysfs_get_class_devices'
>> /usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
>> x2e4): In
>> function `ibverbs_init':
>> : undefined reference to `dlist_start'
>> /usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
>> x2f1): In
>> function `ibverbs_init':
>> : undefined reference to `_dlist_mark_move'
>> /usr/ofed/lib64/libibverbs.a(src_libibverbs_la-init.o)(.text+0
>> x334): In
>> function `ibverbs_init':
>> : undefined reference to `sysfs_open_class_device'
>> collect2: ld returned 1 exit status
>>
>> [root_at_dlogin2 examples]# nm /usr/lib64/libsysfs.a |grep
>> sysfs_open_class_device
>>
>> 0000000000000d60 T sysfs_open_class_device
>> 0000000000000170 T sysfs_open_class_device_path
>>
>>
>> I configured ompi with '--with-openib=/usr/ofed --with-tm=/apps/
>> torque
>> --enable-static --without-libnuma' .
>>
>> Is there something I missed doing in order to be able to
>> compile static
>> binaries? Any help is greatly appreciated.
>>
>> -JE
>>
>> --
>>
>> -----------------------------------------------
>> Josh England
>> Sandia National Laboratory, Albuquerque, NM
>> Infrastructure Computing Systems
>> email: jjengla_at_[hidden]
>> phone: (505) 284-0963
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems