Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] Bug report: using static build
From: Ralph Castain (rhc_at_[hidden])
Date: 2014-07-12 19:07:01


<snipping the 1.6 stuff as we are no longer supporting the 1.6 series>

Hmmm... --enable-static works just fine (at least, for the 1.8.2rc) on CentOS, so this may be a Debian thing. We have some protection in there for getpwuid on the Cray, where static builds take similar exception to it, but that requires you configure with --disable-getpwuid. I was unaware of problems regarding the other functions - is that something also specific to Debian? Do you have some preferred workaround to suggest?

The problem regarding libnuma et al is caused by linking against the external versions of hwloc, libevent, and libltdtl. I'm not sure how we resolve that one as we don't control what those libs were linked against - it may require that you simply include them in an LDFLAGS argument. I can poke at it a bit, but would likely have to defer to 1.8.3 for a solution unless something simple appears.

On Jul 12, 2014, at 12:21 PM, Andrey Gursky <andrey.gursky_at_[hidden]> wrote:

> ====================> 1.8.1 <==================
>
> $ ./configure --prefix=/home/gursky/progs/openmpi-static/usr
> --disable-shared --enable-static --disable-silent-rules
> --with-hwloc=/usr --with-libevent=/usr --with-libltdl=external
> --without-tm --with-devel-headers --enable-heterogeneous --disable-vt
>
> ============================
>
> - mpicc -static is missing even more libraries:
>
> $ MP=/home/gursky/progs/openmpi-static/usr PATH=$MP/bin:$PATH
> LD_LIBRARY_PATH=$MP/lib mpicc -Wall -g -static -o simple_spawn
> simple_spawn.c
> /home/gursky/progs/openmpi-static/usr/lib/libopen-rte.a(session_dir.o):
> In function `orte_session_dir_get_name':
> session_dir.c:(.text+0x1b1): warning: Using 'getpwuid' in statically
> linked applications requires at runtime the shared libraries from the
> glibc version used for linking
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(if.o): In
> function `opal_ifaddrtoname':
> if.c:(.text+0x21a): warning: Using 'getaddrinfo' in statically linked
> applications requires at runtime the shared libraries from the glibc
> version used for linking
> /home/gursky/progs/openmpi-static/usr/lib/libopen-rte.a(ras_slurm_module.o):
> In function `init':
> ras_slurm_module.c:(.text+0x5a8): warning: Using 'gethostbyname' in
> statically linked applications requires at runtime the shared
> libraries from the glibc version used for linking
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libevent.a(evutil.o):
> In function `evutil_getaddrinfo_common':
> (.text+0x14cf): warning: Using 'getprotobynumber' in statically linked
> applications requires at runtime the shared libraries from the glibc
> version used for linking
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(mca_base_component_find.o):
> In function `open_component':
> mca_base_component_find.c:(.text+0x2aa): undefined reference to
> `lt_dlopenadvise'
> mca_base_component_find.c:(.text+0x36c): undefined reference to `lt_dlsym'
> mca_base_component_find.c:(.text+0x5e4): undefined reference to `lt_dlclose'
> mca_base_component_find.c:(.text+0x63c): undefined reference to `lt_dlclose'
> mca_base_component_find.c:(.text+0x6af): undefined reference to `lt_dlclose'
> mca_base_component_find.c:(.text+0x6d6): undefined reference to `lt_dlerror'
> mca_base_component_find.c:(.text+0x744): undefined reference to `lt_dlerror'
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(mca_base_component_find.o):
> In function `mca_base_component_find':
> mca_base_component_find.c:(.text+0x1196): undefined reference to
> `lt_dlforeachfile'
> mca_base_component_find.c:(.text+0x11e2): undefined reference to
> `lt_dlforeachfile'
> mca_base_component_find.c:(.text+0x1603): undefined reference to
> `lt_dlforeachfile'
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(mca_base_component_repository.o):
> In function `ri_destructor':
> mca_base_component_repository.c:(.text+0xd3): undefined reference to
> `lt_dlclose'
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(mca_base_component_repository.o):
> In function `mca_base_component_repository_init':
> mca_base_component_repository.c:(.text+0x291): undefined reference to
> `lt_dlinit'
> mca_base_component_repository.c:(.text+0x2a3): undefined reference to
> `lt_dladvise_init'
> mca_base_component_repository.c:(.text+0x2b5): undefined reference to
> `lt_dladvise_ext'
> mca_base_component_repository.c:(.text+0x2c7): undefined reference to
> `lt_dladvise_global'
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(mca_base_component_repository.o):
> In function `mca_base_component_repository_finalize':
> mca_base_component_repository.c:(.text+0x7d2): undefined reference to
> `lt_dladvise_destroy'
> mca_base_component_repository.c:(.text+0x7e6): undefined reference to
> `lt_dlexit'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib/libibverbs.a(src_libibverbs_la-init.o):
> In function `load_driver':
> (.text+0x387): undefined reference to `dlopen'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib/libibverbs.a(src_libibverbs_la-init.o):
> In function `load_driver':
> (.text+0x3c1): undefined reference to `dlerror'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib/libibverbs.a(src_libibverbs_la-init.o):
> In function `ibverbs_init':
> (.text+0xc2f): undefined reference to `dlopen'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib/libibverbs.a(src_libibverbs_la-init.o):
> In function `ibverbs_init':
> (.text+0xc40): undefined reference to `dlclose'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc__dlforeach_cb':
> (.text+0x77): undefined reference to `lt_dlopenext'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc__dlforeach_cb':
> (.text+0x91): undefined reference to `lt_dlerror'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc__dlforeach_cb':
> (.text+0x107): undefined reference to `lt_dlsym'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc__dlforeach_cb':
> (.text+0x3d5): undefined reference to `lt_dlclose'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc_plugins_exit':
> (.text+0x446): undefined reference to `lt_dlclose'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc_plugins_exit':
> (.text+0x490): undefined reference to `lt_dlexit'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc_plugins_init':
> (.text+0x4da): undefined reference to `lt_dlinit'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(components.o):
> In function `hwloc_plugins_init':
> (.text+0x556): undefined reference to `lt_dlforeachfile'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_set_area_membind':
> (.text+0x1ce2): undefined reference to `mbind'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_set_area_membind':
> (.text+0x1d52): undefined reference to `mbind'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_set_thisthread_membind':
> (.text+0x1e72): undefined reference to `set_mempolicy'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_set_thisthread_membind':
> (.text+0x1f08): undefined reference to `migrate_pages'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_set_thisthread_membind':
> (.text+0x1f4b): undefined reference to `set_mempolicy'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_find_kernel_max_numnodes':
> (.text+0x1fef): undefined reference to `get_mempolicy'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_get_thisthread_membind':
> (.text+0x210b): undefined reference to `get_mempolicy'
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libhwloc.a(topology-linux.o):
> In function `hwloc_linux_get_area_membind':
> (.text+0x2282): undefined reference to `get_mempolicy'
> collect2: error: ld returned 1 exit status
> gursky_at_hpprobook:~/projects/openmpi/test$
> $
>
> ===============================
>
> $ MP=/home/gursky/progs/openmpi-static/usr PATH=$MP/bin:$PATH
> LD_LIBRARY_PATH=$MP/lib mpicc -show
> gcc -I/home/gursky/progs/openmpi-static/usr/include
> -I/home/gursky/progs/openmpi-static/usr/include/openmpi -pthread
> -Wl,-rpath -Wl,/home/gursky/progs/openmpi-static/usr/lib
> -Wl,--enable-new-dtags -L/home/gursky/progs/openmpi-static/usr/lib
> -lmpi -lopen-rte -lopen-pal -levent -levent_pthreads -ldl -libverbs
> -lrt -lnsl -lutil -lm -lhwloc
> $
>
> ===============================
>
> - Using output of mpicc still gives another result:
>
> $ MP=/home/gursky/progs/openmpi-static/usr PATH=$MP/bin:$PATH
> LD_LIBRARY_PATH=$MP/lib gcc
> -I/home/gursky/progs/openmpi-static/usr/include
> -I/home/gursky/progs/openmpi-static/usr/include/openmpi -pthread
> -Wl,-rpath -Wl,/home/gursky/progs/openmpi-static/usr/lib
> -Wl,--enable-new-dtags -L/home/gursky/progs/openmpi-static/usr/lib
> -lmpi -lopen-rte -lopen-pal -levent -levent_pthreads -ldl -libverbs
> -lrt -lnsl -lutil -lm -lhwloc -static -o simple_spawn simple_spawn.o
> simple_spawn.o: In function `main':
> simple_spawn.c:(.text+0x42): undefined reference to `MPI_Init'
> simple_spawn.c:(.text+0x4e): undefined reference to `ompi_mpi_comm_world'
> simple_spawn.c:(.text+0x53): undefined reference to `MPI_Comm_rank'
> simple_spawn.c:(.text+0x73): undefined reference to `ompi_mpi_comm_world'
> simple_spawn.c:(.text+0x78): undefined reference to `MPI_Comm_size'
> simple_spawn.c:(.text+0x84): undefined reference to `MPI_Comm_get_parent'
> simple_spawn.c:(.text+0x8e): undefined reference to `ompi_mpi_comm_null'
> simple_spawn.c:(.text+0xca): undefined reference to `ompi_mpi_comm_world'
> simple_spawn.c:(.text+0xd5): undefined reference to `ompi_mpi_info_null'
> simple_spawn.c:(.text+0xe7): undefined reference to `MPI_Comm_spawn'
> simple_spawn.c:(.text+0x143): undefined reference to `ompi_mpi_int'
> simple_spawn.c:(.text+0x150): undefined reference to `MPI_Send'
> simple_spawn.c:(.text+0x15c): undefined reference to `MPI_Comm_disconnect'
> simple_spawn.c:(.text+0x177): undefined reference to `ompi_mpi_comm_world'
> simple_spawn.c:(.text+0x17c): undefined reference to `MPI_Comm_rank'
> simple_spawn.c:(.text+0x188): undefined reference to `ompi_mpi_comm_world'
> simple_spawn.c:(.text+0x18d): undefined reference to `MPI_Comm_size'
> simple_spawn.c:(.text+0x1fd): undefined reference to `ompi_mpi_int'
> simple_spawn.c:(.text+0x20a): undefined reference to `MPI_Recv'
> simple_spawn.c:(.text+0x231): undefined reference to `MPI_Comm_disconnect'
> simple_spawn.c:(.text+0x24a): undefined reference to `MPI_Finalize'
> collect2: error: ld returned 1 exit status
> $
>
> ===============================
>
> $ gcc -static -DNDEBUG -pthread -Wall -g -o simple_spawn
> simple_spawn.o /home/gursky/progs/openmpi-static/usr/lib/libmpi.a
> -libverbs /home/gursky/progs/openmpi-static/usr/lib/libopen-rte.a
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a -lrt -lnsl
> -lutil -lm -lhwloc -lnuma -levent -levent_pthreads -lltdl -ldl
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../../lib/libibverbs.a(src_libibverbs_la-init.o):
> In function `load_driver':
> (.text+0x387): warning: Using 'dlopen' in statically linked
> applications requires at runtime the shared libraries from the glibc
> version used for linking
> /home/gursky/progs/openmpi-static/usr/lib/libopen-rte.a(session_dir.o):
> In function `orte_session_dir_get_name':
> session_dir.c:(.text+0x1b1): warning: Using 'getpwuid' in statically
> linked applications requires at runtime the shared libraries from the
> glibc version used for linking
> /home/gursky/progs/openmpi-static/usr/lib/libopen-pal.a(if.o): In
> function `opal_ifaddrtoname':
> if.c:(.text+0x21a): warning: Using 'getaddrinfo' in statically linked
> applications requires at runtime the shared libraries from the glibc
> version used for linking
> /home/gursky/progs/openmpi-static/usr/lib/libopen-rte.a(ras_slurm_module.o):
> In function `init':
> ras_slurm_module.c:(.text+0x5a8): warning: Using 'gethostbyname' in
> statically linked applications requires at runtime the shared
> libraries from the glibc version used for linking
> /usr/lib/gcc/x86_64-linux-gnu/4.9/../../../x86_64-linux-gnu/libevent.a(evutil.o):
> In function `evutil_getaddrinfo_common':
> (.text+0x14cf): warning: Using 'getprotobynumber' in statically linked
> applications requires at runtime the shared libraries from the glibc
> version used for linking
> $
>
> ===============================
>
> $ ldd ./simple_spawn
> not a dynamic executable
> $ ./simple_spawn
> [pid 2805] starting up!
> --------------------------------------------------------------------------
> Open MPI was unable to obtain the username in order to create a path
> for its required temporary directories. This type of error is usually
> caused by a transient failure of network-based authentication services
> (e.g., LDAP or NIS failure due to network congestion), but can also be
> an indication of system misconfiguration.
>
> Please consult your system administrator about these issues and try
> again.
> --------------------------------------------------------------------------
> 0 completed MPI_Init
> Parent [pid 2805] about to spawn!
> [hpprobook:02805] [[52016,1],0] FORKING HNP: orted --hnp --set-sid
> --report-uri 16 --singleton-died-pipe 17 -mca state_novm_select 1 -mca
> ess_base_jobid 3408920576
> [pid 2809] starting up!
> [pid 2810] starting up!
> [pid 2811] starting up!
> [hpprobook:02809] [[52016,2],0] ORTE_ERROR_LOG: Data for specified key
> not found in file runtime/orte_globals.c at line 504
> -------------------------------------------------------
> Child job 2 terminated normally, but 1 process returned
> a non-zero exit code.. Per user-direction, the job has been aborted.
> -------------------------------------------------------
> [hpprobook:02805] 1 more process has sent help message
> help-orte-runtime.txt / orte:session:dir:nopwname
> [hpprobook:02805] Set MCA parameter "orte_base_help_aggregate" to 0 to
> see all help / error messages
>
>
>
> ^C
> $ --------------------------------------------------------------------------
> (null) detected that one or more processes exited with non-zero
> status, thus causing
> the job to be terminated. The first process to do so was:
>
> Process name: [[52016,2],0]
> Exit code: 64
> --------------------------------------------------------------------------
>
> - also doesn't work
>
> =======================================================
>
>
> Thanks,
> Andrey
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/devel
> Link to this post: http://www.open-mpi.org/community/lists/devel/2014/07/15131.php