Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] openmpi MPI_Init doesn't work after fork under gdb
From: George Bosilca (bosilca_at_[hidden])
Date: 2013-02-22 14:18:04


Geoffrey,

If I understand correctly your problem, I think it is a gdb issue. You should inform gdb that after the fork call instead of following the parent you expect to follow the child.

set follow-fork-mode child (should do the trick).

For more info: http://sourceware.org/gdb/onlinedocs/gdb/Forks.html

  George.

On Feb 22, 2013, at 19:55 , Geoffrey Irving <irving_at_[hidden]> wrote:

> No luck with gdb 7.5.1. I can get it to run, but it hits an internal
> error and bails out. Thus, I'm stuck with Apple's 6.3.50. I suppose
> I'll try to restructure the app so that I don't have to spawn the mpi
> process with a fork from the other one.
>
> Geoffrey
>
> On Thu, Feb 21, 2013 at 4:38 PM, Geoffrey Irving <irving_at_[hidden]> wrote:
>> No, I don't really have any idea what it's doing. You have to add
>> manual sleeps and attach another instance of gdb if you want to step
>> through the child, since I also can't get follow-fork-mode child to
>> work. I only tried that once, and it segfaulted accessing the zero
>> pointer destroying the stack in the process. I haven't done manual
>> stepping or printf searches, so I don't know where in MPI_Init it
>> breaks.
>>
>> I'm trying to install gdb 7.5.1 now (a nontrivial process on Mac,
>> unfortunately) to see if that fixes it.
>>
>> Geoffrey
>>
>> On Thu, Feb 21, 2013 at 4:28 PM, Ralph Castain <rhc_at_[hidden]> wrote:
>>> Hmmm...how about that? Yeah, it doesn't work with the devel trunk either - I'd missed that point.
>>>
>>> No idea why, I'm afraid - never tried it before. Are you sure it "crashes"? I'm still getting a child status of "0", but no message output. My guess is that the I/O is being lost for some reason.
>>>
>>> On Feb 21, 2013, at 4:09 PM, Geoffrey Irving <irving_at_[hidden]> wrote:
>>>
>>>> Actually, I don't see it printing "We're an MPI program!" under gdb,
>>>> which means it isn't working.
>>>>
>>>> Geoffrey
>>>>
>>>> On Thu, Feb 21, 2013 at 4:07 PM, Ralph Castain <rhc_at_[hidden]> wrote:
>>>>> Hmmm...works with 1.6.4 for me on Mac 10.8.2:
>>>>>
>>>>> Ralphs-iMac:v1.6 rhc$ ./fork-bug
>>>>> We're an MPI program!
>>>>> child status = 0
>>>>> Ralphs-iMac:v1.6 rhc$ gdb ./fork-bug
>>>>> GNU gdb 6.3.50-20050815 (Apple version gdb-1820) (Sat Jun 16 02:40:11 UTC 2012)
>>>>> Copyright 2004 Free Software Foundation, Inc.
>>>>> GDB is free software, covered by the GNU General Public License, and you are
>>>>> welcome to change it and/or distribute copies of it under certain conditions.
>>>>> Type "show copying" to see the conditions.
>>>>> There is absolutely no warranty for GDB. Type "show warranty" for details.
>>>>> This GDB was configured as "x86_64-apple-darwin"...Reading symbols for shared libraries ... done
>>>>>
>>>>> (gdb) r
>>>>> Starting program: /Volumes/RHCHD/rhc/openmpi/v1.6/fork-bug
>>>>> Reading symbols for shared libraries ++............................. done
>>>>> child status = 0
>>>>>
>>>>> Program exited normally.
>>>>> (gdb)
>>>>>
>>>>> Afraid I have no idea why you might be hitting the problem, though...
>>>>>
>>>>>
>>>>> On Feb 21, 2013, at 3:50 PM, Geoffrey Irving <irving_at_[hidden]> wrote:
>>>>>
>>>>>> The singleton fork/exec itself is fine, since normal MPI programs work
>>>>>> under gdb (e.g., fork-bug.c without the fork). gdb is has
>>>>>> follow-fork-mode set to parent, so it's odd that gdb is looking at the
>>>>>> child process's trickery at all.
>>>>>>
>>>>>> I've confirmed that it's still broken under 1.6.4, unfortunately.
>>>>>>
>>>>>> Geoffrey
>>>>>>
>>>>>> On Thu, Feb 21, 2013 at 3:36 PM, Ralph Castain <rhc_at_[hidden]> wrote:
>>>>>>> Singletons fork/exec a daemon to support them - my guess is that gdb may not like it on your machine?
>>>>>>>
>>>>>>> FWIW - it runs fine for me using the developer's trunk. You might try with 1.6.4 in case it's a bug in 1.6.0
>>>>>>>
>>>>>>>
>>>>>>> On Feb 21, 2013, at 3:18 PM, Geoffrey Irving <irving_at_[hidden]> wrote:
>>>>>>>
>>>>>>>> The attached program illustrates the problem. It forks, and the child
>>>>>>>> calls MPI_Init. This works fine unless I'm inside gdb. Inside gdb,
>>>>>>>> MPI_Init silently crashes.
>>>>>>>>
>>>>>>>> I'm using OpenMPI 1.6.0 on Mac 10.8.2. I'm running the program
>>>>>>>> directly, not through mpirun.
>>>>>>>>
>>>>>>>> Any ideas what might be wrong?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Geoffrey
>>>>>>>>
>>>>>>>> cone:scratch% /usr/local/bin/mpicc -o fork-bug fork-bug.c
>>>>>>>> cone:scratch% ./fork-bug
>>>>>>>> We're an MPI program!
>>>>>>>> child status = 0
>>>>>>>> cone:scratch% gdb ./fork-bug
>>>>>>>> gdb ./fork-bug
>>>>>>>> GNU gdb 6.3.50-20050815 (Apple version gdb-1824) (Thu Nov 15 10:42:43 UTC 2012)
>>>>>>>> Copyright 2004 Free Software Foundation, Inc.
>>>>>>>> GDB is free software, covered by the GNU General Public License, and you are
>>>>>>>> welcome to change it and/or distribute copies of it under certain conditions.
>>>>>>>> Type "show copying" to see the conditions.
>>>>>>>> There is absolutely no warranty for GDB. Type "show warranty" for details.
>>>>>>>> This GDB was configured as "x86_64-apple-darwin"...Reading symbols for
>>>>>>>> shared libraries ... done
>>>>>>>>
>>>>>>>> (gdb) run
>>>>>>>> Reading symbols for shared libraries ++............................. done
>>>>>>>> child status = 5
>>>>>>>>
>>>>>>>> Program exited normally.
>>>>>>>> (gdb) cone:scratch%
>>>>>>>> <fork-bug.c>_______________________________________________
>>>>>>>> users mailing list
>>>>>>>> users_at_[hidden]
>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> users mailing list
>>>>>>> users_at_[hidden]
>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> users_at_[hidden]
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> users_at_[hidden]
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users