Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] Thread binding problem
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-09-06 04:04:04


Le 06/09/2012 09:56, Gabriele Fatigati a écrit :
> Hi Brice, hi Jeff,
>
> >Can you add some printf inside hwloc_linux_set_area_membind() in
> src/topology-linux.c to see if ENOMEM comes from the mbind >syscall or
> not?
>
> I added printf inside that function, but ENOMEM does not come from there.

Not from hwloc_linux_set_area_membind() at all? Or not from mbind?

> >Have you run your application through valgrind or another
> memory-checking debugger?
>
> I tried with valgrind :
>
> valgrind --track-origins=yes --log-file=output_valgrind
> --leak-check=full --tool=memcheck --show-reachable=yes
> ./main_hybrid_bind_mem
>
> ==25687== Warning: set address range perms: large range [0x39454040,
> 0x2218d4040) (undefined)
> ==25687==
> ==25687== Valgrind's memory management: out of memory:
> ==25687== newSuperblock's request for 4194304 bytes failed.
> ==25687== 34253180928 bytes have already been allocated.
> ==25687== Valgrind cannot continue. Sorry.

There's really somebody allocating way too much memory here.

You should reduce your array size so that it doesn't fail, and then run
valgrind again to check if somebody is allocated a lot of memory without
ever freeing it.

Brice

>
>
> I attach the full output.
>
>
> The code dies also using OpenMP pure code. Very misteriously.
>
>
> 2012/9/5 Jeff Squyres <jsquyres_at_[hidden] <mailto:jsquyres_at_[hidden]>>
>
> On Sep 5, 2012, at 2:36 PM, Gabriele Fatigati wrote:
>
> > I don't think is a simply out of memory since NUMA node has 48
> GB, and I'm allocating just 8 GB.
>
> Mmm. Probably right.
>
> Have you run your application through valgrind or another
> memory-checking debugger?
>
> I've seen cases of heap corruption lead to malloc incorrectly
> failing with ENOMEM.
>
> --
> Jeff Squyres
> jsquyres_at_[hidden] <mailto:jsquyres_at_[hidden]>
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> _______________________________________________
> hwloc-users mailing list
> hwloc-users_at_[hidden] <mailto:hwloc-users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users
>
>
>
>
> --
> Ing. Gabriele Fatigati
>
> HPC specialist
>
> SuperComputing Applications and Innovation Department
>
> Via Magnanelli 6/3, Casalecchio di Reno (BO) Italy
>
> www.cineca.it <http://www.cineca.it> Tel: +39 051
> 6171722
>
> g.fatigati [AT] cineca.it <http://cineca.it>
>
>
> _______________________________________________
> hwloc-users mailing list
> hwloc-users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-users