Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] Strange binding issue on 40 core nodes and cgroups
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-11-05 17:22:58


Le 05/11/2012 22:57, Brock Palen a écrit :
> Ok more information (had to build newer hwloc) My job today only 2 processes are running at half speed and they indeed are sharing the same core:
>
> [root_at_nyx7000 ~]# for x in `cat /tmp/pids `; do echo -n "$x "; hwloc-bind --get-last-cpu-location --pid $x; done | sort -k 2
> 1164 0x00000001,0x0
> 1158 0x00000010,0x0
> 1165 0x00000010,0x0
> 1167 0x00000020
> 1157 0x00000200
> 1159 0x00000400
> 1160 0x00002000
> 1163 0x00004000
> 1166 0x00020000
> 1161 0x00040000
> 1168 0x00200000
> 1162 0x00400000
>
> 1157 brockp 20 0 1885m 1.8g 456 R 99.6 0.2 9:49.55 stream
> 1159 brockp 20 0 1885m 1.8g 456 R 99.6 0.2 8:10.91 stream
> 1161 brockp 20 0 1885m 1.8g 456 R 99.6 0.2 9:49.55 stream
> 1162 brockp 20 0 1885m 1.8g 456 R 99.6 0.2 9:49.54 stream
> 1163 brockp 20 0 1885m 1.8g 456 R 99.6 0.2 9:49.55 stream
> 1164 brockp 20 0 1885m 1.8g 456 R 99.6 0.2 9:49.53 stream
> 1160 brockp 20 0 1885m 1.8g 456 R 97.7 0.2 9:49.54 stream
> 1166 brockp 20 0 1885m 1.8g 456 R 97.7 0.2 9:49.53 stream
> 1167 brockp 20 0 1885m 1.8g 456 R 97.7 0.2 9:49.46 stream
> 1168 brockp 20 0 1885m 1.8g 456 R 97.7 0.2 8:10.86 stream
> 1158 brockp 20 0 1885m 1.8g 456 R 48.9 0.2 4:54.78 stream
> 1165 brockp 20 0 1885m 1.8g 456 R 48.9 0.2 4:54.76 stream
>
>
> This is very strange. Is there a way to ask hwloc to show me all processes that are using a given cpu?
>

No there's no easy way to do that.
You should first check whether this given cpu is idle or not. Running
top and pressing 1 will show one line per CPU (yours should be the
second CPU line).

top can also sort by the last used CPU. Type f to enter the config menu,
hilight the "last cpu" line, and hit 's' to make it the sort column.
Assuming your top version isn't too different from mine, you should be
able to quickly see if any process used your given cpu recently.

Brice