Hendryk Bockelmann, le Thu 10 Feb 2011 09:08:11 +0100, a écrit :
> On our clusters the job scheduler binds the MPI tasks, but it is not
> always clear to which resources. So for us it would be great to know
> where a task runs such that we might adopt the MPI communicators to
> increase performance.
Ok, so get_cpubind should be enough to know what binding the job
> Maybe just a note on the hwloc output on the cluster: while on my locale
> machine all MPI tasks are able to explore the whole topology, on the
> cluster each task only sees itself, e.g. for task 7:
> cpuset: 0x0000c000
> 7: NUMANode#0, cpuset: 0x0000c000
> 7: L2Cache#0(0KB line=0), cpuset: 0x0000c000
> 7: Core#0, cpuset: 0x0000c000
> 7: PU, cpuset: 0x00004000
> 7: PU#0, cpuset: 0x00008000
> 7:--> root_cpuset of process 7 is 0x0000c000
Yes, because by default hwloc restricts itself to what you are allowed
to use anyway. To see more, use --whole-system.
> Nevertheless, all MPI-tasks have different cpusets and since the nodes
> are homogeneous one can guess the whole binding using the information of
> lstopo and the HostName of each task. Perhaps you can tell me whether
> such a restricted topology is due to hwloc or due to the fixed binding
> by the job scheduler?
It's because by default hwloc follows the fixed binding :)