From: Josh Hursey <firstname.lastname@example.org>
To: Open MPI Users <email@example.com>
Cc: Mudassar Majeed <firstname.lastname@example.org>
Sent: Thursday, November 10, 2011 5:11 PM
Subject: Re: [OMPI users] Process Migration
Note that the "migrate me from my current node to node <foo>" scenario
is covered by the migration API exported by the C/R infrastructure, as
I noted earlier.
The "move rank N to node <foo>" scenario could probably be added as an
extension of this interface (since you can do that via the command
line now) if that is what you are looking for.
On Thu, Nov 10, 2011 at 11:03 AM, Ralph Castain <email@example.com
> So what you are looking for is an MPI extension API that let's you say
> "migrate me from my current node to node <foo>"? Or do you have a rank that
> is the "master" that would order "move rank N to node <foo>"?
> Either could be provided, I imagine - just want to ensure I understand what
> you need.
Can you pass along a brief description of the syntax and
> functionality you would need?
> On Nov 10, 2011, at 8:27 AM, Mudassar Majeed wrote:
> Thank you for your reply. In our previous publication, we have figured it
> out that run more than one processes on cores and balancing the
> computational load considerably reduces the total execution time. You know
> the MPI_Graph_create function, we created another function MPI_Load_create
> that maps the processes on cores such that balance of computational load can
> be achieved on cores. We were having some issues with increase in
> communication cost due to ranks rearrangements (due to MPI_Comm_split, with
> color=0), so in this research work we will see how can we balance both
> computation load on each core and communication load on each node. Those
> processes that communicate more will reside on the same node
> computational load balance over the cores. I solved this problem using ILP
> but ILP takes time and can't be used in run time so I am thinking about an
> heuristic. That's why I want to see if it is possible to migrate a process
> from one core to another or not. Then I will see how good my heuristic will
> From: Jeff Squyres <firstname.lastname@example.org
> To: Mudassar Majeed <email@example.com
>; Open MPI Users
> Cc: Ralph Castain <firstname.lastname@example.org
> Sent: Thursday, November 10, 2011 2:19 PM
> Subject: Re: [OMPI users] Process Migration
> On Nov 10, 2011, at 8:11 AM, Mudassar Majeed wrote:
>> Thank you for your reply. I am implementing a load balancing function for
>> MPI, that will balance the computation load and the communication both at a
>> time. So my algorithm assumes that all the cores may at the end get
>> different number of processes to run.
> Are you talking about over-subscribing cores? I.e., putting more than 1 MPI
> process on each core?
> In general, that's not a good idea.
>> In the beginning (before that function will be called), each core will
>> have equal number of processes. So I am thinking either to start more
>> processes on each core (than needed) and run my function for
>> and then block the remaining processes (on each core). In this way I will be
>> able to achieve different number of processes per core.
> Open MPI spins aggressively looking for network progress. For example, if
> you block in an MPI_RECV waiting for a message, Open MPI is actively banging
> on the CPU looking for network progress. Because of this (and other
> reasons), you probably do not want to over-subscribe your processors
> (meaning: you probably don't want to put more than 1 process per core).
> Jeff Squyres
> For corporate legal information go to:
> users mailing
Postdoctoral Research Associate
Oak Ridge National Laboratory