> cid's are in fact not recycled in the block algorithm. The problem
> is that comm_free is not collective, so you can not make any
> assumptions whether other procs have also released that communicator.
well, that's not quite correct. The MPI standard says the following
about MPI_Comm_free (MPI 2.1, p 201, l 43): "This collective operation
marks the communication object for deallocation.". So MPI_Comm_free is
collective which makes the prescribed problem(s) easier so solve.
However, I admit that there exist an advice to implementors that
anticipates a local implementation. Personally I find this advice
rather strange and (if nobody can give a good reason for it) would
encourage its removal...