Are there any "hard coded" limits in the size of a Open MPI pool? How about
the maximum amount of nodes running a single job? Which factors should be
taken in to consideration when building large clusters? For an example what
kind of limits does the OS set, etc.
Most commercial parallel computing environments seem to have a restriction
of 100-256 nodes. Our project requires more. Any solutions? It's not always
possible to "split" the data into x starting points and use a batch system
to launch x jobs on several independent pools and then wait for the results
and combine them.
Help would be appreciated. Thank you!