I have OpenMPI running fine for a small/medium number of tasks (simple hello or cpi program). But when I try 700 or 800 tasks, it hangs, apparently on startup. I think this might be related to LDAP, since if I try to log into my account while the job is hung, I get told my username doesn’t exist. However, I tried adding –debug to the mpirun, and got the same sequence of output as for successful smaller runs, until it hung again. So I added –-debug-daemons and got this (with an exit, i.e. no hanging):

…

[blade1:31733] [0,0,0] wrote setup file

--------------------------------------------------------------------------

The rsh launcher has been given a number of 128 concurrent daemons to

launch and is in a debug-daemons option. However, the total number of

daemons to launch (200) is greater than this value. This is a scenario that

will cause the system to deadlock.

To avoid deadlock, either increase the number of concurrent daemons, or

remove the debug-daemons flag.

--------------------------------------------------------------------------

[blade1:31733] [0,0,0] ORTE_ERROR_LOG: Fatal in file ../../../../../orte/mca/rmgr/urm/

rmgr_urm.c at line 455

[blade1:31733] mpirun: spawn failed with errno=-6

[blade1:31733] sess_dir_finalize: proc session dir not empty - leaving

Any ideas or suggestions appreciated.

Todd Heywood