Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] OpenMPI PMI2 with SLURM 14.03 not working
From: Anthony Alba (ascanio.alba7_at_[hidden])
Date: 2014-04-11 15:04:58


Not sure if this is a SLURM or OMPI issue so please bear with the
cross-posting...

The OpenMPI FAQ mentions an issue with slurm 2.6.3/pmi2.
https://www.open-mpi.org/faq/?category=slurm#slurm-2.6.3-issue

I have built both 1.7.5/1.8 against slurm 14.03/pmi2.

When I launch openmpi/examples/hello_c on a single node allocation:

srun --mpi=pmi2 -N 1 hello_c:

srun -N 1 --mpi=pmi2 hello_c
srun: error: _server_read: fd 18 got error or unexpected eof reading header
srun: error: step_launch_notify_io_failure: aborting, io error with
slurmstepd on node 0
srun: Job step aborted: Waiting up to 2 seconds for job step to finish.
srun: error: Timed out waiting for job step to complete

with --slurmd-debug=9: (I'm not sure what is the meaning of "ip
111.110.61.48 sd 14"
below, is that ip as in ip address? It is not the ip address of any Nodes
in my partition)

slurmstepd: mpi/pmi2: client_resp_send: 26 cmd=kvs-put-response;rc=0;
slurmstepd: mpi/pmi2: _tree_listen_readable
slurmstepd: mpi/pmi2: _task_readable
slurmstepd: mpi/pmi2: got client request: 14 cmd=kvs-fence;
slurmstepd: mpi/pmi2: _tree_listen_readable
slurmstepd: mpi/pmi2: _task_readable
slurmstepd: mpi/pmi2: _tree_listen_read
slurmstepd: _tree_listen_read: accepted tree connection: ip 111.110.61.48
sd 14
slurmstepd: _handle_accept_rank: going to read() client rank
slurmstepd: _handle_accept_rank: got client rank 1478164480 on fd 14
srun: error: _server_read: fd 18 got error or unexpected eof reading header
srun: error: step_launch_notify_io_failure: aborting, io error with
slurmstepd on node 0
srun: Job step aborted: Waiting up to 2 seconds for job step to finish.
srun: error: Timed out waiting for job step to complete

Launching with salloc/sbatch works.

- Anthony