Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] PBSPro/OpenMPI Errors
From: Robert Jackson (rjackson_at_[hidden])
Date: 2009-06-25 12:06:58


When using OpenMPI and nwchem standalone (mpirun --byslot --mca btl
self,sm,tcp --mca btl_base_verbose 30 --mca btl_tcp_if_exclude lo,eth1
$NWCHEM h2o.nw > & h2o.nwo.$$) the job runs fine.

 

When running the same job via the PBSPro scheduler I get errors. The PBS
script is called nwrun and is run with the following command - qsub -V
-S /bin/bash ./nwrun.

 

Nwrun listing:

#!/bin/tcsh

#PBS -N h2o

#PBS -l select=4:ncpus=4:mpiprocs=4

#PBS -l walltime=0:10:00

#PBS -e .

#PBS -j eo

#PBS -k eo

#

# set working directory

set echo

cd $PBS_O_WORKDIR

#

# make sure that the proper mpirun is installed

##module load hpc/openmpi-1.2.6-intel

#

# load NWChem

#module load hpc/nwchem-5.1

setenv LD_LIBRARY_PATH
/share/apps/openmpi-1.2.6-intel/lib:/share/apps/intel/mkl/10.0.1.014/lib
/em64t:/s

hare/apps/intel/cce/10.1.015/lib:/share/apps/intel/fce/10.1.015/lib

setenv NWCHEM /share/apps/nwchem-5.1/bin/nwchem

setenv PERMANENT_DIR $PBS_O_WORKDIR

setenv SCRATCH_DIR $TMPDIR

#

setenv | grep LD_LIB

which mpirun

cat $PBS_NODEFILE

# run a parallel job

mpirun --byslot --mca btl self,sm,tcp --mca btl_tcp_if_exclude lo,eth1
$NWCHEM h2o.nw >& h2o.nwo.$$

exit

 

Error listing from error file:

ARMCI configured for 4 cluster nodes. Network protocol is 'TCP/IP
Sockets'.

1:trying connect to host=compute-1-4.local, port=35506 t=5 111

1:armci_CreateSocketAndConnect: connect failed: -1

trying to connect:: Connection refused

1:armci_CreateSocketAndConnect: connect failed: -1

Last System Error Message from Task 1:: Connection refused

[compute-1-4.local:04739] MPI_ABORT invoked on rank 1 in communicator
MPI_COMM_WORLD with errorcode -1

3:trying connect to host=compute-1-4.local, port=35508 t=5 111

trying to connect:: Connection refused

3:armci_CreateSocketAndConnect: connect failed: -1

Last System Error Message from Task 3:: Connection refused

3:armci_CreateSocketAndConnect: connect failed: -1

[compute-1-4.local:04741] MPI_ABORT invoked on rank 3 in communicator
MPI_COMM_WORLD with errorcode -1

6:trying connect to host=compute-1-5.local, port=48920 t=5 111

10:trying connect to host=compute-1-6.local, port=36350 t=5 111

4:armci_CreateSocketAndConnect: connect failed: -1

4:trying connect to host=compute-1-5.local, port=48918 t=5 111

trying to connect:: Connection refused

4:armci_CreateSocketAndConnect: connect failed: -1

Last System Error Message from Task 4:: Connection refused

5:armci_CreateSocketAndConnect: connect failed: -1

5:trying connect to host=compute-1-5.local, port=48919 t=5 111

trying to connect:: Connection refused

5:armci_CreateSocketAndConnect: connect failed: -1

Last System Error Message from Task 5:: Connection refused

[compute-1-5.local:01175] MPI_ABORT invoked on rank 5 in communicator
MPI_COMM_WORLD with errorcode -1

6:armci_CreateSocketAndConnect: connect failed: -1

trying to connect:: Connection refused

6:armci_CreateSocketAndConnect: connect failed: -1

Last System Error Message from Task 6:: Connection refused

 

Is anybody familiar with this error?

 

Robert C. Jackson

Software Systems Specialist III

The University of Texas - Pan American

1201 W. University Dr.

Edinburg Texas 78539

Academic Computing Department

ASB 2.162E

956-381-2455 office 956-381-2355 fax

email: rjackson_at_[hidden]