We have a diskless cluster with these specs:
1) A server which has some disks. Root directories (/usr, /lib, ...) are
on /dev/sda while /home is on /dev/sdb and these are two physical
2) Some compute nodes. These don't have any disk drive instead they
are connected through a 10/100/1000 switch to the server
3) Nodes uses a NFS directory for booting which resides on /dev/sda.
In our cluster we use openmpi with openfoam. Both were compiled using
default options. Problem is when the openfoam solver with openmpi is
sent to a compute node, the opnempi uses a lot of *WRITE* operations
which causes a low cpu utilization and hence the processes are mainly
in 'D' state. A brief description of our tests are:
We ssh to the compute node and run the application there
Test 1) One process of openfoam is launched without openmpi. Everything
is fine and cpu is utilized 100%
Test 2) Two processes of openfoam is launched (mpirun -np 2 ....). Two
openfoam processes has about 30% cpu utilization and they are in 'D'
state most of the time.
Is there any suggestion on that. That is a really poor performance