Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] how to get mpirun to scale from 16 to 64 cores
From: Ralph Castain (rhc_at_[hidden])
Date: 2014-06-17 00:02:12


Well, yes and no. Besides the real question would be if this app, which this person didn't write, was written as a threaded application.

On Jun 16, 2014, at 8:32 PM, Zehan Cui <zehan.cui_at_[hidden]> wrote:

> Hi Yuping,
>
> Maybe using multi-threads inside a socket, and MPI among sockets is better choice for such NUMA platform.
>
> Multi-threads can exploit the benefit of share memory, and MPI can alleviate the cost of non-uniform memory access.
>
>
> regards,
> Zehan
>
>
>
>
> On Tue, Jun 17, 2014 at 6:19 AM, Yuping Sun <yupingpaulasun_at_[hidden]> wrote:
> Dear All:
>
> I bought a 64 core workstation and installed NASA fun3d with open mpi 1.6.5. Then I started to test run fun3d using 16, 32, 48 cores. However the performance of the fun3d run is bad. I got data below:
>
> the run command is (it is for 32 core as an example)
> mpiexec -np 32 --bysocket --bind-to-socket ~ysun/Codes/NASA/fun3d-12.3-66687/Mpi/FUN3D_90/nodet_mpi --time_timestep_loop --animation_freq -1 > screen.dump_bs30
>
> CPUs times iterations time/it
> 60 678s 30it 22.61s
> 48 702s 30it 23.40s
> 32 734s 30it 24.50s
> 16 894s 30it 29.80s
>
> You can see using 60 cores, to run 30 iteration, FUN3D will complete in 678 seconds, roughly 22.61 second per iteration.
>
> Using 16 cores, to run 30 iteration, FUN3D will complete in 894 seconds, roughly 29.8 seconds per iteration.
>
> the data above shows FUN3D run using mpirun does not scale at all! I used to run fun3d with mpirun on a 8 core WS, and it scales well.
> The same job to run on a linux cluster scales well.
>
> Would you all give me some advice to improve the performance loss when I increase the use of more cores, or how to run mpirun with proper options to get a linear scaling when using 16 to 32 to 48 cores?
>
> Thank you.
>
> Yuping
>
>
>
>
>
>
>
>
>
>
>
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24654.php
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> Subscription: http://www.open-mpi.org/mailman/listinfo.cgi/users
> Link to this post: http://www.open-mpi.org/community/lists/users/2014/06/24658.php