This web mail archive is frozen.
This page is part of a frozen web archive of this mailing list.
You can still navigate around this archive, but know that no new mails
have been added to it since July of 2016.
Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.
On Tue, 2006-11-28 at 09:28 -0800, Matt Leininger wrote:
> On Tue, 2006-11-28 at 10:00 -0700, Li-Ta Lo wrote:
> > On Mon, 2006-11-27 at 17:21 -0800, Matt Leininger wrote:
> > > On Mon, 2006-11-27 at 16:45 -0800, Matt Leininger wrote:
> > > > Has anyone testing OMPI's alltoall at > 2000 MPI tasks? I'm seeing each
> > > > MPI task eat up > 1GB of memory (just for OMPI - not the app).
> > >
> > > I gathered some more data using the alltoall benchmark in mpiBench.
> > > mpiBench is pretty smart about how large its buffers are. I set it to
> > > use <= 100MB.
> > >
> > > num nodes num MPI tasks system mem mpibench buffer mem
> > > 128 1024 1 GB 65 MB
> > > 160 1280 1.2 GB 82 MB
> > > 192 1536 1.4 GB 98 MB
> > > 224 1792 1.6 GB 57 MB
> > > 256 2048 1.6-1.8 GB < 100 MB
> > >
> > > The 256 node run was killed by the OOM for using too much memory. For
> > > all these tests the OMPI alltoall is using 1 GB or more of system
> > > memory. I know LANL is looking into optimized alltoall, but is anyone
> > > looking into the scalability of the memory footprint?
> > >
> > I am the one who is looking into those collective communications. Which
> > mca/coll are you using for alltoall?
> The ompi_info output had some mca/coll information in it. I'm not
> sure which mca/coll parameter you are interested in.
Could you try "mpirun -mca coll basic mpibench"?