On Mon, Dec 11, 2006 at 02:52:40PM -0500, Brock Palen wrote:
> On Dec 11, 2006, at 2:45 PM, Reese Faucette wrote:
> >> Also I have no idea what the memory window question is, i will
> >> look it up on google.
> >> aon075:~ root# dmesg | grep GM
> >> GM: gm_register_memory will be able to lock 96000 pages (375 MBytes)
> > This just answered it - there is 375MB available for GM to
> > register, which
> > is the IOMMU window size available to the GM driver. This is quite
> > small,
> > and it would be very helpful if this could be increased. helpful
> > == much
> > better performance for your jobs.
> Always a plus
> > It's possible that OMPI is not managing registered space well, as is
> > required when the aggregate registered memory needed by a job is
> > larger than
> > the memory available to be registered. I do some research locally
> > into
> > OMPIs management of registered memory when using GM. If it comes
> > to it,
> > would you be willing to run an OMPI with some debug statements in
> > it for me?
> > thanks,
> Sure i can run them.
I added to OMPI possibility to limit the amount of registered memory.
There is a new parameter mpool_rdma_rcache_size_limit that controls how
much memory can be pinned at once. But this work is not yet on the trunk.
If you can checkout /tmp/gleb-mpool branch and test it this would be
> > -reese
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> users mailing list