Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] kernel 2.6.23 vs 2.6.24 - communication/wait times
From: Ralph Castain (rhc_at_[hidden])
Date: 2010-04-19 12:12:34


On Apr 19, 2010, at 9:12 AM, Oliver Geisler wrote:

>
>
>> Ah, that could do it. Open MPI's shared memory files are under /tmp. So if /tmp is NFS, you could get extremely high latencies because of dirty page writes out through NFS.
>>
>> You don't necessarily have to make /tmp disk-full -- if you just make OMPI's session directories go into a ramdisk instead of to NFS, that should also be sufficient.
>>
>
> I just browsed FAQ and "ompi_info --param all all", but didn't find the
> answer:
> How do I set the OMPI session directory to point it to a ramdisk?
>

Set the MCA param orte_tmpdir_base to point at the ramdisk using any of the MCA parameter methods (cmd line, envar, default mca param file).

> And another question:
> What would be a good size for the ram disk? One general value was
> supposed by the FAQ with 128MB, but what is your experience?
> (maybe a large topic by itself, so I have to try out, I guess)

I don't recall the default shared memory size per process, but you can get that value from ompi_info --param btl sm. Take that value, multiply by your expected ppn, and then give yourself a fudge factor.

>
> Thanks a lot.
>
> Oli
>
>
> --
> This message has been scanned for viruses and
> dangerous content by MailScanner, and is
> believed to be clean.
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel