Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] How to time data transfers?
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-10-14 08:37:56


On Oct 13, 2010, at 4:52 PM, Michael E. Thomadakis wrote:

> Does MPI_Wtime of OMPI 1.4.3/1.5.0 rely on high resolution clocks (for Linux) or does still rely on gettimeofday() ? How would one request at OMPI built time to let it use high resolution clocks?

Check the man page for MPI_Wtime(3):

       On POSIX platforms, this function may utilize a timer that is cheaper
       to invoke than the gettimeofday() system call, but will fall back to
       gettimeofday() if a cheap high-resolution timer is not available. The
       ompi_info command can be consulted to see if Open MPI supports a native
       high-resolution timer on your platform; see the value for "MPI_WTIME
       support" (or "options:mpi-wtime" when viewing the parsable output). If
       this value is "native", a method that is likely to be cheaper than get-
       timeofday() will be used to obtain the time when MPI_Wtime is invoked.

IIRC, the problem on Linux is that the native x86 timers are a per-chip value (e.g., on a multi-socket system, the value is different on each socket). Hence, if you MPI_Wtime() while your MPI process is on socket A, but then the OS moves it to socket B and you call MPI_Wtime() again, the two values are completely unrelated.

That being said, I see clock_gettime() has CLOCK_REALTIME, which is supposed to be system-wide. Is it cheaper and/or more accurate than gettimeofday? I see that it has nanosecond precision in the man page, but what is it (typically) actually implemented as?

Additionally, I suppose that if a process is bound to a single native clock scope (I don't know offhand if it's per socket or per core -- I said "socket" up above, but that was a guess), we could make the *assumption* that the process will never move and might be able to use the native x86 timer (there's some complications here, but we might be able to figure it out). Bad Things could happen if the process ever moved, though (e.g., if the application ever manually changed the binding).

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/