Proposal by Jeff Squyres and Fab Tillier for user-level timers in MPI-3.

1. MPI Timers Tick tock tick tock

2. Wouldn’t it be nice… To block in MPI_WAIT* …but only for a while? (even if nothing completes) Useful for: Checking progress of non-MPI things Avoid putting MPI into its own progress thread Avoid using MPI_THREAD_MULTIPLE Dynamically updating array of requests to MPI_WAIT* MPI_TEST The missing middle MPI_WAIT Immediate Infinite

3. Why not generalized requests? Generalized requests are completed by the application calling MPI_GREQUEST_COMPLETE To “break out” of MPI_WAITANY, impl. must support: Multi-threaded applications and MPI_THREAD_MULTIPLE Or calling GREQUEST_COMPLETE from signal handler Thread 1 MPI_WAITANY (including a grequest) Causes generalized request to complete; WAITANY returns Threaded example Time  Thread 2 (or signal handler) MPI_GREQUEST_COMPLETE

4. New type of MPI_Request: timer MPI_TIMER_CREATE(double completion_time, MPI_Request *request) Creates a new timer request Request can be tested, waited, canceled, and freed – just like any other request Will complete when MPI_Wtime() >= completion_time Completed timer requests return the empty status

5. Use cases MPI_WAIT on a timer request Completes when MPI_Wtime() >= completion_time High quality implementations can block / progress MPI_WAITANY (including a timer request) Timer request may be the first to complete Forces a “premature” return from WAITANY MPI_WAITALL (including a timer request) Same as usual: completes when all complete Timer request sets a lower bound on when all complete Not enumerated here, but works with all MPI_WAIT* and MPI_TEST* functions – just like any other request

6. Use cases MPI_CANCEL a timer request Cancels just like any other request (i.e., optional)  Although we expect all MPI implementations to be able to implement canceling timers easily Still has to be tested or waited – just like any other request MPI_REQUEST_FREE a timer request Moral equivalent of a successful MPI_CANCEL followed by an MPI_TEST

7. Use cases: Dave/HP FT example Master Master Master MPI_Allgather P P P P P P P P P P P

8. Convenience Function MPI_Timer_reset(double completion_time, MPI_Request *request) Lets application re-use timer requests cheaply Any timer can be reset if it has not yet been completed via TEST or WAIT Specifically: it is ok if MPI_Wtime() >= completion_time See example on next slide…

9. Restarting timer in a loop MPI_Requestreq[21]; fill_20_requests(req); MPI_Timer_create(MPI_Wtime() + 5, &req[20]); while (1) { MPI_Waitany(21, req, &index, MPI_STATUS_IGNORE); if (index != 20) { go_handle_completed_request(req, index); check_for_other_progress(); MPI_Timer_restart(MPI_Wtime() + 5, &req[20]); } else { check_for_other_progress(); MPI_Timer_create(MPI_Wtime() + 5, &req[20]); } }

10. If the Forum likes the idea… What chapter should text about timers go in? Environment control (with WTIME, WTICK) Point to point (with TEST, WAIT, etc.) …? Optional ideas Timer request query function to find out when it will expired