High Order Automatic Differentiation with MPI
Student: Mu Wang (Purdue University)
Supervisor: Alex Pothen (Purdue University)
Abstract: Automatic Differentiation (AD) is an algorithmic technique for augmenting computer programs to compute derivatives accurately and efficiently. AD has two major modes: forward and reverse. Previous work on parallel reverse mode AD algorithms mainly focus on first order. In those work, the communication is considered as a remote value assignment. Then the communication is also reversed when evaluating derivatives.
Here, we describe a new parallel Reverse mode algorithm to compute higher order derivative for an MPI program. The innovation is that in a parallel reverse mode AD algorithm, a Send/Recv pair should not be considered only as a remote value assignment, but as dummy independent/dependent variables. The innovation permits communication to be treated analogous to computation. Hence communication does not need to be reversed, but can be re-performed in a separate phase. So complications created by reversing the communications is avoided.
Two-page extended abstract: pdf